NATO STANDARD

ADMP-02
GUIDANCE FOR DEPENDABILITY
IN-SERVICE
Edition B, Version 1

MAY 2022

NORTH ATLANTIC TREATY ORGANIZATION

ALLIED DEPENDABILITY MANAGEMENT PUBLICATION

Published by the
NATO STANDARDIZATION OFFICE (NSO)
© NATO/OTAN
INTENTIONALLY BLANK
INTENTIONALLY BLANK
 ADMP-02

RESERVED FOR NATIONAL LETTER OF PROMULGATION

I Edition B, Version 1
 ADMP-02

INTENTIONALLY BLANK

II Edition B, Version 1
 ADMP-02

TABLE OF CONTENTS

CHAPTER 1 INTRODUCTION....................................................................................... 1
1.1. GENERAL .................................................................................................... 1
1.2. PURPOSE .................................................................................................... 1
1.3. APPLICABILITY ........................................................................................... 2
1.4. NORMATIVE REFERENCES ...................................................................... 2
CHAPTER 2 CONCEPTS AND FACTORS ................................................................... 3
2.1. EVOLVING MAINTENANCE AND SUPPORT CONCEPTS ........................ 3
2.2. FACTORS AFFECTING IN-SERVICE DEPENDABILITY ............................ 3
2.3. OTHER FACTORS TO BE CONSIDERED FOR IN-SERVICE
DEPENDABILITY ....................................................................................................... 4
CHAPTER 3 IN-SERVICE DEPENDABILITY ................................................................ 6
3.1. GENERAL .................................................................................................... 6
3.2. MONITORING IN-SERVICE DEPENDABILITY ........................................... 6
3.2.1. OBJECTIVES ........................................................................................ 7
3.2.2. IN-SERVICE DEPENDABILITY CONCERNS ....................................... 7
3.3. DATA COLLECTION FOR DEPENDABILITY .............................................. 8
3.3.1. AIM AND OBJECTIVES ........................................................................ 8
3.3.2. DATA COLLECTION PROCESS .......................................................... 9
3.3.2.1. USAGE DATA .................................................................................. 9
3.3.2.2. FAILURE EVENT DATA ................................................................. 10
3.3.2.3. MAINTENANCE ACTION DATA .................................................... 11
3.3.3. DATA COLLECTION ISSUES............................................................. 11
3.4. DATA ANALYSIS FOR DEPENDABILITY ................................................. 12
3.5. MAKING DECISIONS ................................................................................ 13
3.5.1. GENERAL ........................................................................................... 13
3.5.2. CORRECTIVE MAINTENANCE ACTION OPTIONS .......................... 14
3.5.3. LIVING WITH DEPENDABILITY PROBLEMS .................................... 14
3.5.4. MAINTENANCE IMPROVEMENT ...................................................... 14
3.5.5. MODIFICATIONS AND UPGRADES ................................................. 15
3.5.6. LIFE EXTENSION ............................................................................... 15
3.5.7. REPLACEMENT ................................................................................. 16
3.5.7.1. LIFE-LIMITING FACTORS ............................................................. 16
3.5.7.2. DEVELOPING AN END-OF-LIFE STRATEGY .............................. 17
3.5.7.3. REPLACEMENT DECISION CRITERIA......................................... 18
CHAPTER 4 FAILURE ANALYSIS...............................................................................19
4.1. GENERAL .................................................................................................. 19
4.2. FAILURE REPORTING ANALYSIS & CORRECTIVE ACTION SYSTEM
(FRACAS)................................................................................................................. 19
4.3. FAILURE INVESTIGATION ....................................................................... 20
4.3.1. FAILURE ANALYSIS .......................................................................... 20
4.3.2. TECHNICAL DATA REVIEW .............................................................. 20
4.4. ROOT CAUSE ANALYSIS ......................................................................... 21

III Edition B, Version 1

 ADMP-02

INTENTIONALLY BLANK

IV Edition B, Version 1
 ADMP-02

CHAPTER 1 INTRODUCTION

1.1. GENERAL

1. Dependability is a key characteristic of all items1, having a direct impact on mission

performance and thus mission success. The dependability characteristics of any item are
inherent in its design, thus dependability should be considered from the very beginning of the
pre-concept stage and be continued, in a disciplined manner, throughout the whole life cycle
by the implementation of dependability disciplines as described in the IEC 60300 series
standards referenced at Section 1.4 in this document.

2. Dependability is the collective term describing the continued and safe operation of any
simple or complex item. The factors that influence the dependability performance of any item
are reliability, maintainability, availability, testability, maintenance, and safety. In most items
reliability and maintainability are the key performance characteristics of interest as they have
a direct impact on mission success and life cycle cost. The logistic and maintenance strategy
of the item are mainly external, but can have significant impact on its availability performance,
as it reflects the ability to provide the necessary resources to implement optimized
maintenance procedures developed and refined through the life cycle of the item.

3. When in-service, it is necessary to assure that the inherent levels of dependability

capability as described in the requirements documents are achieved while in use. It is
necessary as well to understand any impact of changes on Life Cycle Cost (LCC) while
meeting operational commitment over the life of the item.

4. The primary challenges are:

a. to be able to quickly identify and correct technical problems that cause levels of
dependability performance to deteriorate relative to requirements; and
b. to ensure dependability is appropriately factored into changes in design, support,
operating environment and procedures that will arise over an item's life cycle.

5. This requires a continuous process of collecting data from operations and maintenance,
analyzing the data to extract information about dependability performance, and when required,
making decisions for sustaining dependability performance and optimizing life cycle cost.
Depending on the provisions made for the in-service support, responsibility for conducting
some or all of these activities may be contracted out. In cases where a certain level of
dependability performance has been guaranteed, it is necessary to ensure that adequate
knowledge and visibility of support activities is retained so as to minimize disputes over
responsibility.

1.2. PURPOSE

The continuing assessment of in-service dependability performance is commercially and

operationally important, and enables the cost effective management of defence materiel
throughout its life cycle. The purpose of this document is to provide guidance on in-service
dependability. To achieve this, the following actions should be performed:

1 Item includes systems, equipment, be it hardware or software based, and services.

1 Edition B, Version 1
 ADMP-02

a. monitoring in-service performance

b. collecting data
c. analyzing data
d. finding and taking action

1.3. APPLICABILITY

This document applies to dependability activities of all items procured for military use within
NATO Nations when in-service. It should be used by all members of projects and in-service
organizations, including the various NATO Agencies, who are responsible for dependability.

1.4. NORMATIVE REFERENCES

A. ADMP-01 (B) Guidance for Developing Dependability Requirements

B. ADMP-03 (A) Guidance for Classification and Analysis of Dependability Events
C. AAP-20 (C) NATO Programme Management Framework (NATO Life Cycle Model)
D. AAP-48 (B) NATO System Life Cycle Processes
E. ALP-10 Guidance on Integrated Logistics Support for Multinational Armament
Programmes
F. ALCCP-01 NATO Guidance on Life Cycle Costs
G. IEC 60300-1:2014 Ed 3 Dependability management - Part 1: Guidance for management
and application
H. IEC 60300-3-2:2004 Ed 2 Dependability Management Part 3-2: Application Guide -
Collection of Dependability data from the field
I. IEC 60050-192:2015 Ed 1 International Electrotechnical Vocabulary – Part
192:Dependability

2 Edition B, Version 1
 ADMP-02

CHAPTER 2 CONCEPTS AND FACTORS

2.1. EVOLVING MAINTENANCE AND SUPPORT CONCEPTS

1. Since the early 2000s, Industry has become increasingly involved in the provision of
support services during the in-service life of an item. Performance-based contracts have been
implemented in which some or all support activities, including dependability activities, have
been contracted out.

2. The primary objective of these performance-based contracts is to establish a minimum

acceptable level of performance and support criteria for a particular item, and configure the
support of it to achieve this performance at an optimum cost. These support contracts often
link the acquisition contract with the long term support contract and may include incentives tied
to performance.

3. Performance can be defined in terms of military objectives using the following criteria:
a. Availability;
b. Reliability;
c. Cost Per Unit Usage;
d. Logistics Support Footprint;
e. Logistics Response Time.

2.2. FACTORS AFFECTING IN-SERVICE DEPENDABILITY

When in-service, a number of factors may affect dependability.

a. Age: Time can have a detrimental effect on dependability, since many materials degrade
as they get older. This includes electronic as well as mechanical systems, but does not
apply to software. Deterioration will occur whether an item is in use or not.

b. Use: The life of an item is very dependent on how it is used. This would have been
defined in the usage or mission profile, which is referred to in NATO documents as the
Life Cycle Environment Profile (LCEP) and is the term that will be used throughout the
remainder of this document. Any changes to the LCEP will have an impact on
dependability. Even without changing the original LCEP, all items have a finite life and
will gradually deteriorate through the normal process of wear, for example, distance
travelled, hours run or number of cycles. The effects of wear can be mitigated by
preventive maintenance.

c. Abuse: The accidental, negligent or deliberate abuse of an item can have a significant
effect on its dependability. Undue stresses caused by poor servicing, use outside the
design envelope, operational damage and inappropriate transportation or handling, will
accelerate deterioration and lead to premature failure. Although little can be done to
overcome accidental or deliberate damage, negligence can be minimized through well
managed servicing and storage, up to date documentation and sound training.

3 Edition B, Version 1
 ADMP-02

d. Repair: As an item gets older it may be subject to an increasing number of repairs. Every
repair will have a small but tangible effect on its integrity, and hence its subsequent
dependability. This is especially true of electronic circuit boards, which can only be
repaired a finite number of times. At appropriate intervals, item replacement should be
carried out.

e. Maintenance: Items that are maintained too often or not enough can induce a negative
impact on dependability. It is therefore imperative to review the periodicity and
effectiveness of the maintenance on a regular basis.

f. Obsolescence: All items become progressively obsolete as component parts and sub-
assemblies are superseded or discontinued. This is especially true for electronics and
COTS items.

g. Configuration: Most items with a long in-service life will be subject to successive
modification programs, to improve performance or overcome safety and obsolescence
issues. Unless this activity is strictly controlled, the modification state of individual items
and spares within a large fleet will tend to differ. Since certain combinations of
modifications could affect dependability, configuration management should always be
performed.

h. Upgrades: Any form of upgrade or improvement program should be managed with care,
since both hardware and software changes could affect dependability. The measurement
of performance parameters before and after upgrade will be very important to verify
successful implementation. Such opportunities should always be used to improve
dependability where possible.

i. Major Damage: Major accidents and battle damage can affect the subsequent
dependability of an item. Although an item may appear to be restored to full working
order, over stresses, distortion and intermittent faults may well be overlooked. Examples
might include heavy landings for aircraft, battle damages of vehicles and collisions for
ships. Any item suffering significant damage should be clearly identified and treated as
a special case when analyzing dependability data.

j. Training: Training for Users and Maintainers must keep pace with changes to item and
procedures. External validation to ensure training remains up to date and effective should
be carried out.

2.3. OTHER FACTORS TO BE CONSIDERED FOR IN-SERVICE DEPENDABILITY

The following factors should be considered as well:

a. Boundaries: Whatever the nature of an item and the extent of a performance-based

contract, there will be an interface between the User and Supplier. The nature of this
boundary may change between peacetime activities, operations and war. Serious
consideration must be given to the functions carried out by the User and the Supplier on
either side of the interface. If support is to be effective it is essential that these
boundaries and responsibilities are clearly defined in the contract.

b. Dispute Resolution: If a performance-based contract is not fully inclusive, it may be

necessary to review incidents, to determine whether rectification lies within the scope of

4 Edition B, Version 1
 ADMP-02

the contract. Most are unlikely to be contentious; however, a proportion may need to be
formally resolved under a dispute resolution process.

c. Partnership: However well written a performance-based contract may be, issues will
inevitably arise which have not been foreseen. Therefore, a good working relationship,
based on honesty and trust between User and Supplier is essential if difficulties are to
be avoided or resolved by mutual consent.

d. Any item supported by an in-service performance-based contract should seek to sustain

or enhance the levels of dependability achieved during development. Contracts should
be flexible, with well-defined boundaries and include transition arrangements in case of
termination or transfer.

5 Edition B, Version 1
 ADMP-02

CHAPTER 3 IN-SERVICE DEPENDABILITY

3.1. GENERAL

Managing in-service dependability requires the monitoring of performance through a

continuous process of collecting data from operations and maintenance, analyzing the data to
extract information about significant events and trends in dependability performance. The
processed data is used to make decisions on optimizing the item or its support, taking action
when required. It can be costly and time consuming to achieve this; however, failing to do so
can be more costly and lead to a decrease in mission success and an increase in Life Cycle
Cost. It is therefore essential that the objectives and processes are identified well in advance
and arrangements made to meet them in an effective manner. These should be captured in
an in-service dependability plan and implemented throughout the useful life of the item. The
process for managing in-service dependability is shown in Figure 1.

FIGURE 1: In-Service Dependability Management Process

3.2. MONITORING IN-SERVICE DEPENDABILITY

1. Monitoring in-service dependability begins with an appreciation of the range of

dependability performance issues that are relevant to in-service materiel management. It is
critical that the process for monitoring of dependability data is considered early during
acquisition and designed into the item rather than reverse-engineering it when in-service.

2. The purpose of monitoring in-service dependability is to assess the adequacy of support

processes on a continuous basis, so that areas of concern or potentially detrimental trends
can be identified early. This permits action to be taken before problems begin to seriously
impact operations and/or expenditures. Effective parameters such as output/capacity,
availability, reliability, spares and operating & maintenance cost are all candidates for
performance monitoring.

6 Edition B, Version 1
 ADMP-02

3. The performance monitoring philosophy makes use of standard data sources already
available for other purposes and taps into them for performance monitoring purposes. The
performance monitoring should generate exception reports, which highlight the occurrence of
any anomalies or trends that occur.

3.2.1. OBJECTIVES

The objectives of in-service dependability activity may vary between items and their different
roles, but will generally include one or more of the following:
a. To quantify the achieved dependability.
b. To demonstrate compliance with specified dependability requirements.
c. To identify the factors inhibiting an item from achieving the specified levels of
dependability.
d. To validate Life Cycle Environment Profile assumptions used in the predicted
dependability studies.
e. To improve dependability. This could include incentives, penalties, or the establishment
of additional dependability goals in a cost effective manner with the use of specialized
warranty arrangements if applicable.
f. To assess the requirements and capabilities of technologies which are new to the military
services and to provide information for the user and for future acquisitions.
g. To assess the effectiveness of maintenance procedures and built-in equipment,
including the training of personnel engaged in such tasks, in the identification of failures.
h. To assess the remaining useful life of the item.
i. To use the observed dependability results to provide inputs to other functions such as
obsolescence management, business planning and configuration management.

3.2.2. IN-SERVICE DEPENDABILITY CONCERNS

1. Understanding the materiel management issues related to in-service dependability

performance is the vital first step in establishing a monitoring program. It provides the focus
for the dependability data collection and analysis program by defining the key questions to be
answered.

2. The identification of deficiencies in dependability performance: Dependability

performance is of great concern to the users, and a proactive approach to identifying
deficiencies will be invaluable in keeping them satisfied with their items. Examples of
dependability performance deficiencies include item reliability degradation (higher failure rates,
unexpected failure modes, premature wearout and so on), excessive maintenance costs (for
parts, manpower, and other resources), high false alarm rates from automated test equipment,
and low system availability.

3. Continuous flow of routine performance data: Support planning needs to be updated to

reflect current realities. The support for a new item is initially designed using predictions
developed in the design process that are often based on generic data. Information on actual
field performance is needed so that adjustment can be made to the support system in time to
avoid resource shortages or over-investment in inventory. This continues through the life of

7 Edition B, Version 1
 ADMP-02

the item. To meet these needs, information related to issues such as part consumption,
support equipment and facility utilization, and the projected service life of the item is required.

4. Maintenance planning: Maintenance plans developed during the acquisition process

must be validated for their applicability and effectiveness in the real world, and rationalized
where necessary to reflect experience gained during the in-service period. This will involve
such issues as monitoring rates of maintenance-induced failures for evidence of shortfalls in
training or procedures, reviewing component life limits in light of growing experience, and
evaluating the effectiveness of maintenance strategies, including opportunities to benefit from
new predictive maintenance and diagnostic techniques.

5. Dependability performance data: This is required to support decisions on how best to

resolve the problems that have been identified. The system evolution process encompasses
modification/upgrade, service life extension and replacement choices. A good foundation of
historical dependability data can be useful to ensure that proposed solutions address all the
right problems, and to evaluate the likely impact of decisions on dependability performance
and life cycle cost.

6. While the day-to-day issues of materiel management have a way of dominating the
focus, it is also important to prepare for the future. Data on in-service dependability
performance can play an important role in shaping the concept and acquisition of future
acquisition programmes. It can provide performance targets for technical specifications and
identify important “lessons learned”.

3.3. DATA COLLECTION FOR DEPENDABILITY

3.3.1. AIM AND OBJECTIVES

1. The aim of data collection is to generate information to make informed decisions which
could lead to improvement of items and processes in any organization. Collected data with
appropriate analysis close the learning loop back to design, manufacturing and service. Sub-
targets can be risk minimization, cost optimization or the check for conformity with given
requirements. Data should be collected for a purpose: to enable analysis, focused on
increasing understanding of item operation and failure, and application of this knowledge to a
goal or objective. Without a definition of the objective for the future data analysis and the
application of its findings, collection of data is likely to be aimless and will omit important data,
allow corruption of data, or may waste time and resources by including data that offer little
benefit.

2. While planning data collection during the acquisition phase, several questions have to
be considered to determine dependability requirements. It is important to remember that the
underlying reason for performing data collection as a dependability task is to improve product
quality, monitor performance, modify support, to determine if required reliability is achieved,
identify deficiencies for root cause analysis leading to product improvement by modification, to
improve performance and, in the longer term, to improve quality of service.

3. This aim leads to the need to understand all the costs associated with a particular
project. These costs are known as the life cycle costs and include all costs involved in the
design, manufacture, use and disposal of an item. Data collection plays a part in the
identification of these costs since it allows management to make assessments of such things
as value-for-money, cost effectiveness, and life-cycle cost.

8 Edition B, Version 1
 ADMP-02

3.3.2. DATA COLLECTION PROCESS

1. A comprehensive and accurate dependability data collection process is the key to

managing in-service dependability performance. Data is the raw materials that allow informed
decisions to be made regarding all of the dependability issues discussed above. Collecting
the right data, and collecting it consistently over time, is the foundation of a good dependability
performance monitoring program. Incomplete, inaccurate or sporadically-collected information
generates more questions than answers, and is probably the biggest reason that many in-
service dependability monitoring programs are abandoned in frustration.

2. Data collection to enable dependability analysis needs to record three basic types of
data:
a. Usage data describes how much the item has been used, and under what
conditions the usage occurred. It provides a context for the failure and
maintenance data.
b. Failure event data captures the details of each failure - what happened, when and
where it happened, why/how it happened and what impact it had.
c. Maintenance action data describes all corrective (CM) and preventive (PM)
maintenance actions performed - what was done, what resources (time and
materiel) were consumed and how satisfactory were the results.

3. If the User or Supplier already has a proven data collection system, then there is no need
for a dedicated system for collecting dependability data.

3.3.2.1. USAGE DATA

1. Forming a coherent picture of dependability performance is only possible when

comprehensive data on usage is available to put events in context. For example, it is important
to see whether any special factors (such as unusual operating conditions or a change in
mission/role) may have had an effect on operation. All usage data should be recorded.
Considering only the items that have failed can skew the results in an unreasonably pessimistic
way.

2. Usage data is usually collected electronically through the maintenance transactions.

From these maintenance transactions, reports can provide historical record of the usage and
events as defined by the user. Some of the most important information to be collected
includes:
a. Operating Time/Usage: the accumulated operating time (or other usage measure)
should be recorded at regular intervals (this is used to establish usage patterns
over time and across the fleet). Any actions, which alter operating time records
(for example, odometer replacement), should be captured.
b. Availability Related Information: item status (up, standby or downtime, with
reasons for downtime) should be recorded to allow the time spent in each category
to be computed.
c. Events: the occurrence of all events (for example, failures, maintenance actions,
overhauls and inspections) should be recorded (with a cross-reference to the
associated reports). Major changes to the operating role (for example, transfer
from operations to training) or location (for example, deployment to the Arctic)

9 Edition B, Version 1
 ADMP-02

should also be recorded, to provide historical information about the operation

experienced over time.
d. Environmental data: Environmental data such as temperature, humidity and
shocks should be recorded to provide historical information about the
environments experienced over time.
e. Configuration: changes in the configuration status (for example, incorporation of
permanent or temporary modifications/upgrades) should be recorded to identify
the physical status of the item.

f. Unique Identification: If an item has a unique identification number, it should be

recorded. This data can be used to establish the history of the items when
analyzing its usage.

3.3.2.2. FAILURE EVENT DATA

1. Meaningful analysis of failure events demands as much qualitative and quantitative

information as possible about each event, including the circumstances surrounding its
occurrence, the effects that were observed and any cause that was eventually established.
The information will be needed when grouping them for analysis, when looking for special
factors that might have contributed to them, and when assessing the effectiveness of
preventive maintenance strategies. It is important to record data on all events, including events
where no fault was found, or where maintenance was performed quickly and without
consuming parts. Such events often provide important clues about subsequent failures.

2. Failure data are often collected on a dedicated failure report, but failure and maintenance
data may equally well be combined in a single report. Failure data are best captured by
someone who observed or discovered the failure, but the cause of the failure should be verified
by maintenance or other technical personnel. Some of the important information to be
captured on each failure event includes:
a. Description: a brief description of the failure should be provided to summarize the
event (this will often be the first piece of information examined by anyone looking
for failure patterns, so it needs to be clear and concise), along with a cross-
reference to any related failure events (such as a primary failure that caused this
failure event, or secondary failures that were triggered by the failure event).
b. Identification: the name and identification/registration number of the major system
(for example, vehicle, aircraft, ship, etc.) sustaining the failure must be recorded.
c. Time of Occurrence: the date and time of the failure must be recorded (indicating
whether this represents the time that the failure occurred, or the time it was
discovered), as well as the accumulated system operating time (including all
clocks/meters related to usage of the failed item). This is necessary for ordering
failures chronologically or by age at failure, and placing them in a historical context
in terms of system usage.
d. Conditions Prevalent at the Time: the major operating and environmental
conditions at the time of failure should be recorded, emphasizing anything that
might have put significant/unusual stress on the failed item.
e. Detection Methods: the manner in which failure was detected should be captured
for use in validating preventive maintenance strategies and refining diagnostic
methods.

10 Edition B, Version 1
 ADMP-02

f. Effects of Occurrence: the effects of failure on the failed item and on system
performance should be recorded to permit assessment of failure severity/criticality
and to validate RCM analysis.
g. Root Cause: the cause of the failure (including attribution to hardware, software,
operator error, maintainer error, accident, etc.) should be recorded, including the
initial impressions or suspicions of Users and first line maintenance personnel, as
well as any confirmation provided by failure investigations or further maintenance
(“cannot duplicate”, “no fault found” or “re-test ok” findings should be included).
This is vital when grouping failures for analysis.
h. Recovery: the action taken to remedy the failure should be captured, including any
initial steps taken by Users to restore mission capability, as well as maintenance
performed to return the item to a serviceable state.
i. Additional Data: some types of failures, such as software failures, may require the
capture of additional information, perhaps requiring downloading of this information
to assist in troubleshooting and problem isolation.

3.3.2.3. MAINTENANCE ACTION DATA

Maintenance action data describes all corrective (CM) and preventive (PM) maintenance
actions performed. Some of the important information to be captured on each maintenance
action includes:
a. Task Identification: the maintenance task must be described briefly, along with the
reason for performing it (for example, a reference to a failure report for a corrective
maintenance action, or to a preventive maintenance schedule for preventive
maintenance).
b. Task Time: the duration of the maintenance should be recorded, along with the calendar
date and time that the action started and finished. The source and duration of any delays
encountered before or during the work should also be identified.
c. Labor Consumed: the maintenance manpower required to perform the task should be
identified by maintainer, including trade and skill level as well as labors expended.
d. Parts Consumed: the identity and quantity of parts consumed by the task should be
listed, and the disposition of repairable items should be recorded. If a spare or repair part
has a unique identification, the identity of the parts removed and replaced should be
recorded.
e. Problems: any problems with maintenance resources (for example, incorrect diagnostic
or maintenance procedures, inadequate tools or test equipment, defective spare or repair
parts, or insufficient training) should be recorded

3.3.3. DATA COLLECTION ISSUES

1. There are several general principles that can contribute greatly to the success of a
dependability data collection system, all of them focusing on data quality. Without good data
quality, dependability performance analyses will produce misleading results (if they can
provide anything at all), which can lead to unjustified complacency or unwarranted concern.
Either way, the wrong course of action will be suggested.

11 Edition B, Version 1
 ADMP-02

2. Simple transactions, quick to complete, are an important way to ensure the timely
collection of data. Each question should be clear, even to a new user. Pre-defined check
boxes and codes could be used to minimize data entry wherever is possible. Over-collecting
data is a burden to the user, it leads to poor data quality as they become frustrated with
questions that are not applicable to their situation, or require more information than they have.
Data quality is more important than data quantity.

3. Every effort should be made to avoid the duplication of data entry. For example, if a
maintenance report is also used for work authorization and for ordering parts, it will streamline
the workload rather than add to it.

4. Data collected electronically needs to consider bandwidth requirements, security issues,

storage issues and so on.

5. There will always be some degree of variation in the quality of data collection -- between
units, between users, and for the same user at different times (depending on workload, attitude
and training). The data collection process should be routinely audited to assess data quality
and point out areas that need improvement or reinforcement, so that problems can be
addressed before the system is irretrievably corrupted with unreliable data.

3.4. DATA ANALYSIS FOR DEPENDABILITY

1. Whether data is used for routine dependability performance monitoring or for more
focused investigations, it needs to be reviewed for completeness and classified according to a
logical and structured process. A method for executing this process is described in ADMP-03.
Implementing this process is how the “collected data” becomes “information” that can be used
to inform the decision making process.

2. There are many types of information that can be extracted from data, and a
corresponding selection of analysis and presentation methods. Data analysis can help to
identify the major contributors to a problem (for example, what is causing most of these
failures?). It can also reveal which factors have a significant impact on a problem (for example,
will controlling or compensating for this factor help to fix the problem?). Data analysis over
time helps to detect deviations or anomalies that should be investigated (for example, has
something just happened that needs attention/explanation?). It also identifies trends over time
(for example, is this process undergoing a permanent change from its usual state?). Relatively
simple analysis methods include Pareto analysis, stratification, scatter diagrams, and control
charting. Information on additional advanced techniques (such as Analysis of Variance) can
be found in texts on statistics and quality assurance. Dependability data can be used with
techniques such as Root Cause Analysis (RCA), Reliability Centered Maintenance Analysis
(RCM), and Condition Based Maintenance (CBM). A variety of statistical analysis software
packages are available to automate these analyses. The International Electrotechnical
Commission (IEC) offers several publications (through the Technical Committee (TC-56)
publications standards on Dependability Management) that directly explain how to use these
techniques.

3. A wide range of dependability measures are available for application, but selections will
vary from project to project, depending on the type and function of the item, the needs of the
users and the available data sources. At the top level, these metrics can be grouped under
the typical measures under dependability management such as reliability, maintainability,
availability, testability, maintenance, and safety, and more details can be found in ADMP-01.

12 Edition B, Version 1
 ADMP-02

3.5. MAKING DECISIONS

3.5.1. GENERAL

1. Once a dependability problem has been identified in analysis, the next step is
determining if anything can be done to improve the situation. Decisions will seek to satisfy
operational effectiveness requirements and maximize cost effectiveness over the remaining
service life.

2. Figure 2 below shows a process for identifying which dependability improvements are
worth including in a program.

FIGURE 2: Dependability Improvement Decision Process

3. Any change provides a chance to improve performance, and also a risk of inadvertently
damaging it. Careful project management fosters the improvements and minimizes the
penalties. From a dependability perspective, all of the considerations encountered during the
item design come into play again. Every change must be evaluated for its impact on
dependability performance. However, there is a benefit to working on a mature, fielded system.
Performance monitoring records and other dependability performance indicators can provide
a list of the biggest shortcomings faced by users under actual field conditions.

13 Edition B, Version 1
 ADMP-02

3.5.2. CORRECTIVE MAINTENANCE ACTION OPTIONS

1. Corrective maintenance action options, such as better preventive maintenance,

modification to design and process, overhaul, life extension or replacement will depend on
what is technologically feasible or cost effective;

2. An improvement in the maintenance program may reduce exposure to the

consequences of the item failure, or decrease maintenance costs without jeopardizing
dependability. A modification or upgrade to the item may improve dependability performance
by fixing major problems. A life extension program may renew the item, postponing the
problems of wear out until some point in the future. Replacement of the item might eliminate
dependability problems associated with old technology and poor design, allowing a fresh start
to be made. However, it is not always possible to fix the problem and the user must simply live
with it.

3.5.3. LIVING WITH DEPENDABILITY PROBLEMS

1. In some cases, a dependability performance problem may be identified but no viable

options exist for fixing it. This may happen when a technical solution cannot be identified, or
the necessary fix is not cost-effective. Or perhaps there is no money available to perform a
fix, no matter how cost-effective it is. It may also happen towards the end of the item’s life,
when a decision on modification or replacement has already been taken.

2. The person responsible can still take useful actions to ease the pain of living with the
problem. The most important step is to alert people to the existence of the problem, so that
resources can be adjusted to meet any increased demand in time that may be required to
avoid a negative impact on operations. Increased quantities of spare and repair parts may be
required. Repair and Overhaul (R&O) facility may have to prepare for increased throughput.
Turn-around time for spares procurement and R&O may have to be shortened. More
maintenance resources (personnel, tools, test equipment, facilities) may be needed to cope
with increased workloads. Educating users and maintainers about the problem may allow
them to identify work-arounds (for example, conscientiously avoiding “rough treatment” or
heavy use that might trigger the problem). Increased efforts to balance usage rates across the
fleet may postpone the onset of age-related problems.

3. It may also be possible to minimize exposure to a dependability problem by formally

restricting the operational use of the item. For example, if operating in particularly cold weather
is known to cause the problem, it may be possible to avoid using the item in such situations
unless it is absolutely necessary.

4. Finally, gathering data and documenting the problem is a proactive step to prepare for a
day when a fix may be possible. Having the information available will make it easier to
influence the specifications for a future modification or replacement program, to make sure that
the problem does not recur.

3.5.4. MAINTENANCE IMPROVEMENT

1. The maintenance program for any item should be expected to evolve during its life as
more experience is gained with its operation, reliability and failure modes. In-service events
may force a change in the maintenance program when it becomes evident that the causes
and/or consequences of some failure mode have been misunderstood. For example, a

14 Edition B, Version 1
 ADMP-02

maintenance review should be conducted immediately after the first occurrence of a safety
failure, to see whether it was just “bad luck” (random variation), or whether the maintenance
strategy failed to reduce the probability of failure to an acceptable level. Similarly, the
persistent occurrence of any failure modes that were supposed to be covered by a preventive
maintenance task should cause a review of the task’s effectiveness (Does condition monitoring
or inspection provide adequate warning of failure? Is the scheduled overhaul or replacement
interval too long?). The discovery that a functional failure is actually hidden from the users
should also cause a review of the Reliability Centered Maintenance Analysis (RCM) for that
failure mode.

2. Analyzing the effectiveness of preventive maintenance tasks may reveal an opportunity

for improvement. If a scheduled inspection is rarely finding anything wrong, it may be time to
increase the inspection interval. If teardown of items removed for scheduled rework shows
significant life remaining, an increase in the life of the item may be justified. If failure reports
indicate a high incidence of maintenance-induced failures on items that are being preventively-
maintained, it may be that preventive maintenance is doing more harm than good. A different
strategy (or perhaps better training of maintenance personnel) may be called for.

3. Analysis of preventive maintenance tasks may reveal that costs could be reduced by a
change in policy, based on actual experience with failure rates, maintenance costs and failure
costs.

4. Technological change can cause a review of preventive maintenance strategies. A new

inspection method may be capable of detecting potential failures significantly sooner,
permitting the inspection frequency to be decreased. Condition monitoring may become more
cost-effective, allowing predictive maintenance to replace time-based inspection strategies.
Newer versions of interchangeable replacement parts may provide longer life. Modifications
or upgrades may also provide an avenue to improve preventive maintenance methods.

3.5.5. MODIFICATIONS AND UPGRADES

1. Many items go through at least one significant modification or upgrade during their
service life. There are many reasons to contemplate such changes (for example, to boost
performance, or to maintain compatibility with newer item), and not all of them relate to
dependability performance. Whether or not dependability drives the initiation of the work, it
should always be a major factor in the planning; however, at times external factors can mean
that dependability is not the primary driver (for example, national/international legislations) and
can decrease.

2. The important idea about any modification is that it represents a “second chance”, an
opportunity to fix problems with the benefit of hindsight, while doing the job right on new areas
of design. There is a danger, however, in making improvements to long-standing deficiencies
simply because it has become possible. Instead, a disciplined approach for justifying
dependability improvements on a cost-benefit basis is appropriate. Any redesign effort will
involve some degree of expense, and it is important to identify a payoff that will offset the costs.

3.5.6. LIFE EXTENSION

1. Life extension involves overhauling an item to renew its service life. It may also include
modifications or upgrades to cope with changing operational requirements, or to postpone

15 Edition B, Version 1
 ADMP-02

obsolescence. A life extension program may be an attractive alternative to replacement for

reasons of cost, schedule and/or performance.

2. If service life comes to an end more quickly than expected, life extension may be the
only choice available in the short term, until a replacement item can be developed or acquired.
Even if replacement options are available, it may be useful to allow some time for assessing
the merits and risks of each one, or to wait for a promising new alternative to prove itself.

3. Life extension may be the most cost-effective way to renew an item. If life is being limited
by a small portion of an item’s components, then an overhaul may restore service life relatively
cheaply. Similarly, life extension may be the best option to fill a role that will be phased out
over the medium term (rather than acquire a new item that will only be used for a time).

4. If an item is still quite capable of meeting all relevant performance requirements, then a
life extension program may be the best way to retain that capability. The considerable
overhead and disruption caused by acquiring and fielding a new item is difficult to justify if the
old one can be renewed to do the job.

3.5.7. REPLACEMENT

Every item reaches the end of its service life at some point, and it becomes undesirable
(perhaps even impossible) to keep the system operating. Service life may be ending when the
item can no longer deliver the required level of performance, in terms of either cost or benefit.
Its operating and maintenance costs may be rising to unacceptable levels. The quality and/or
quantity of its output may have dropped to a point where it can no longer meet the operational
requirement that it is supposed to fulfil.

3.5.7.1. LIFE-LIMITING FACTORS

1. For most items, several factors are competing to bring service life to a close, either
sharply or gradually. From a dependability perspective, wearout is the life-limiting factor of
most concern.

2. Wearout is an inevitable process of degradation, due to mechanical and electro-chemical

processes, that renders an item increasingly incapable of performing its function. Maintenance
can counteract the initial effects of wearout to some degree, but the process is relentless. The
item will finally reach the end of its durable life, at which point no reasonable amount of
maintenance can restore it to a serviceable state. Examples of such conditions include fatigue
cracking of major structural components, or widespread deterioration of wiring harnesses that
are “built into” a system such as a ship or aircraft.

3. Overstress occurs when the user demands more performance from an item than it was
designed to deliver. Overstress is not inevitable, but it is an increasingly likely occurrence for
older items. The user’s needs often change as time goes on resulting in a demand for the
item to operate at higher speeds or handle heavier loads. Understanding the impact of the
overstress of the system is paramount as it can significantly reduce the life of the system.

4. Defects can also bring an item to the end of its life. When major design or manufacturing
flaws are discovered after an item has been fielded, it may be fundamentally unable to perform
the function that it was designed for. No amount of maintenance will help to restore a capability
that never existed, and the only available option may be to start again almost from scratch.

16 Edition B, Version 1
 ADMP-02

For example, if the steel used to build an item was of insufficient strength, the item may be
completely unable to handle its planned operating loads. A faulty thermal design might make
an item’s electronic circuits prone to fatal overheating under some operating conditions.

5. Obsolescence ends an item’s service life when it can no longer function in the current
technological and support environment. If an item cannot interface or communicate with newer
systems that it needs to work with, then it has reached a point of technological obsolescence.
For example, a computer that cannot connect to the new network or run the latest software
may no longer be of use, depending on the user’s needs. When an item can no longer be
supported because spare and repair parts or consumables are unobtainable, or because the
necessary maintenance skills are no longer in ready supply, then its service life is over just as
effectively. Obsolescence is as much of a problem for software as for hardware.

6. It is important to understand in advance which factors will effectively limit the life of an
item, and to determine when that limit will be reached. This is necessary so that a strategy for
item renewal or replacement can be put in place before the end of service life arrives. It is too
late to begin procuring a complex new item on the day that the old item crashes irretrievably.

3.5.7.2. DEVELOPING AN END-OF-LIFE STRATEGY

Planning for the end of service life must be proactive. Figure 3 below illustrates the general
process of identifying how and when service life will end, then evaluating replacement or
renewal all options to determine the best choice. Whatever decision is reached, the remaining
life of the item can now be managed appropriately. For example, expensive repairs or
modifications can be avoided if they will not provide an adequate pay-off before the end of
service life.

FIGURE 3: Developing an End-of-Life Strategy

17 Edition B, Version 1
 ADMP-02

3.5.7.3. REPLACEMENT DECISION CRITERIA

1. Notwithstanding the possibility of life extension, most items will eventually have to be
replaced. The decision on exactly when to replace an item is often driven by dependability
considerations, such as increasing maintenance costs, falling levels of item effectiveness and
growing risk of unacceptable failure consequences. The common factor in all of these criteria
is that they are continuous measures. For example, there is no definitive point at which an
item is “too expensive to maintain”, only an arbitrary threshold established to guide decision-
making, or a point at which alternatives are perceived to be more cost-effective. The role of
dependability data in replacement decisions is to define current and expected levels of
reliability, maintainability and availability for an existing item and its potential replacements, so
that an informed choice can be made.

2. With dependability performance predictions available, better informed replacement

decisions can be made. There are three main types of criteria generally used to make such
decisions. Optimization criteria seek the solution that will minimize costs or maximize benefits.
Acceptability criteria define performance thresholds that must not be crossed if operational
requirements are to be met. Obsolescence criteria ensure that new performance requirements
and technological change are taken into account. It is appropriate to blend these criteria by
seeking the optimal replacement policy which meets all relevant acceptability and
obsolescence criteria.

3. Acceptability criteria are often used as the trigger for a replacement decision, defining
the worst level of performance that is tolerable from an item. Replacement may be required
when an item cannot deliver a minimum level or quality of performance, when it is down too
often, when it costs too much to operate and maintain, or when a low level of safety or
dependability presents an unacceptable risk.

4. Dependability information can help to forecast when an item is likely to breach an

acceptability threshold (such as a maximum downtime or minimum dependability requirement).
An accurate forecast will allow time to plan for replacement action, rather than waiting until
performance has actually reached unacceptable levels and operations (or budgets, or safety)
begin to suffer.

5. In addition to meeting acceptability criteria, the best replacement decision will optimize
cost (or cost per unit of benefit) over an appropriate planning horizon.

18 Edition B, Version 1
 ADMP-02

CHAPTER 4 FAILURE ANALYSIS

4.1. GENERAL

Failures are obviously undesirable events, but they are also useful. They provide important
information about an item’s weaknesses, so every failure becomes an opportunity for
improvement. Taking advantage of these opportunities requires a rigorous approach to the
failure analysis process, so that none of the potential insights provided by a failure are wasted.

4.2. FAILURE REPORTING ANALYSIS & CORRECTIVE ACTION SYSTEM (FRACAS)

1. A closed-loop Failure Reporting and Corrective Action System (FRACAS) promotes a

thorough approach to failure analysis. It ensures that the valuable information provided by
failures is captured for input into any subsequent failure investigations. Failure investigation is
a continuous systematic process of physical failure analysis, failure data review and root cause
analysis to understand why a failure occurred. The investigation results provide the foundation
for taking appropriate corrective action. (See Figure 4).

FIGURE 4: Failure Reporting, Analysis & Corrective Action System (FRACAS)

2. A FRACAS should normally be set up during the design and development phase of a
program, to support the “test, analyse and fix” process. It continues to operate during
production, to deal with failures identified in the factory and during installation. By the end of
this period, the FRACAS may contain a large volume of failure data, failure analysis reports
and information about failure patterns.

19 Edition B, Version 1
 ADMP-02

3. Although its focus is rather different, FRACAS should continue in-service. Changes to
product designs and manufacturing processes are less likely to be possible, so corrective
actions will take other forms. The existing maintenance management information system can
be used for the failure reporting function.

4.3. FAILURE INVESTIGATION

1. The failure investigation process begins with the identification of a problem, in the form
of a failure (or failures). Its objective is to determine the root cause of the problem, so that
appropriate corrective actions can be identified. It is important not to jump to conclusions about
the root cause. Labelling and fixing the wrong thing will only create a false sense of confidence.

2. The top level of the failure investigation process is straightforward. First, the failed item
is examined to confirm that it has truly failed, and that there is a problem worth investigating
(this avoids wasting time on failure analysis when the problem lies with a faulty maintenance
diagnosis). Next a physical failure analysis is performed to establish exactly how the item
failed. At the same time, a review of technical data is conducted to provide further information
about the failure and its possible causes. Finally, the results of the physical failure analysis
and the data review are used as inputs to a root cause analysis, which determines why the
failure occurred and how far its impact might extend. This knowledge forms the basis for
proposing corrective actions, and ends the failure investigation process.

4.3.1. FAILURE ANALYSIS

The failure analysis examines the failed item to establish the mode, mechanism and proximate
(immediate) cause of its failure. The analysis may use any of a number of techniques,
depending on the technology and materials used in the item, the nature of the failure, and the
resources available.

4.3.2. TECHNICAL DATA REVIEW

1. The data search, which proceeds concurrently with the failure analysis, has two
purposes. First, it looks for information that may help to direct the failure analysis. This
includes technical data which define the intended performance of the failed item, and describe
the parts, materials and manufacturing processes used in its construction. It also includes
failure reports, operational alerts or other technical literature relating to similar failures, which
may indicate that particular inspections and tests would be useful.

2. Second, the data search looks for information that could help to identify the root cause
of the failure. This involves an examination of failure reports to identify any patterns of similar
failures. It also includes a review of historical data related to the failed item (such as
manufacturing records, quality records, acceptance test reports, in-service operating logs and
maintenance reports). It may also include a review of technical manuals that describe the
intended operation and maintenance procedures for the failed item.

20 Edition B, Version 1
 ADMP-02

4.4. ROOT CAUSE ANALYSIS

1. Once the proximate cause of the item’s failure has been established, further investigation
may be needed to determine the root cause, and to determine whether any other items might
be vulnerable to the same root cause. Root causes can be found in many places and take
many forms. Typical sources of root causes which should be considered in any analysis are
shown below:
a. Design
b. Components & Material
c. Manufacturing
d. Overstress
e. Maintenance
f. Wearout

2. In some cases, the root cause will be quite apparent and the corrective action obvious.
In other cases, the root cause will be more difficult to pinpoint. Additional analyses may be
required, and more failures may have to occur before enough of a pattern emerges to justify a
conclusion.

21 Edition B, Version 1
ADMP-02(B)(1)