Guidelines for Validation & Verification, including Change Control

Introduction
This is a general guideline aimed at providing RGL with a practical framework for the
introduction or use of any new or change of established processes, equipment, facilities or
systems. The following areas are covered in detail:
 Change Control - a formal system for managing proposed or actual change
 Validation - documented evidence that the requirements for a specified intended use
or application have been fulfilled i.e. a collection and evaluation of data from the
process design stage through to production which establishes evidence that the
process is capable of consistently delivering the required quality. It is about gaining
knowledge and an understanding of the product/process. This is generally the pre-
purchasing stage and provides assurance that product meets the needs of
stakeholders.
 Verification – documented evidence that specified requirements have been fulfilled
i.e. a collection and evaluation of data from production through to implementation,
which establishes evidence that the process is in reality consistently delivering the
required quality. In some quality standards this equates to Process Qualification.

Validation and verification is a requirement of ISO15189:2012 standard, 5.5.1. Selection,

verification and validation of examination procedures.

It is the intention of RGL to maintain critical processes and systems in the intended state for
which they were developed and which are in line with regulations, standards and guidelines
that underpin the quality management system.
For this reason, the RGL should:
 understand the process variations
 detect these process variations and assess their extent
 understand the influences on the process
 and control such variations depending on the risk they represent

It’s important to remember that a sound understanding of the process will not inherently lead
to a safe process. The validation and verification process needs to include evaluation of
materials, equipment, the environment, and personnel changes to ensure the process can
be maintained when in routine operation.

Change Control
Any new or changed process, equipment, facilities and systems must be through the change
control process. Uncontrolled change carries significant risk of loss of the validated state for
laboratory processes, equipment, facilities or systems. Requests for change may arise from:
 Planned change as a result of opportunities
 Review of current procedure
 Audit findings
 Incidents
 Complaints

Some laboratory changes may result of using new equipment of the same type or a
relocation of the process in which case the validation may be limited but must still be
documented.

Minor changes or amendments to documented procedures may not need to go through this
process, but will still be managed via the document control procedure.

Regarding new equipment see Equipment Management Guidelines (Equip 1)

Risk Assessment
A risk assessment must be completed to assess the possible consequences of the change
so that action can be taken to eliminate or mitigate the risk, and will inform the extent of the
validation and verification required. The risks need to be well defined and should identify
what might go wrong, what are the consequences if it goes wrong and what is the likelihood
of it going wrong. Consideration may also need to be made of the probability of identifying
when things are going wrong.

Specification
A documented specification is an essential document and should take account of the
opinions of all stakeholders. It needs to be produced before the purchase of equipment. The
specification needs to include:
 Purpose for which the new process, equipment, facilities or systems are required.
Also Include stakeholder requirements
 Describe essential and desirable requirements and functions
 Define the operating environment in which the system will operate to ensure staff
safety
 Define other requirements such as consumables, time/staff resources (inc. training)
and maintenance schedules and compliance with regulations and standards e.g.
EQA, UKAS, HTA.

Each requirement needs to be measureable or verifiable in some way.

It may be possible and desirable to prioritise or weight requirements.

The specification needs to be approved and authorised by an appropriate person.

Procurement
RGL will comply with the following trust and RGL Policy and Procedures:
 Trust ICT Procurement Policy
 Trust Standards of Business Conduct
 RGL Equipment Management Guidelines (Equip 1)

Validation & Verification Plans

A plan will be developed that defines the requirements for a small discrete validation and
verification process or in more complex changes a series of validation and verification
processes.
As a minimum:
 Summary of the change
 the responsibilities for the various activities should be stated
 a schedule for the activities to validate in line with specification

A summary report will be written detailing the outcome of the validation process and decision
to purchase will be made authorised and recorded.

Following purchase the plan needs to be extended to plan the initial verification
implementation and on-going verification phases.

Once all the SOPs, training and records are in place the new process can be authorised for
use. A summary report (see Appendix 1) will be written detailing the outcome of the
verification process will submitted and authorised and recorded.
Final authorisation before implementation will be made by the appropriate Consultant
Clinical Scientist.

If a process is to be developed in house then a more formal validation plan needs to be

developed. The plan needs to define and evaluate the following:
  Include the rational for why methods are sound with sufficient specificity, sensitivity
and accuracy to meet patient needs.
 Demonstrate lab equipment is operating sufficiently
 Controls and EQA
 Parameters
 Limits
 Raw materials
 What data needs to collected and what it will be used for
 Each step must have a specific pass/fail instruction under defined conditions
 Plans need to be more aggressive than would be expected in normal routine use
 Evaluate inter and intra batch acceptance criteria, some guidelines suggest 3
batches need to be assessed.
 If excluding outliers, the plan should explain in advance
 Preventative Maintenance and calibration procedures

Reports must draw a clear conclusion as to whether or not the process is for purpose and
approved for use. It should also state any limitations and Uncertainty of Measurement (see
Process for assigning Uncertainty of Measurement)

Implementation
All documentation should be kept for a minimum of 11 years or the lifetime of the process,
equipment, facilities or system.

SOPs and record templates must be written and authorised immediately prior to the change
being made.

All staff involved in using the equipment must be trained before using the new process.
Training and competency records will be completed and records kept.

Once implemented considerations should be made and documented as to how the new
process will be maintained as intended.

On-going Verification
There needs to be a periodic review of trends in process and maintenance data when in
routine use. Unplanned process variations i.e. errors and incidents need to be reported and
prioritised and resolved, with regular review, until the new process has become established.
The trends in process data will establish the capability of the process and can be used for
future monitoring. The aim is to improve the process; however improvements need to be
made in a structured way in line with these guidelines.
Appendix I: Change request pro-forma
This form is intended for identifying and collating proposed changes in service affecting
processes, systems or equipment, in order to manage the change process to ensure all
documentation is completed in accordance with the ISO15189:2012 standard.

This form should be completed prior to any significant detailed investigation into the change
to ensure the utility of investigating the change request, in order to minimise any
unnecessary use of resource. More importantly, the completion of this form ensures that
any proposed change follows the specific validation/verification procedures required by the
RGL for compliance with the ISO standard 15189:2012.

1. Section 1.1 should be completed as far as possible to establish the goals of the change
request.
 There needs to be sufficient information to enable the authoriser to make a judgment
as to whether to proceed with the proposal or not.

2. Section 1.2 should be completed by the appropriate senior scientist in order to initiate
further detailed investigation of the change request.
 The authoriser needs to decide whether a full business case is required and what
detail is expected within this case.
 In the case of new equipment being required, the authoriser will need to request
whether a capital (>£5k) or non-capital business case is submitted through the
appropriate channel.

3. Section 1.3 will be completed once the business case has been submitted to the
appropriate persons for detailed consideration. This will be dependent on the type of
change requested and the potential associated cost. Once authorised the full
validation/verification procedure will be implemented.
 Authorisation level will depend on the associated cost and impact of the change.
1. Change Request Details
Request name Internal descriptive name Reference Internal ref.

1.1 Change request details

Description of Outline the change being requested and the reasons why this action is
change request required.

Process/ System/
Specify whether this is a change to a process, system or equipment.
Equipment

List any benefit to introducing this change, e.g. improved testing, better
Benefit results, streamlining, quality improvement, reduction in labour/consumable
cost, improved TAT

Needs Investment in equipment (Capital v non-capital), reagents, staffing

SOP Details Detail SOPs which change affects

List any relevant documentation both internal and external indicating the
References derivation of the test and including performance specifications, publications
and any previous validations

1.2 Change request response

Full business case required

Capital business case required

Non-capital business case required

Rejected

Comments Requires submission to RGL Capital Programme, SMT, EMT.

Authorisation Name Signature Date

Lead

Authoriser
1.3 Change request outcome
Approved

Rejected

Comments Proceed to full validation procedure.

Authorisation Name Signature Date

Authoriser
Appendix II: Generic validation/verification pro-forma
This form is intended as a guide to aid the department in developing a suitable validation/
verification procedure. A suggested methodology for using the form has been given below.

1. Section 1 should be completed as far as possible to establish the goals and general
format of the validation/verification.
 Sections 1.1 “Intended use or application” and 1.2 “Requirements” must be
completed at the start of the procedure. The assessment of the validation/verification
depends formally on the confirmation, through the provision of objective evidence,
that these requirements have been fulfilled.
 If mentioned (1.2), the “Expected Performance” should be distinguished from the
“Requirements”, which must be shown to have been fulfilled.
o Example: the statement “should detect all known point mutations of hemophilia A”
could be included as a guide in the Expected performance; if it were stated as a
requirement, it would need to be proven!

2. Section 2 covers the validation of the new system which should be carried out for all
validations and verifications. In the majority of cases this section can be completed on
objective evidence from publications, developmental work, design procedures (e.g. SNP
checking primers) or by the use of limitations or controls in the on-going test. Where this
is not the case, work plans for relevant parameters should be prepared as in 3 below.

3. Appropriate parameters for experimental investigation should be identified with the aid of
appendix A – a checklist is also provided at the top of section 3. For each parameter
required, the investigating scientist develops a work plan based on section 3 (these are
referenced 3.1, 3.2 to 3.n) by completing copies of sections 3.n.1 (‘Aims’, ‘Samples’ and
‘Methodology’). It is suggested that these be maintained in a single document.
Note: several parameters may be tested in a single experiment, for example sensitivity
and specificity.

4. The work plan[s] should be agreed and authorised by the investigator and the senior
scientist/Authoriser by signing and dating in the boxes provided.

5. The experimental work is performed and analysed by the investigator who should then
complete the ‘experimental results’ and ‘interpretation’ sections 3.n.2.

6. The ‘outcome and limitations’ should be agreed between the investigating and senior
scientists by signing and dating in the boxes provided.

7. Points 3 to 6 should be repeated for each parameter to be tested.

8. If there is any non-compliance between the experimental results and the required
performance specification detailed in section 1.2 the parameter in question should to be
re-examined to determine if the methodology can be changed or new limitations
introduced to rectify the non-compliance. Any further work should be recorded in a new
section 3 work plan. Alternatively the implementation can be abandoned.

9. Once all the parameters have been satisfactorily investigated the investigating and
senior scientist can agree and sign off the final conclusions in section 4.
10. Assuming the validation / verification has been completed satisfactorily an independent
review can be performed and the whole process signed-off in section 5 by a Consultant
Scientist.
1. Validation / Verification (delete as appropriate) Details

System name Internal descriptive name Reference Internal ref.

1.1 System details

Intended use or Outline the intended results of the test and how they will be used including
application any interpretative considerations; linked to 1.2 Performance Requirements.

Locus / Gene /
Specify analyte[s]
Marker

Reference
e.g. NCBI accession number
Sequence

Outline
Describe the technology and how it will be employed
methodology

SOP Reference internal SOP

List any relevant documentation both internal and external indicating the
References derivation of the test and including performance specifications, publications
and any previous validations

1.2 Validation/Verification details

Overall Aims State clearly the overall aim of the validation/verification

Define the levels of performance that must be attained (accuracy, precision,

and any other performance requirements such as robustness or maximum
Requirements
failure rate). Linked to 1.1 Intended Use or Application. Link to equipment
specification if necessary.

Validation / State whether the study is a validation or verification and the justification for
Verification this course of action.

State the type of test and if the validation is to be performed at

Type
implementation or on an on-going basis.

List any pre-existing limitations (e.g. test to be performed on DNA extracted

Scope / limitations
from EDTA peripheral blood samples only)

Turn around time The required TAT

Other List any other factors that may affect the utility of the test for the intended
considerations purpose. Expected performance can be mentioned here.
2. Validation of new system

System name Internal descriptive name Reference Internal ref.

Applicability of Is what is being tested appropriate and sufficient to achieve the desired
measurements results? Compared with specification.

Detail any test specific selectivity issues together with limitations and/or
control measures taken to ensure test utility. A validation work plan (section
Selectivity
3) should be drawn up for any specific potential selectivity issues that
cannot be eliminated by limitation or control measure[s].

Detail any test specific interference factors together with limitations and/or
control measures taken to ensure test utility. A validation work plan (section
Interferences
3) should be drawn up for any specific potential interference that cannot be
eliminated by limitation or control measure[s].

Detail any test specific cross-reactivity together with limitations and/or

control measures taken to ensure test utility. A validation work plan (section
Cross-reactivity
3) should be drawn up for any specific potential cross-reactivity that cannot
be eliminated by limitation or control measure[s].

Authorisation Name Signature Date

Investigator

Authoriser
3.n Validation / Verification (delete as appropriate) for [insert parameter]

A copy of this section should be filled in for all parameters to be tested.

Test name Internal descriptive name Reference Internal ref.

Limit of
Sensitivity Accuracy Trueness Cut-offs
detection

Specificity Repeatability Reproducibility Robustness Probability

3.n.1 Work plan

Section aims Describe the specific aims of this section of the validation

Describe the samples to be used including [where relevant] numbers and

type[s] of mutations present, relevant physical characteristics and how the
Samples
sample status has been derived (i.e. the reference or ‘gold standard’ test).
It may be appropriate to reference another document or database here.

Methodology Describe the method to be used to evaluate the specific parameter

Authorisation Name Signature Date

Investigator

Authoriser

3.n.2 Partial results and conclusions

Experimental Summarise the experimental results

results Cross-reference to or include the data.

Summarise the interpretation of experimental results (e.g. estimated level of

Interpretation
accuracy with confidence limits)

 State whether the results fulfil the validation requirements listed in 1.2
Outcome /
 List any specific derived limitations to reproduce the outcome (e.g.
limitations controls and how they should be used)

Authorisation Name Signature Date

Investigator

Authoriser
4. Validation / Verification (delete as appropriate) Final Conclusions

Test name Internal descriptive name Reference Internal ref.

Overall State explicitly if the requirements in 1.2 have been fulfilled;

Conclusion give any other relevant conclusions.

Estimates of
Give experimentally-derived values for the relevant metrics.
accuracy and
Comment on the potential influence of the uncertainty on the reliability
measures of
of the result.
uncertainty

Limitations and/or
List all limitations and control measures required to maintain the on-
predictable
going test performance
interferences

Detail internal quality control measures to be implemented,

Internal QC
addressing in particular the limitations and interferences identified.

External QA Details of external quality assurance measures

Authorisation Name Signature Date

Investigator

Authoriser
5. Implementation

Test name Internal descriptive name Reference Internal ref.

5.1 Implementation Checklist

Details Date
Complete SOP

Risk Assessment, inc COSHH

Order reagents, consumables

Training

Competence assessment

Report template

Subscribe to EQA

Update request forms

Update website

Billing procedure

LIMS functionality

Inform users and other stakeholders

5.2 Independent review

Authorisation Name Signature Date
Consultant
Scientist
(Authorisation)

Date introduced to service

…......................................................................................................
Appendix III: Validation Parameters
Limits of
Description Examples Sensitivity1 Specificity2 Accuracy3 Trueness Precision
detection Probability4

Truly quantitative tests where the Determination of methylation load (%),

A result can have any value between
two limits (including decimals).
characterisation of a mosaic mutation,
heteroplasmy of mitochondrial variant.
v ++ ++ ++

Semi-quantitative tests where the

Sizing a PCR product, Determination of
quantitative signal is placed into one
B of a series of [essentially] undefined
triplet repeat size in Huntington disease,
myotonic dystrophy, etc.
+ ++ ++ ++ +
categories to give the final result.

Determination of a specific copy number

Semi-quantitative tests where the
using QF-PCR, qPCR, or MLPA. E.g.
quantitative signal is placed into one To establish correction
C of a limited series of predefined
chromosome copy number, exon deletion /
duplication in BRCA1, PMP22 gene dosage
+ factors and/or cut-offs ++
categories to give the final result.
in CMT1A and HNPP.

Qualitative tests where the true

quantitative signal can have one of Fluorescent DNA sequencing/ mutation
To establish correction
D many possible values, but the
required result can only have one of
scanning for unknown mutations e.g. by high
resolution melting curve analysis.
++ ++ + factors and/or cut-offs ++
two possible values.

Qualitative tests where the true Genotyping for the presence or absence of a
To establish correction
E quantitative signal can only have one
of two possible values
specific mutation (e.g. F508del in CF or
C282Y in hemochromatosis).
++ ++ + factors and/or cut-offs ++ +

Legend Notes
1. Sensitivity = True Positive / (True Positive + False Negative)
Metric used for implementation validation
2. Specificity = True Negative / (True Negative + False Positive)
Metric used for implementation or on going validation
3. Accuracy = True Result / (True Result + False Result)
Metric used for on going validation
4. The term ‘probability’ is used to describe situations where a
++ Recommended parameter probability that the result is correct can be assigned – primarily in
on-going validation

+ Applicable parameter (less used)

Appendix IV: Flowchart

Rejected
Request for Change
No further action
(Complete Form 1)

Approved

Rejected
Complete & submit
No further action
business case

Approved

Develop Specification
Document

Follow Validation pro-

forma

Tender and review

possible suppliers in
line with specification

Plan, Purchase, Install Perform verification &

document Write SOPs
& train key staff

Training to SOPs
Verification Report
Implement Competency
sign off
Assessment

Maintenance of
intended process