WORKSHOP ON PLC BASED INTERLOCK SYSTEMS FOR

ACCELERATORS AND OTHER LARGE RESEARCH INSTALLATIONS

E.Blanco1), S.Karstensen2), T.Ladzinski1), J.Lindkvist5), T.Lensch2), A.Marqueta3), D.McGinnis6),
A.Nordt6), D.Piso Fernandez6), F.Plewinski6), I.Romera1), R.Schmidt1), A.Vergara3), M.Werner2),
F.Valentini1), M.Zaera Sanz4), M.Zerlauth1)
1)
CERN, Geneva, Switzerland, 2) DESY, Hamburg, Germany, 3) ITER, Cadarache, France, 4) GSI,
Darmstadt, Germany, 5) MAXlab, Lund, Sweden, 6) ESS, Lund, Sweden,

Abstract • No development of hardware required, easy to

ESS was hosting a workshop in collaboration with purchase and exchange, only limited local stock
CERN, GSI, DESY, ITER, IFMIF, MAX IV, CEA-Saclay, needed.
INFN and Cosylab on 29th and 30th of August 2013, with • Easy to connect to a host computer or network.
about 50 participants, including 30 from outside ESS. • Meets the requirements of high EM1 anti-
About 30 presentations were given [1]. disturbance taking into account industrial
The main focus was on PLC based interlock systems, conditions.
however also fast interlock systems based on FPGAs • Stability, reliability, fast response (can be down to
were discussed. 1 ms), compact, modular, easy to install and easy
The workshop covered topics such as PLCs for to maintain.
protection and safety systems (Machine Protection • Standardized for general purposes, large range of
Systems, Access Systems, Target Safety Systems, available commercial products, both for hardware
Detector Safety Systems, Magnet Protection Systems). and software.
PLCs for other sytems were also discusssed (Vacuum • The experience at several labs with using PLCs in
systems, Cryogenics). Here some keywords for the interlock systems is excellent.
discussions in the workshop: Some examples at CERN for the use of PLCs are
- Different PLC architectures (how safe, how fast, Cryogenic System, Vacuum System, Beam Dumping
how complex) System, Magnet Protection Systems, Access System, and
- Different PLC types (experience with vendors, Safety Systems for experiments. At ESS, PLCs will be
future trends) used for Machine Protection Systems, Access Systems,
- Fieldbus technologies Target Safety Systems and Controls of other systems.
- PLC test benches The main focus during the workshop was given on PLC
- Operational experience (radiation issues, based interlock systems, however fast FPGA based
availability, reliability, process safety, etc) interlock systems were also discussed, to better
- Interfaces to control systems understand the areas of application for PLC / FPGA based
- Services (logging, archiving, post mortem, systems.
synchronization to external clocks)
The event was driven by the ESS challenge, with a Some of the questions for the workshop:
design (average) beam power of 5 MWatt. At ESS, PLCs • What are the requirements for Machine Protection
will be used for control systems, for Machine Protection Systems and Personnel Safety Systems?
Systems as well as for Safety Systems. The workshop • Safety level and availability, e.g. is a PLC based
included several sessions: access system without hardware wires acceptable?
• Introduction • What can be done with PLCs, what requires other
• Machine protection and interlock systems at different hardware systems?
labs • Electrical distribution failures –UPS backup power
• PLCs for protection and safety systems required?
• Operational experience • What performance can be achieved, depending on
• Machine protection and fast interlock systems the type of PLC?
• What architectures for PLC based interlock
SESSION 1: INTRODUCTION systems are proposed/used: Ring, tree, how many
Scope of workshop and introduction, Rüdiger crates, what redundancy (1oo2, 2oo2, 2oo3, …)?
• What buses could be used: Profibus, Profisafe,
Schmidt (CERN - ESS) Profinet, others?
Programmable Logic Controllers (PLCs) are widely
used in industry, and since some years in many 1
EM: Electromagnetic
accelerators and other large research installations. The
reasons are manifold:
 • How to connect PLC input and output signals to • Synchronization between PLCs and an external
external systems? clock or timing infrastructure (precision of ~1 ms
• What programming languages are used? required).
• Number of input / output signals to be handled. • System updates and connection to networks.
Direct inputs, inputs via electronic boards, others? • Security: how to avoid problems with hackers?
Outputs, how to connect PLCs in a dependable • Simulations of safety systems, with hardware and
(=reliable, available and safe) way to actuators? software in the loop.
• Algorithms required for logic decision.
• Management of PLC firmware. ESS Project Overview, David McGinnis (ESS)
• Fast processors in PLC based systems. The European Spallation Source, ESS, is a source of
spallation neutrons used for neutron scattering
What is the experience using PLCs in existing systems? experiments, complementary to synchrotron light sources.
• Experience from operation. Reactors are flux limited and provide continuous neutron
• How much effort is required in the development fluxes. Spallation sources produce pulsed neutron beams,
phase? and for many experiments pulsed neutron beams are
• How much effort is required during the operational much better than c.w. neutron beams.
and maintenance phase? ESS has very ambitious goals; the average beam power
• To build and operate a PLC based interlock (5 MW) for ESS is a factor of 5 and the peak power
system, what profile for engineers and technicians (125 MW) a factor of 7 higher than in other facilities, like
is required? at SNS. Experimentation with neutrons at ESS should be
between one and two orders of magnitude more
• How many different types of PLCs in one lab are
performing compared to other sources. One beam pulse
acceptable?
has an energy of 360 kJ – 20 times more than at SNS.
• Response time (cycle / scan time), as function of
ESS uses a long pulse concept: 3 ms pulse length, 14 Hz
input / output signals and SW complexity.
repetition rate, 4 % duty factor. Such pulses still allow
• Pitfalls and risks – near misses. using time of flight for the instruments. The neutrons are
• Connectivity (cables, optical, copper wires, moderated to lower energy with a time constant of
connector types etc.). moderation of about 100 µs. The goal for availability is
• Testing and verification of a PLC based system. with 95 % very high. One of the challenges for the target
• What are the essentials to build a successful test station is the stress generated by the pulses when hitting
stand (PLC lab)? the rotating and helium cooled tungsten wheel.
• What can be outsourced, what should be done in During a recent redesign the energy was reduced from
the lab? During design, commissioning and 2.5 GeV to 2 GeV. This implies an increased technical
operation. risk, since the beam current will increase from 50 mA to
• How to make sure that the system does what it is 62.5 mA. For acceleration, a warm, copper-based part
intended to do? (10%) and a superconducting linac-part (90%) will be
• Start sequences for processes. used. Each superconducting cavity has a power source.
• Transition phases between test phase – operation – There are three families of superconducting cavities:
maintenance and tests during the operational phase. spoke cavities and two types of elliptical cavities
• How to force inputs and outputs without (medium and high β) in the same type of cryo-modules.
compromising safety? Spoke cavities are used to operate with an energy as low
• PLCs from different vendors: what PLCs are being as possible with superconducting technology. The power
used? What is the experience with PLCs from sources (klystrons, IOTs) still need to be decided.
different vendors? How is the support? Another challenge is the Lorentz detuning of cavities
• From requirements to PLC program - the due to the high field of about 400 Hz, dynamic de-tuning
development cycle. is required using piezo-electrical tuners.
Between target and neutron instruments neutron
Questions related to the interface to the controls choppers are installed. There will be more than 20
system: neutron instruments; some of them are installed more than
• Configuration and its management, databases, 100 m away from the target station, linked by neutron
access, software updates. How to download guides. ESS will use very clean and reliable power (only
programs? How to ensure that the correct program 3% of the power comes from fossil fuel). It is planned to
is loaded into the PLC? recycle a substantial part of the total power required for
• PLCs vs EPICS, JAVA based systems, SCADA ESS.
systems, what drivers are available, etc.. 50% of the funding of ESS comes from Scandinavia
(Sweden, Denmark, Norway), 50% from other countries.
Cost drivers for the accelerator are the superconducting

2
cavities and the RF system. It is expected that first An example for the outcome of the risk analysis is the
neutron operation starts in 2019. redesign of the powering for the bending magnets for
vertical deflection of the beam from the accelerator
SESSION 2: MACHINE PROTECTION towards the target. Initially, it was planned to use two
AND INTERLOCK SYSTEMS AT power supplies for the 4 bending magnets, the hazard
analysis however showed that powering by a common
DIFFERENT LABS power supply reduces the risk of erroneous beam
ESS Machine Protection - first ideas, Annika deflection significantly, thus reducing the criticality of the
protection systems. After this design change it is much
Nordt (ESS) simpler to protect the equipment from this hazard with the
The Target Safety System (TSS) and the Personnel MPS.
Safety System (PSS) need to be compliant with the One proposal for the interlocks is to separate slow and
Swedish regulations on nuclear safety. This is not fast signals. The Fast Interlock System will use 250
required for the Machine Protection System (MPS). MPS BLMs6, a few BCMs7 and BPMs8 to detect failures, plus
must protect the machine’s equipment from damage instruments in the RF equipment such as arc detectors to
induced directly or indirectly by beam losses, and at the trigger a beam stop. BLMs are not efficient below
same time it should take into account the ESS overall 90 MeV, since secondary particles do not escape from the
objective of achieving 95% beam availability with high vacuum chamber. A Slow Interlock system will
reliability (95%). For the development of the MPS, the complement the protection, and will be used for power
IEC615082 standard will be used where applicable. After converters, vacuum valves, cryogenics, access system,
defining the MPS concept and overall scope, a etc. The assessment of the reliability of the MPS, but in
Preliminary Hazard Identification (PHI) and a risk general for the entire ESS complex will be performed in
assessment are performed for the accelerator, the target collaboration with colleagues from other labs.
station, the neutron instruments and the conventional
facilities. The risk matrix used to derive the criticality for
certain events is based on a severity ranking taking PLCs at CERN for machine protection and
property losses and production losses into account. Some access interlocks, Ivan Romera and Tomasz
risks are unacceptable; some are tolerable if it is very Ladzinski (CERN)
costly to avoid them (ALARA3 zone) and some risks are
PLCs at CERN are used to protect equipment,
tolerable. The scales are logarithmic and calculating how
personnel and the environment. There are several other
much a risk must be reduced to reach at least the ALARA
systems using PLCs without protection functionality.
region gives then the SIL (Safety Integrity Level) for the
Some examples for both types of applications:
different MPS functions. The required MPS response time
is as well derived from the risk assessment. It is intended • Powering interlock controllers for protection of
to store the results of the risk analysis in a project wide superconducting magnets with 36 PLCs and a few
risk-database and use this database for an automated 1000 signals.
follow-up procedure assuring the proper mitigation of • LHC Cryogenic system: About 80 PLCs with as
causes for the identified risks and top-events. In total, 166 many as 50000 channels.
MPS safety functions / safety requirements were defined • LHC Access safety system has 10 failsafe-
so far for the accelerator systems (example: a vacuum redundant PLCs.
valve closes accidentally). Interlock and protection • Normal conducting magnet interlock system with
systems will require in general SIL2 according to the more than 20 PLCs and 100 remote I/O crates.
latest results from the PHI (sensors to detect a failure are: • Collimation system: 15 PLCs, for environmental
BLMs, RF, BCMs, etc., actuators to stop beam operation measurements slow interlocks.
are choppers in LEBT4 and MEBT5 and the RF • LHC Vacuum system: about 130 PLCs.
magnetron of the Ion Source). The analysis showed that • LHC Experiments detector safety system: few
for some failure cases it is required to stop the beam PLCs.
within a pulse, for other failures it is acceptable to stop The requirements for protection systems are: failsafe
the next pulse(s). The maximum allowable delay for design, redundancy of critical components, critical actions
stopping the beam is as short as 10 µs for some of the to be executed by hardware, dependable. Under certain
failures. circumstances masking of input signal is required. This
can be a critical action and care needs to be taken how to
2
IEC61508: International Standard “Functional safety of electrical / permit such masking.
electronic / programmable electronic safety-related systems”, IEC, 1998,
2000 6
BLM: Beam Loss Monitor
3
ALARA: As Low As Reasonable Achievable 7
4
LEBT: Low Energy Beam Transport BCM: Beam Current Monitor
5 8
MEBT: Medium Energy Beam Transport BPM: Beam Position Monitor

3
 The PLCs are integrated into the controls system the safety of the system. In particular, the current loops
(configuration, logging and SCADA9). The systems are turned out to be an excellent choice.
designed based on technical requirements and
environmental parameters (EMC10, radiation, others). Access system: The system is split into two parts, a
safety system (LASS12) and an access control system
Powering Interlock System: it is based on a hybrid (LACS13). The system has a limited scope and considers
design, using PLCs and custom made electronics for the only radiation hazards but no other hazards such as
most safety critical parts, with about 2500 hard-wired consequences of a helium leak. There are two modes, the
current loops in the LHC ensuring the 1st level of Beam mode and Access mode: if beam is on, there is no
protection. The hardware loops operate with a current of access. And if access is on-going, there is no beam.
10-20 mA and 15-24 V. The system complies with SIL3. The response time is
Less critical functions are implemented within the PLC slow (few seconds). Safety PLCs from Siemens with
software, using signal exchanges, PLC-to-PLC and via redundant equipment are connected in a private network
SCADA. with optical fibres. There is a gateway to the external
In case of a powering failure, a beam dump request is world. Powering is from normal powering, secure
transmitted to the Beam Interlock System via the PLC and powering and from batteries.
in parallel via a CPLD that is integrated in the custom Some elements are responsible for inhibiting beam
made electronics, (part of the PLC based powering operation (“important safety elements”, EIS=elément
interlock system). Commercially available remote I/Os Important pour la Sûreté). There are three per interlock
from the main vendor are used but also low cost I/O chain, technologically diverse and inherently failsafe.
modules, due to radiation considerations in specific The system has about 3800 digital inputs and 800
locations and the need for high numbers of I/O channels. digital outputs. It uses 1oo2 voting, with two
It was pointed out that cabling and connectivity is an complementary inputs and two outputs acting in series on
important consideration, for both, cost and system the power supplies of the actuators. It takes a few 10s ms
availability. for receiving signals. The total delay is a few seconds, up
The system uses many PLCs with only one generic to 8 seconds in case of failures in the PLCs. The French
program; the location specific parameters for the different authorities required an additional hardware relay loop for
PLCs are stored in a database and loaded in the PLCs via the most critical interlocks to guarantee a safer system
dedicated configuration files. In the code, safety and and a shorter maximum response time.
protection functions are separated from monitoring A safety file was prepared, including a proof to
functions through the use of different function blocks. demonstrate the SIL level. This was first done during the
The main interlock functions for a given circuit family are design phase and later after building the system. The
described in state machines. entire development was outsourced, including the
The configuration data is kept in the LHC database, provision of a test platform and the execution of tests. The
with strict version and access control. The generated contractor required about three years to build and test the
configuration files are signed with a CRC11, and the system. The external contractor had two teams. The 1st
SCADA system uses checksum verification tests to team was building the system. A 2nd team, different from
ensure that the matching configuration is present in each the 1st team, performed testing. Further tests of the PLC
PLC and CPLD. software were performed on the test platform by the
Commissioning and operation: the system is first tested CERN team. The test platform is identical to a subset of
in the Lab using a PLC based test bench, configured with the installation. All functionalities can be tested, except
dedicated test software. When deployed in the LHC, load testing. The contractor and the CERN team
100% of all critical functions are tested in the performed hardware tests.
commissioning phase. After commissioning, no changes Final validation was given by CERN after a two
are done during operation and consistency checks are daylong test for each site (9 in total), with many people
performed after every cycle. involved. Moreover, a CERN Safety Officer performed an
The experience with this system is very good and the independent weeklong test.
hardware exceeds reliability predictions made during the Since the installation of the system is completed, only
design phase. 11 failures during more than 5 years of very few upgrades were required; exhaustive annual tests
operation were observed, many failures in the shadow of are partially repeated to maintain the SIL level and
operation (single event upsets due to radiation, problems independent tests are done at the end of every annual
with the connectivity…). Most of the failures were related shutdown for a sample of elements by the CERN Safety
to single event upsets, later the PLCs were moved to Officer.
radiation free zones. None of the failures compromised Operational experience: the LASS has always been
available. The impact on LHC operation: 2 spurious
9
SCADA: Supervisory Control and Data Acquisition
10 12
EMC: Electro-Magnetic Compatibility LASS: LHS Access Safety System
11 13
CRC: Cyclic Redundancy Check LACS: LHS Access Control System

4
trips/year at most, due to bad switches and connections. with neutral beam injection and radiofrequency waves
An indirect impact on operation are patrols that are that need to be switched off in case of non-nominal
required when access integrity is lost, e.g. due to faulty conditions.
position switches. Maintainability: very little time is During the period when ITER does not operate, remote
available for corrective maintenance since such system is handling using robots during maintenance periods require
always needed, during beam operation, but also outside interlocks to avoid any equipment damage.
beam operation. This led to the idea to introduce an Protection is not straightforward since there are some
additional door behind the access point, to allow for better complex functions. Fail-safe states cannot always be
servicing of the access point material while in Beam identified and intelligent redundancy is required.
mode. Interlock triggers that are not justified need to be
On some occasions the signals from an EIS element avoided; a discharge of the large ITER magnets stresses
need to be bypassed, this is done following a strict the structure and the total number of powering cycles is
procedure and using keys for bypassing. limited (e.g. an internal failure of the protection systems
The lifetime of the system is estimated to about 20 should not happen). In the same way, triggering of the
years. It does not pose a problem for the PLCs, but one disruption mitigation system has a cost on the availability
has to remember that in today’s systems there are also and tokamak lifetime. High availability of the protection
servers and client computers to maintain and they have system is required.
lower life-time. A recent upgrade of the operating system The ITER design is not fully frozen and there is a lack
revealed to be quite complex as an underlying safety of experience with such machines. Due to the rules of the
library changed as well. ITER organisation, procurement is distributed around the
For the CERN injectors, the access system is being world. Many different interlock systems need to be
refurbished, using the same types of PLCs (not integrated into a common system. Around 30 plant
redundant). The test platform is being improved as well. interlock controllers will be delivered to ITER from
outside partners.
Discussion: The decision has been taken to separate interlock and
• Outsourcing is possible, but in-house expertise is safety systems. An ITER Interlock Integrity Level is
required to follow-up the work and to operate and derived from the SIL standard in order to ease the
maintain the system. communication with all ITER partners and avoid
• Nuclear authority regulators were on site several confusion with the systems in charge of the nuclear safety
times; did not participate in the tests, but asked that contrary to the interlocks systems are under authority
detailed questions. licensing.
• A reduction of the processing time is not needed Around 130 slow and fast interlock functions have
since a human violating the access conditions will been identified so far. Most interlock functions will be
need some time to access critical zones. done with PLC technology. It was decided to purchase the
• The hardware loop with relays for ensuring safety hardware from a single vendor throughout the project.
is recommended and turned out to be very useful, Outside partners will profit from the central knowledge at
at least for the peace of mind for the safety the ITER site. It is still challenging to coordinate between
responsible in certain situations (e.g. Stuxnet virus the different actors and to ensure compatibility. A
media hype). reliability within 20 years of operation of the CIS has to
• Weak points for availability are position switches. be of about 99.6-99.9 %..
For the controls of ITER, EPICS14 will be used. For the
ITER and IFMIF Machine Protection and controls of the interlock system it is planned to use
PLCs, Alvaro Marqueta, Antonio Vergara another supervision tool (WinCC15). Communication that
Fenandez (ITER) is non-critical for protection between both systems will be
possible.
ITER is progressing and the construction of the The magnet interlock system will rely on slow and
buildings is in full swing. fast controllers as well as on hardwired loops. Most
Risk assessment for the ITER systems is performed in critical actions will be done with hardware loops. To
parallel to prototyping and building of the interlock achieve the high availability as well as the needed safety
equipment. The main risk that the Machine Protection level, 2oo3 logics will be implemented. For less critical
System has to cope with comes from the magnetic field functions, communication between PLCs will be
and also from the plasma current (17 MA) and energy, performed using Siemens network protocols. Two
with a strong coupling between the two systems. redundant consoles for the operation of the interlock
Sensors are required measuring the position of the very systems will operate using WinCC. For an interlock
hot plasma, since it is not acceptable that the plasma system, this is considered to be safer and easier for the
touches the surrounding walls. Current disruptions need
to be taken into account, as strong mechanical forces act 14
EPICS: Experimental Physics and Industrial Control System
on the reactor during disruption. The plasma is heated 15
WinCC: SCADA system from Siemens

5
developer compared to using EPICS. Centralized masking The communication between different crates uses
of signals will be implemented in the supervision layer. A optical frames, with some information, including the
gateway will ensure communication in one direction from beam stop signal and the beam current. The fastest
the interlock system towards the controls system. reaction time is some 10 µs.
The MPS will include modules for supervision, system The configuration is limited to the modification of
protection, coil protection and plasma protection (fast thresholds and some logical combinations (one alarm
interlocks are only required for plasma related functions). input masks another). Changes on the configuration are
rarely done, about once per year. Remote changing of the
DESY Machine Protection and PLCs, Matthias FPGA program is not possible.
Werner and Timmy Lensch (DESY)
At DESY, slow and fast interlock systems are used for SESSION 3: PLCS FOR PROTECTION
several accelerators. AND SAFETY SYSTEMS
The vacuum interlock system for PETRA16 uses one
PLC with distributed controllers. MAX IV, Johan Lindkvist (MAX lab)
The MPS for FLASH17 uses PLCs and fast interlocks MAX IV includes a 3 GeV linac and two storage rings,
(FPGA / TTL18). The PLC controls the masking of the one ring will start operating at 3 GeV in 2015 and a
fast interlock system and reads back the status via second ring operating at 1.5 GeV in 2016. Linac
PROFIBUS. commissioning will start in March 2014.
Interlocks for specific failures that stop the electron Control systems for “slow signals”, Machine Protection
gun can be very fast (some 100 ns). Interlock crates use and Personnel Safety Systems are using PLCs from
redundant power supplies. Rockwell / Allen-Bradley together with remote I/O. There
A magnet current monitoring interlock has been is a lot of experience with these PLCs, in particular with
developed for HERA19 about 10 years ago, and a second the software. MAX IV has in-house experience with this
version was developed in collaboration between DESY vendor and gets good support from the supplier. A safety
and CERN for LHC. This interlock has been successfully level SIL3 for personnel safety can be achieved with
used for many years at DESY and now at CERN. safety PLCs from this vendor. The point I/O family,
PETRA III is operating as X-ray source, with 14 which is the preferred I/O card for MAX-IV, offer long-
experimental stations. For PETRA III, the risk is limited. term production. In the system, regular and safety I/Os
In case of a failure it is estimated that repairs would take can be combined. The point I/O family is distributed
not more than three days, plus the repair cost that need to (remote I/Os).
be considered. The interlock system has a latency of less At MAX IV, a naming convention has been
than 70 µs, mainly due to the time needed for signal introduced. TANGO20 is used as control system including
transmission. The beam can be stopped in 400 µs by the communication with the PLCs. RSLinx handle the
switching off the RF system. Logical combinations of communication between TANGO and PLCs for the
alarm inputs and flexible thresholds are possible. A Post different sub-systems. Tags that will be used for
Mortem trigger is generated simultaneously if beam loss communication between PLCs and TANGO have a
was detected with the MPS’ beam current monitor certain tag name according to the naming convention.
(DCCT). Four PLC controllers will be installed for the linac to
The system can be configured to trigger events only for control vacuum, magnets, power supplies and water,
analysis, without stopping the beam. using remote I/Os. To protect magnets from overheating,
The system includes 10 crates, with optical fibres in one thermo–switch is read out via a PLC; a second switch
between. The electronics is based on FPGAs. Each crate is directly connected to the power supply (hard-wired).
has 112 inputs, as output a dump trigger and a Post For the access system, the Swedish authorities accepted
Mortem trigger. Logical combinations are required for a solution without hardware loop for personal safety
some inputs (configurable). systems, as the system is compliant with SIL3
A DCCT provides digital information about the beam requirements.
current, similar to the safe beam flag used for LHC at
CERN. The beam current thresholds can be set in the Discussion:
MPS for each alarm input individually. How to ensure SIL3 in a PLC based system if remote
Each BPMs gives an alarm to the MPS if the beam is I/Os are being used? Does this need a safe protocol? Can
outside certain orbit thresholds which are configured in this be done with the Ethernet link that is used for other
the BPM system. systems?
16 20
PETRA: Synchrotron light source at DESY TANGO is an object oriented distributed control system
17
FLASH: Free electron laser at DESY
18
FPGA/TTL: Electronics hardware components
19
HERA: Proton-Lepton collider (decommissioned)

6
Architectures for PLC based interlocks, Manuel the current, the use of discrete relays of a IO module for
Zaera Sanz (GSI) the PLC is one of the options.
For a distributed system, fieldbus technologies such as
Dependability means guarantee of correct functioning. Profibus, Profinet and industrial Ethernet are good
It implies reliability, safety, availability, maintainability candidates.
and security. One option to design a dependable system is If a very fast reaction time is required, the Siemens
using methods of fault avoidance and fault tolerance. FM352-5 Boolean processors (up to 12 inputs, 8 outputs)
When failures are considered, both, hardware faults and can be used. This processor using an FPGA is a standard
software faults need to be taken into account. stand-alone module which can be connected through
Safety means that the system prevents catastrophic Profibus to the PLC. It should be noted that time stamping
failures. There are many commercial safety PLCs for on a level of µs is not possible. Accuracy of time
distributed safety and process safety, such as the safety stamping depends on the CPU and is limited to one ms or
PLC family of Siemens. Redundancy can be implemented more.
by hardware (using two PLCs) or by software (using two For ITER, an interlocks rack to protect the High
PLC programs/codes). Temperature Superconducting current leads was built and
A safety program in Siemens Fail-safe S7 CPUs is ready to be used. It uses both, PLCs and a current loop.
includes libraries for fault detection and watchdogs.
When a faulty input or output is detected, the system is Discussion:
put into a safe state. For safety PLCs, additional • To build an interlock system, an analysis of the
mechanisms to ensure safety are available, such as the use hazards must precede the design. It was pointed out
of password, etc. Implications of safety PLCs are a much in the discussion that in a research environment the
slower processing time resulting in increased delays, a design of the architecture needs to start before the
reduced MTBF for the equipment since it is more full risk analysis is available, in particular if the
complex, and a reduced set of programming tools parameters of the systems are evolving.
compared to standard PLCs.
• PLCs are extensively used with very good results.
Siemens fail-safe PLC F-series I/O: safe
The vendor states MTBFs between 20 and 60 years
communication, access to periphery (however it is not
for single modules. It is interesting to note that
possible to directly use Step 7 tools; it is required to use
safety PLCs could be less reliable than normal
specific software blocks), monitoring of the health of the
PLCs, explained by the fact that the electronics is
system (e.g. detection of wire breaks).
more complex.
Siemens PLC H-series: redundant system with two
• Even if the vendor sells a PLC with compliance to
CPUs, redundancy by communication with fibre
SIL3, it is not guaranteed that the final system is
synchronization. One CPU is running, in case a failure is
compliant with SIL3. Safety depends on many
detected, the other CPU takes over. For a fully redundant
factors inside and outside the PLC environment. It
system redundant I/O modules are required.
is delicate to conclude that a system has a level of
Siemens PLC F+H series: safety and redundancy are
SIL3 without in-depth analysis, proper design and
combined.
extensive testing.
Impact of using the F series: large impact on software,
MTBF21 is lower than compared to standard PLCs, the ITER magnet powering interlock prototype
PLC processing time is much slower, the price is higher, using PLCs, Manuel Zaera Sanz (GSI)
it is a closed environment.
Impact of using the H series: in general little effect The magnet protection system is one of the essential
compared to standard PLCs. There is an increase in investment protection systems at ITER and is based on
response time when a switchover occurs, e.g. after the PLC Siemens S7-400-FH. Protection is ensured by a
failure of one PLC. hardware loop with 2oo3 voting. This approach is based
Impact of using both, the H-F series: the impact of each on dependability studies on ITER powering interlock
series adds up. systems done by Sigrid Wagner showing that this
The experience for PLCs from other vendors was not architecture shows the best balance between safety and
presented, but similar solutions exist on the market. availability. Clients are connected with a user interface
For applications where a high SIL level is required, box that can be configured. All interfaces are identical.
current loops are an alternative for interlocking. They are There are several options to design such a system.
relatively easy to build, include a fast response time, but • Safety PLCs and modules ensure that in case of an
there is the risk of a short circuit. Current loops can be internal failure, the PLC will always react in a
operated in a PLC-based environment. A PLC module can defined state. This requires additional PLC internal
be used to generate the current for the loop. To interrupt components and functionality; therefore such PLCs
have a lower MTBF than standard PLCs.
21
MTBF: Mean Time Between Failure

7
 • Failsafe and redundant PLCs are much slower than and a hardware system for the simulation. Initially, the
other PLCs with a minimum cycle time about test system had many output and inputs, but there were
7 ms, fast PLCs have a cycle time of 1 ms. issues in flexibility and scalability. It was difficult to
• For the final choice of the architecture, it is simulate the complete set of equipment.
essential to measure the key response times for Therefore a new safety test bench to validate safety,
“Failsafe and Redundant”, or only “Redundant” operation and usability was built, using a commercial
configurations. product, the Siemens SIMBA box. This allows full-scale
The proposal for the ITER prototype is to use standard testing of the PLC software by emulating all I/O cards via
S7400FH system using standard periphery modules, the a Profibus connection of the SIMBA box to the system
2oo3 logic, and possibly fast processors (FM352-5) under test.
within the PLC environment. The F CPU system is SIMBA boxes can be connected in series; it is possible
needed because many signal exchanges, using Profinet to add other hardware. The system can be programmed in
with the Profisafe profile, have to be performed. C++ for automatic tests. The new system takes much less
Hardware loops with 2oo3 architecture will be space and is more flexible (now only one rack).
implemented to ensure the required SIL level. FM is a The specification of functions is using a formal
Siemens module, very interesting for fast processing in language. This approach is simpler, safer, the correctness
the same PLC environment, and it can be combined with can be demonstrated, the definition of the function is
safety modules (e.g. with the F CPU module S7400-FH). improved and a validation plan can be derived.
The module FM-352-5 does not exist as failsafe module. A validation of the safety functions includes a
The software for the PLC and FPGA are independent, verification of all outputs for all possible events. Tests
ensuring high dependability. State machines are used for can be derived automatically.
writing the software. The reaction time for such a system
is down to 20 µs (FPGA) and around 10 ms (PLC). Discussion:
• The SIMBA box allows simulating complex
Discussion: systems, and generates both digital and analogue
• The proposed hardware loop can at its voltage and signals. It is possible to combine it with a
current rating accommodate up to about 14 users. simulation tool for dynamic simulations. Different
• Care has to be taken in case of micro cuts field buses can be connected.
generated by the safety PLC to test connections to • The formal language is very useful to formulate the
users (e.g. every few seconds, for 500 µs). Other functions, but it cannot automatically generate PLC
systems could interpret such cuts as failure signal. code.
A filter might be required to avoid problems
(watch out that the filter does not lead to an
PLC for the ESS target protection and safety,
unacceptable response time). Francois Plewinski (ESS)
During normal ESS operation the proton beam will be
Test benches for PLCs, Francesco Valentini directed towards the large (2.5 m in diameter) rotating
(CERN) tungsten target-wheel, which is housed inside the target
The CERN-PS has many access zones, each with an monolith in order to confine created radiation. The target-
own set of safety rules. Siemens S7 PLCs are used for wheel is helium cooled and consists of 33 sectors. The
controlling the zones and a simple hardware system with wheel rotation will be synchronized with the 14 Hz pulse
relay logics is added for diverse redundancy. The system rate of the accelerator, and one complete wheel rotation
is complex, since some zones can be accessed when other takes about 2 s. In order to be able to tune the LINAC
zones can operate with beam at the same time. For this before sending beam to the target wheel, a beam dump
reason, complex functions need to be implemented. A will be installed that can be operated with a power of up
clear specification of the safety functions is important and to 50 kW.
a specific formal language is used. Safety Instrumented There are many critical elements, in particular the
Function Formalization is very critical. separation window between accelerator and target, the so-
For the system development method, the norm called proton beam window (PBW).
IEC61511- 1122 is used with 11 phases, starting with a The controls, protection and safety systems must limit
risk analysis until the final phase of dismantling. This can the transfer of radioactive radiation towards the
be visualised as V-cycle, one branch shows the product environment and to workers, suppress any radiologic
definition, and the second branch the integration tests and hazard induced by the beams (proton and neutron beams)
operation. and protect the investment from damage. It should operate
A first safety test bench was developed, including the with high reliability since there are 2000-3000 expected
simulation of two sites, using real hardware, a console, users/year. There are several safety functions: stopping
the proton beam, evacuating H2 from the target zone, etc.
22 In case of damage, a target replacement would cost
IEC61511: International Standard for Functional safety – Safety
instrumented systems for the process industry sector. IEC 2003
8
several million Euros and would lead to a downtime of controls. This includes PLCs for safety and protection
several weeks. systems. A test stand would be an ideal platform for
As soon as the target is operated it will be activated. In development of code and code validation.
the target station, radiation will always be present as well 1st stage: procure basic equipment, perform basic tests,
as a high inventory of activated material. Activation is and gain experience with integration into the controls
also an issue for the beam dump and collimators close to system. Start with first use cases. Tests for different
the target station. communication protocols can be performed.
The target uses cold moderators (liquid H2, 20 K) to 2nd stage: address synchronisation to an external clock,
reduce the neutron energy and thus velocity. The neutrons develop a PLC framework, and test the reliability of PLC
are then guided to the instruments. The tungsten target installations; add more use cases, provide deeper
wheel has an expected lifetime of 5 -10 years. The task of integration into EPICS.
operating such target, while minimizing the radiological Initial users of the test stand are machine protection,
impact to a negligible level, is challenging, considering conventional facilities, and vacuum and target systems.
that it includes a He gas cooling system, many In order to align the PLC installations with other ESS
instruments with their users and that it is located in a systems, standard tools and naming conventions will be
densely populated area. used for electrical schematics and equipment. At a later
The Target Safety System will implement several stage, a configuration database and development
barriers, which confine the expected radiation to well- environment is required. It is challenging to ensure
defined areas and levels: stop proton beam, evacuate coherency between drawings and database.
stored energy, evacuate H2, confine radioactive material, The question was addressed how to measure PLC
ensure heat management and isolate active circuits. The performance, possibly from multiple vendors?
target circuits, the target monolith, and the target station At ESS, there are about 35 systems in the conventional
building ensure confinement of the activated material. facilities, most of the controls will be PLC based. Some
There are a number of standards that can be used for of the PLCs will have an interface to the PSS (Personal
the development of the target safety systems. The selected Safety System) and to MPS.
standards are IEC61508 and IEC 61513, which shall be For safety and protection systems there are specific
used as a guideline to implement such a system. An requirements. One example: automatic code deployment
analysis of the risks is on going, several critical functions should be avoided, redundancy is required, and the
were identified. Preliminary top-level requirements were system should be compliant with IEC61508.
defined and a preliminary design for the target safety There are two issues related to use of PLCs for safety
systems architecture was proposed. and protection functions:
The next step is to develop a helium test stand and to 1. Some of the parameters are critical (e.g. thresholds
perform modelling of the process. It is being discussed related to safety functions). In general, parameters
how to perform this task. The development method uses a should be stored and versioned in a database and
V cycle (as presented in the previous talk). Simulations of then be driven to the PLC.
the processes are required; various software packages are 2. The version of the software operating in the PLC
under discussion (Modelica-Dymola23, Simulink24, needs to be correct.
SCADE25). The work will be done in collaboration with This topic was discussed in more detail later (see last
outside labs or contractors. chapter).

Discussions: Discussion:
• It is required to define what simulations are • A PLC test stand is ideal to perform simulations of
needed, dynamics, mass flow, etc. the physical process. It was pointed out that the
• At CERN, similar simulations were done for the competence of a PLC expert is required, as well as
cryogenics system. someone with competence of the physical process.
• Siemens PLCs are already operating in an EPICS
Proposal for a test bench at ESS, Daniel Piso framework; the EPICs drivers for a single CPU
Fernandez (ESS) exist (e.g. ITER). For other vendors this is not
The motivation for a test stand is to evaluate hardware, clear.
to choose vendors, to gain experience, to test PLCs and • MODBUS-TCP can be used for communication.
the integration with EPICS and to test PLCs for motion • NTP servers provide time stamping, at CERN the
experience is very positive and an accuracy of 1-
23
Modelica-Dymola: Dymola is a commercial modeling and simulation 2 ms is achieved on a regular basis across systems.
environment based on the open Modelica modeling language.
24
• For ESS, it is not fully clear who will perform the
Simulink: is a data flow graphical programming language tool for work, the ESS team, collaborators or contractors.
modeling, simulating and analyzing multidomain dynamic systems. The integration of PLCs into the controls system
25
SCADE: Model-based design, validation and code generation tools
will be performed at ESS. For some systems, PLC
for safety-critical software and hardware applications.

9
 code will be provided by outside contractors. For system is a classical SIS28 control system. Access to the
other systems, code will be developed at ESS. equipment and motor cover systems use Siemens fails-
safe S7 PLCs.
SESSION 4: OPERATIONAL The TIM monorail train in the LHC tunnel uses also
EXPERIENCE safety PLCs for safe stopping the device when, for
instance, an object is in the way of the monorail.
PLCs@CERN, Enrique Blanco (CERN) UNICOS29 was developed at CERN for application
standardization and is now used by many teams in
PLCs are widely used at CERN. For different systems,
different areas at CERN. The initial motivation of the
different solutions were adopted. In a complex
development was for the control of the LHC cryogenics.
environment with many diverse systems the use of
The objective was to create a standardized industrial
standards and a controls framework is important.
control system covering the two layers of the typical
One of the key objectives when using a PLC is to
optimise the availability of the system. Challenging at automation pyramid, the control and the supervision
CERN is the radiation environment where some PLCs layers..
must be close to, the size, the complexity, the precision, The development is based on standards: ISA-88 (IEC
and the required performance of the different systems. 61512) and IEC-61499. The architecture has several
In process control PLCs are active and dynamic, levels (TN30, CERN-LAN, outside). In such a system, the
frequent changes of parameters are required. By opposite, supervision, the controls, the field layer and the
Safety systems are dormant, with little human actions. In communication layer need to be addressed. Time
stamping at the source is done by the TSPP31 protocol, a
some systems it is not obvious how to separate safety and
CERN made protocol.
process control.
Example of process controls: LHC cryogenics, very
UNICOS uses standard objects and standard processes.
complex installation with industrial equipment , long term
storage of data (12 GB/day), many sensors and actuators, UNICOS has a CPC32 object model that standardizes and
PLCs, Industrial PCs (IPCs) to interface the fieldbus facilitates programming of PLCs. Controls and process
WorldFIP26, the last used at the lower level due to its engineers define the requirements, and UNICOS
radiation tolerance. generates code and several services. It comes with
The cryogenics is a large system, with extensive logging services, alarms, etc. The development process
feedback and high communication throughput. PLCs are starts with the specification of instances and functions.
very heavily loaded and connected to many other control This can be done with the help of EXCEL (xml) sheets
and other similar tools. The next step is the automatic
devices. Experience at CERN shows that the availability
generation of instances and standard logics. Automatic
for Siemens PLCs is very high, Schneider PLCs had some
PLC code and SCADA configuration is created. The
issues of reliability, which were analysed and corrected
following step is manual. Specific process logics can be
by the supplier. Some radiation related issues were
inserted. For the analysis of the process a good
observed.
The detector cooling with CO2 deploys both Schneider understanding of the system is important.
and/or Siemens PLCs. Advantages of using such framework are: uniform and
The ISOLDE27 vacuum system is a small/medium maintainable code, less resources are required for the
system, controlled with Siemens PLCs, with WinCC OA development, rapid and homogenized applications can be
used as supervision system. Vacuum controls all along produced, commissioning is simplified (e.g. PLC &
CERN uses Siemens PLCs. The tunnel ventilation SCADA mapping, no development at SCADA other than
the application synoptics). Maintainability is improved;
systems are also based on Siemens PLCs as many other
unified operation in control rooms is possible including
cooling systems, except the LHC cooling water made
centralized monitoring.. Some developers might complain
with Schneider PLCs.
about a reduction of their creativity. Special needs (such
For the detector safety system (for protection of
as safety systems) might need other solutions.
equipment) redundant PLCs are used. During 7 years of
operation one failure in the active backplane was noticed
(single point of failure). PLC working group: A working group was created in
The LHC collimation system monitors jaw and water 1997 to define a PLC policy at CERN valid for 10 years
temperature of about 100 LHC collimators with 15 PLCs: and issued their recommendation in 1999. The motivation
one beam dump was caused due to a PLC failure since the was the large range of PLCs used at CERN before. After
LHC start-up in 2009. a market survey two suppliers were selected based on
PLCs are also used for other installations such as cable 28
Safety Instrumented System
winding machines with an emergency stop system. This 29
UNICOS: Unified Industrial Control System
30
26 TN: Technical network
WorldFIP: Fieldbus protocol 31
27 TSPP: Time Stamp Push Protocol
ISOLDE: Accelerator at CERN 32
CPC: Continuous Process Control

10
certain criteria. Having a limited number of suppliers Availability and Safety of PLC based systems,
allows for a centralized support with expertise and the Alvaro Marqueta (ITER) and Tomasz Ladzinski
availability of spare parts on site, in the case of CERN for
about 1000 Siemens and 160 Schneider PLCs today in
(CERN)
operation. CERN Access Safety System:
The experience with the suppliers is not perfect, but in Siemens FH400 PLCs are adequate for systems where
general satisfactory. It is a good objective to not having safety and high availability is required. Watchdogs ensure
only a supplier, but a partner. This works with both, safety, however, the parameters for such watchdogs have
Schneider and Siemens, they are partners and to be set correctly. This requires some experience. If a
collaborators. As an example, CERN tests new products parameter such as processing time limit is too low,
and new approaches that have not been used elsewhere. availability might suffer, if it is too high, safety might be
Suppliers must always follow tendencies in the market. compromised.
For technical teams, it is recommended to be conservative During five years of LASS operation, four times one of
if possible, and wait until there are indications that a new the CPUs stopped. This was transparent to the operation,
product is sufficiently mature. as the redundant processing unit continued to work.
There is a large investment in PLCs from a few vendors However, there were some issues when hot starting a
at CERN and loyalty is a matter of investment. CPU for the global interlock controller (which
There are new developments in the domain, e.g. for communicates with the entire system): sometimes the
faster cycle times (such as from Beckhoff). synchronisation of the restarted CPU with its pair
It is strongly recommended that the team responsible consumed so many resources that a safety timeout was
for PLCs at a lab provides more than just a brand of triggered. The policy is to wait for a convenient time for
PLCs, such as services, competence and a central support. restarting the CPU – in case of the LHC there is a window
This includes versioning and distribution software, without beam within a few days. The ITER Central
database support, diagnostic and monitoring tools, Interlock System is similar to the LASS global interlock
maintenance capabilities, training and some selected controller in the sense that it also communicates with
hardware. many CPUs. Therefore, further tests on the timeout and
There are still several areas of improvement, such as redundant CPU synchronisation issues were done with
testing and verification methods and tools, virtual our ITER colleagues.
commissioning (e.g. EcosimPro33 modelling and process
simulation software). ITER Central Interlock System:
The vision for Industry 4.0 (fourth industrial Siemens FH400 PLCs were selected for slow
revolution) is a further integration of PLC and SCADA controls/high integrity at ITER, with two power supplies
systems, similar to UNICOS. per CPU. Power supply failures are fully transparent and
Integrated engineering tools address the product life do not stop the system. The distributed I/Os are connected
cycle, including plant asset management, electrical via Profibus, since Profinet does not allow for
diagrams etc. In general these are closed tools and the redundancy. 2oo3 redundancy was chosen for the
integration into a lab is not straightforward. functions demanding highest (SIL3) integrity.
Hardware improvements include a higher speed, and For redundant PLCs, it takes some time until the 2nd
larger memory, improved diagnostics, improved PLC takes over the process in case of a failure of the first
functional safety integrated into the PLCs, field device PLC, in the order of one second. The PLC allows running
with intelligence, redundancy and low cost CPUs. software as a standard part and as a safety part.
Cyber security is a problem that is being addressed. It In F series PLCs, a redundant program is always
requires reinforced security. running – and checks if everything is ok. Several
protection mechanisms are implemented and the PLC
Discussion goes into a fail-safe state in case of problems.
• The experience with PVSS34 is very good – now it Very high availability is required for the central
is available as WinCC Open Architecture (WinCC interlock functions. The maximum delay must be less
OA) by Siemens, using LINUX in the data server. than one second. Due to the complex interlock functions
• Would it be possible / would it make sense to have and the large number of partners it is not obvious how to
a UNICOS version compatible with EPICS? achieve this objective. Therefore many tests were
Simply a matter of resources. performed to understand the performance of a PLC within
a complex system.
A first discharge loop prototype was developed
together with CERN (see above). A test platform for the
33
EcosimPro: Tools for modeling simple and complex physical Central Interlock Systems (CIS) has been built.
dynamic processes. The CIS test platform is based on CPU Siemens S7
34
PVSS: ETM SCADA now known as WinCC OA 414-4H that communicates with 10 partners. This allows

11
performing many different tests such as measuring the • PLC device code skeletons is provided,
execution time of communication and safety functions: • Scripts are using the information in the database,
• Test of different operating configurations (such as and the provided skeleton to generate PLC code
failures and loss of redundancy), blocks,
• Execution time in normal mode, • This allows generating projects.
• Execution time after loss of CPU, For Rockwell PLCs, the vendor provides code for
• Execution time in normal mode without translating e-drawings into PLC data.
redundancy,
• Execution time in normal mode during Discussion:
resynchronisation. • For the selection of the PLC it is proposed to
It was seen that latencies could become critical, establish some criteria that are relevant for ESS.
exceeding 1000 ms, depending on the type of CPU. The Defining a policy for PLCs would be very useful,
Siemens S7 417-5H series (a new PLC model) has a time but such task is time consuming and took at CERN
for fixed point operation reduced by a factor of three. about two years.
After a power off the restart takes between 1.8 s to 0.8 s, • Tests of PLCs from several vendors are time
resynchronisation takes another 0.3 s. consuming.
How representative are these tests for the final • A new model of Siemens PLC S7-1500 will be
installation? This is not yet clear; it depends on released on the market. Before recommending this
parameters of the PLCs, but also on the network traffic, model, it is advisable to wait until there is some
cable length, real code etc. experience with this PLC.
In a future campaign, tests will be performed with
more blocks and more partners. It is difficult to achieve SESSION 5: MACHINE PROTECTION
the objective of a latency of 1 s for all cases. With the
AND FAST INTERLOCK SYSTEMS
CPU 5H series there is an improvement in processing
speed, other components of the system (not only the LHC machine protection and fast interlocks,
CPU) can also be improved. Markus Zerlauth (CERN)
PLCs and services, Gregor Cijan (COSYLAB) CERN has several accelerators, with the LHC as the
An ESS PLC test stand is planned to test PLCs from most complex machine. The energy stored in the beams
three different vendors. A first version will use Siemens and magnets is unprecedented; a failure to dump the beam
PLCs, and different communication protocols between the would lead to serious damage, high cost and long
IOC and PLC will be tested: Modbus TCP, Siemens PLC downtime. Removing beam from the 27 km long LHC
proprietary (s7plc). ring in case of a problem is the most important protection
Is a “framework required? Different engineers might function. This is achieved by deflecting the beam into the
understand the term “framework” differently. A beam dump blocks by kicker magnets.
framework should allow to simplify the task of engineers, Failures impact on the beam within different times
but is in itself complex. scales, there are ultra-fast (less than 1 turn, ~10 µs), fast
The framework for PLCs includes the IOC interface, (90 µs-10 ms, few turns) and slow (seconds, many turns)
PLC interface, system monitoring and debugging, and the failures. Absorbers take care of ultra-fast failures (time in
core application. One possible definition of “framework” the order of µs).
is all tools that are required to develop and operate a PLC Fast failures are detected with many different types of
based system. monitors (e.g. BLMs and monitors for equipment failures,
In general, a framework includes different elements. It such as FMCM detecting the current changes of a
is considered that some elements of the framework are of magnet). The most critical failure that was identified is a
interest for all users, and other elements only for a few trip of the power supply for normal conducting bending
users. Not included in a framework is the code repository magnets. LHC beam loss monitors detect losses in the
and databases for configuration data. order of µs, the BLM system integrates the signal in
For safety applications a framework might create too windows between 40 µs and 84 s.
much overhead and compromise safety, this needs to be During the initial design of the LHC Machine
addressed case-by-case. protection architecture, inputs from some systems were
The generation of PLC codes was discussed. One way anticipated (e.g. input from BLMs). Inputs from some
is organising this process in several steps: other systems came later. The different systems (interlock
• Documentation (electrical and wiring diagrams, system, beam loss monitoring system, etc.) should have
functional specifications), the flexibility to include additional inputs.
• Definition of I/O names, device types, PLC An early separation of Powering Interlock System
connections according to naming convention, (related to the protection of magnets from the stored
energy in the magnets) and the Beam Interlock System
• The information is entered into a database,
(related to the stored energy in the beam) led to a split
12
into a slow and fast interlock system. In total, there are
many 10000 interlock conditions. The beams can be Discussion
dumped in less than 300 µs. • The electronics for interlock systems (both, fast
The concept for the Beam Interlock System is very interlock and PLC based systems) must not be
simple, and can be described as a large AND gate. The installed in radiation areas, since most electronics
realisation of a system distributed around 27 km, with a is not radiation tolerant. Exceptions are the user
reaction time in the order of some 10 µs was challenging, interface boxes that are radiation tolerant. It is
since it should comply with SIL3. hopeless to use VME crates or PLCs in radiation
The architecture is similar to the system at PETRA, areas.
with 17 VME crates, many electronics cards and user • Cables need to be exchanged sometimes due to
connections to the many different systems. aging from radiation.
There is one unique interface to users (so-called user • Interfaces to interlock systems are always Boolean;
interface box). In addition, the system has many test there are no analogue signals transmitted.
options that require additional electronics boards etc. The • Unique interface for all users.
time used by different processes and the communication • The budget for the LHC interlock systems (magnet
lead to a time for dumping the beam in less than 300 µs and beam) was in the order of 5 MCHF for the
(maximum delay). material. A team of about five engineers worked
How to predict the reliability of such system? An for about 10 years on these systems.
FMECA analysis was done. Most important is to identify
all different failure modes. The failure rates of DESY Machine Protection and Fast interlock
components were used to analyse the electronics boards, systems, Matthias Werner (DESY)
which is a very tedious job. This resulted in an absolute
µTCA.4 crates are increasingly used in the physics
number for the MTBF for certain failures. Such number is
community, offering front and back plane modules.
very helpful during the development cycle, even if they
Timing of the modules with a precision of better than 1 ns
are not accurate, and help to improve the system already
for synchronisation is possible. In general, the modules
during the design phase.
offer a high data processing and transmission bandwidth.
It was somewhat surprising how well the predicted
Several cards are available, such as an intelligent digital
numbers matched operational experience.
I/O card developed by DESY, and commercial cards like
For the calculation, random failures were assumed,
the digitizer SIS8300.
ignoring the Bathtub curve. For early life failures, burning
The communication is via PCIe to the control system
in was performed before starting real operation.
and via Gigabit links for direct connections between
From experience, FMECA produces pessimistic
modules. A framework for µTCA software developers
numbers. However, there were issues with combined
exists.
failures and near misses. There are many reasons to have
Several systems at DESY are based on the µTCA-
unexpected situations, including combined failures. An
technology: a prototype for a wirescanner, the MPS for
example: the installation is not quite correct AND the user
XFEL, the BLM system with photomultipliers and
system is not exactly as expected AND the software is not
scintillators and a toroid protection system.
configured correctly AND a simple failure occurs. This
The toroid protection system is under development and
can happen and has been observed.
measures the beam intensity at different locations of a
The prediction of the performance of a complex system
linac. If the difference exceeds a predefined threshold, an
is difficult; this is in particular true for software. For the
interlock is produced. Toroids can be adapted to other
Beam Interlock System, the number of safety critical lines
installations. The toroid system is also used for fast
in the FPGA code is limited. All combinations could be
stabilisation of the beam current and to limit bunch
tested, which is an important criteria for safety critical
charges. The bandwidth allows to measure the intensity of
systems.
individual bunches at a bunch frequency of 4.5 MHz.
Methods to ensure the building of a safe system are
Using several toroids and interleaved fibre chains ensures
reviews, tests and observations during operation.
redundancy.
Other Beam Interlock Systems were deployed at CERN
The BLM system for XFEL consists of about 350
with LHC type hardware: in the SPS, in the transfer lines
BLMs, where 8 BLMs are read out by one card. The
and now in LINAC4. The interlock system for LINAC4
BLM threshold is set in the BLM crate, disabling of
(LINAC4 has many similarities to ESS) uses a tree
BLMs can be done in the MPS crate.
structure. In case a failure is detected, the interlock
In general, the reliability of a system can be improved
system acts on the RF high voltage and choppers to stop
by triple mode redundancy, by CRC checks or by adding
the beam during the same pulse.
TTL hardware for the most critical functions.
In general, for slow interlock systems, PLCs offer some
advantages, since no hardware needs to be developed.
The experience with the LHC Beam Interlock System is
excellent, but the effort for the development was large.
13
FLASH 2 and XFEL, Sven Karstensen (DESY)
There are some worries:
The MPS uses the same technology as other systems
based on µTCA. An essential feature is the scalability of • Someone with physical access to the equipment
the design. The system is configurable (not can change the memory card in a PLC.
programmable). It is independent from the controls • How to ensure that the content in the database is
system. Calculation of thresholds (analogue / digital) is correct, and not changed?
done outside MPS in connected systems. • How to uniquely identify a PLC?
The DAMC2 card has 42 digital inputs and 7 RS422 • Safety PLCs are using a checksum and lock
output channels. Signal transmission is via RS422, modes; this is not the case for other PLCs.
detecting cable breaks. The minimum alarms signal time
is 100 ns. ACKNOWLEDGEMENT
The same firmware operates in every DAMC2 card, but We would like to thank all speakers for their excellent
the functions can be configured. The settings are set by contributions and the session chairs for organising their
the DOOCS system and checked by a server. sessions so efficiently. Special thanks go to C. Prabert for
It is possible to enable / disable inputs and to test input the efficient and smooth organisation.
and test output. Information on beam modes, sections and
slave information is generated. Beam modes are defining
the operation with either only one bunch, a medium
REFERENCES
number of bunches or the full bunch train. The protocol [1] The presentations at the workshop are accessible at:
assigns different priorities. The system is already used at https://indico.esss.lu.se/indico/conferenceDisplay.py?conf
FLASH and a configuration panel for FLASH 2 has been Id=116
developed.
The latency is 82 ns for one system, 780 ns for a slave
and a master and 1400 ns for a master with 2 slaves. The
fibre optics delay needs to be added.
The system was designed with scalability in mind, for
a possible deployment at ILC.

Discussion:
• Safety studies were not yet performed, but the
system would probably not be acceptable for
personnel protection. The risk for XFEL is limited
therefore a system that is designed for, say, SIL3 is
not required.
• VME versus µTCA: µTCA is much more
powerful, but not in the same state of development
as VME. It is not clear how long VME will be on
the market.
Discussion session: how to ensure that correct
program/ configuration is loaded in a PLC or
FPGA, Suzanne Gysin (ESS)
PLCs and FPGAs rely on the correct code being
loaded, as well as the correct configuration data. If this is
done via a framework, the risk of mistakes is reduced.
A problem might occur if a person does not use the
framework and the related procedures/workflow, and
bypasses authentication and authorisation.
Several methods can be used to ensure the correct code
and configuration is present in the PLC:

• Versioning system for PLC code and configuration

data.
• Regular checks of the PLC parameters by a server.
Raise alarms when things are not consistent.
• Use of digital signatures.
• Authorisation and authentication of the user.
14