A data centre Standard for the

Amsterdam Internet Exchange

A Standard defining the minimum acceptable, and also the desirable levels, of technical
implementation, resilience of operation and business continuity management for data
centres providing services to the Amsterdam Internet Exchange

Westeinde12
NL––1017ZNAmsterdam
Netherlands

www.amsͲix.net
Amendment record

1 Draft 12-3-09 BJE

2 First issue 2-4-09 BJE
3 Second issue 6-5-09 BJE
4 Third issue 15-5-09 BJE
5 Fourth issue 19-5-09 BJE
6 Fifth issue 19-10-09 BJE

TableofContents

Contents
1 Introduction ........................................................................................................................ 3
2 MinimalexpectationsofAMSͲIX ........................................................................................ 5
3 Technicaldesignandimplementation ............................................................................... 6
4 OperationalRequirements ............................................................................................... 13
5 BusinessContinuityrequirements ................................................................................... 14
References ............................................................................................................................... 16
Bibliography ............................................................................................................................. 16

Prepared by

B J Elliott BSc, MBA, C.Eng, MIET, MCIBSE

© Capitoline LLP 2009

Capitoline LLP
Capitoline House
7 King Charles Terrace
Sovereign Close
London E1W 3HL

www.capitoline.eu


Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 2 of 18

1 In tr o d uc ti o n 

Thesuccessfulandreliableprovisionofdatacentreservicesdependsuponthefollowing;

x AwelldesignedandconstructedplantthattakesintoaccountappropriateStandardsandbest
practicetoachieveefficientandreliableoperation
x Managementtechniquesthatputintoplaceoperatingproceduresandprotocolsthatensure
thecontinuingefficientandreliableoperationofthedatacentre
x Disasterrecoveryplansthathaveanestablishedandrehearsedprocedurefordealingwithany
incident that impairs the operation of the data centre and puts in to place a recovery
programme


TechnicalStandards

TechnicalStandardsgivethebestpracticemethodsfordesigningandimplementingthedatacentre
fromaphysical,electricalandmechanicalviewpoint.Somerequirementscoveringhealthandsafety
and energy management are covered by European Directives and national standards.  Many
technical standards exist that cover the myriad of engineering disciplines encountered in a data
centrebuttwodocumentsinparticularaddresstheengineeringaspectsofdatacentredesignand
subsequentresilienceandredundancyofthatdesign;theseare:

ANSI/TIAͲ942:2005TelecommunicationsInfrastructureStandardsforDataCenters

and

TierClassificationsdefinesiteinfrastructureperformance.TheUpTimeInstitute,2008

TheUpTimeInstitute’’s(TUI)documentdefinesfour‘‘Tiers’’ofoperationthatdescribetheexpected
downtimeperyearfromadatacentrewhencertaindesignroutesaretaken.TheTIA942standard
draws heavily upon the philosophy offered in the TUI document and offers more detail in
engineeringtermsaboutwhatisrequiredtoachievetheselevels.

OnecentralplankofthephilosophyistheadoptionoftheN,N+1and2Nmethodology.Briefly,N
means enough items to do the job at hand, N+1 means that each system has one redundant
componentand2Nmeansthatsystemsarecompletelyreplicated.Othervariantsmaybe2N+1for
example, meaning two independent systems where each system also has redundant components.
Ultimatelyitisariskversuscoststrategyasmorelevelsofredundancyandresiliencewillinvariably
costmore.Organisationsareinvitedtoconsiderthecostsofdowntimetotheirownbusinessbefore
theydecideuponthestrategymostappropriatetotheirbusiness.

BothoftheabovestandardsareAmerican.TherearenodirectlyequivalentEuropean(CENELEC)or
international standards (ISO/IEC).  The methods described in the two American documents are
mostly universal but for use in Europe the technical references contained in TIA 942 need to be
substitutedforEUDirectivesandCENELECstandardswherevernecessary.

ThehierarchyandorderofprecedenceofstandardsfortheNetherlandsis

x Laws,RegulationsandStatutoryInstrumentsofTheNetherlands
x DirectivesoftheEuropeanUnion
x Dutchnationalstandards
Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 3 of 18

 x CENELECstandards
x ISO/IECstandards
x Otherrelevantnationalstandards,e.g.ANSI,TIA,BSI,VDE
x Industrybestpractice

BusinessContinuitystandards

Buildingadatacentretothelateststandardsandincorporatingtherequisiteavailability/redundancy
techniquesisonlythefirststeptosuccessfullongtermoperation.Thenextstageismanagingthe
datacentreandbeingpreparedfortechnicalfailuresoranyothereventthatimpingesuponthedata
centreoperation.ThisareagenerallycomesundertheheadingofBusinessContinuityManagement,
BCM.  BCM standards can be generally aimed at all businesses and enterprises whereas some are
morefocussedonInformationandCommunicationsTechnology,ICT.

ThefollowingStandardshavebeenidentifiedwhichcoverthisarea.

x BS 7799Ͳ3:2006 Information security management systems. Guidelines for information
securityriskmanagement

x BSISO/IEC20000Ͳ1:2005Informationtechnology.Servicemanagement.Specification

x BS ISO/IEC 27001:2005 Information technology. Security techniques. Information security
managementsystems.Requirements

x BS ISO/IEC 27002:2005, Information technology. Security techniques. Code of practice for
informationsecuritymanagement

x BS25999Ͳ1:2006Businesscontinuitymanagement,Part1:Codeofpractice

x BS25777:2008InformationandCommunicationsTechnology––Continuitymanagement,
CodeofPractice

x BSISO/IEC17799:2005Codeofpracticeforinformationsecuritymanagement

x ISO/PAS22399:2007SocietalsecurityͲGuidelineforincidentpreparednessandoperational
continuitymanagement

x NFPA 1600;2007 Standard on Disaster/Emergency Management and Business Continuity
Programs

InadditionwehaveITIL.ITILconsistsofaseriesofbooksgivingguidanceontheprovisionofquality
ITservices,andontheaccommodationandenvironmentalfacilitiesneeded tosupportIT.ITIL has
been developed in recognition of organisations' growing dependency on IT and embodies best
practicesforITServiceManagement.

WealsohavePRINCE.PRINCE2isageneric,simpletofollowprojectmanagementmethod.Itcovers
howtoorganise,manageandcontrolprojects.Itisaimedatenablingyoutosuccessfullydeliverthe
rightproducts,ontimeandwithinbudget.AProjectmanager canapplythe principlesofPRINCE2
and the associated training to any type of project. It will help to manage risk, control quality and
changeeffectively.

Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 4 of 18


APRINCE2projecthasthefollowingcharacteristics:

x Afiniteanddefinedlifecycle
x Definedandmeasurablebusinessproducts
x Acorrespondingsetofactivitiestoachievethebusinessproducts
x Adefinedamountofresources
x Anorganisationstructure,withdefinedresponsibilities,tomanagetheproject.

InasurveybyContinuityCentralmagazineofdatacentremanagersinNorthAmericaandEurope,
BS 25999:2006 was the most widely used. In 2008 a companion standard was published, BS
25777:2008 Information and Communications Technology –– Continuity management, Code of
Practice.  It is the opinion of Capitoline LLP that, at the time of writing, BS 25777 is the most
appropriatedocumentforuseinEuropetodescribebestpracticebusinesscontinuitymanagement
intheICTenvironment.


2 Mi nim a l e x p e c t a ti o n s of  AMS ͲI X

2.1 Availability:

Maximumnumberofincidents;
x 1outagein5years,maximumdowntimeoftwohours
x MeanTimeBetweenfailuresofnotlessthanfiveyears

2.2 Environmentalconditionsfornetworkequipment

ThetemperatureandhumidityofthecomputerroommustremainwithinthelimitssetbyASHRAE
EnvironmentalGuidelinesforDatacomEquipment:2008.ThisismorefullyexplainedinSection3.6

2.3 Firesuppression:

Thefacilitymustbeprotectedbyahighsensitivityautomaticsmokedetectionsystemcoupledwith
anautomaticgaseoussuppressionsystem.

2.4 Maintenanceregime:

x Maintenanceoncriticalitemsmusttakeplaceatagreedtimes
x Maintenancewindowswillbe00h00Ͳ07h00(GMT+1h)
x Maintenancewillbeannouncedtostakeholdersinadvance
x Therewillbenoroutinemaintenanceworkoncriticalitemsoutsideofthesetimewindows

2.5 Incidentmanagement

x Incidentsmustbemanagedinawaythatallowsadequateresponseandanalysis
x Outageannouncedwithin5mintoAMSͲIXNetworkOperationsCentre,NOC
x Outageannouncementstobemadethroughindependentsystemthatareindependentfrom
powerequipmentproductionenvironment

2.6 Personnel

Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 5 of 18


x Onlyqualifiedpersonnelusedformaintenance
x Communicateescalationprocedures&contactinformation
x RegularlyinformAMSͲIXonupdateescalationcontactinformation
x EfficientsiteaccessforauthorisedAMSͲIXstaff

Therequirementswillbemetthroughthreeareas;

x Thetechnicaldesignandinstallationofthedatacentre
x Themanagementandmaintenanceprocedurestooperatethedatacentre
x Abusinesscontinuityanddisastermanagementprogramme
x TheMinimumRequirementsofAMSͲIXshallbeprovenbyanauditofthedatacentrefacility
by an external qualified auditor acceptable to both AMSͲIX and the operator of the data
centre


3 T e c h n i c a l  de sig n  an d i m p l e m e n t a ti o n 


3.1 Location

 Minimumrequirements
3.1.1 Thebuildingmustbeinanareawithfloodprotection
3.1.2 The building should be no closer than 0.10 km from a railroad or major interstate
highway
3.1.3 Thebuildingmustbeatleast500mfromaradartransmitter
3.1.4 Thebuildingmustbeatleast100mfromamobiletelephonemast
3.1.5 Thebuildingmustbeatleast100mfromanysourceofelectricalinterference
3.1.6 Thebuildingmusthaveadequateaccessbyroadandnearbyparkingfacilities


3.2 Architecturalandbuildingrequirements


 Minimumrequirements
3.2.1 Anycomputerroomwindowsfacingeast,westorsouthmustbecoveredtoprevent
thermalsolargain.
3.2.2 Computerroomnottobelocatedbelowplumbedareassuchasrestrooms,janitor
closets,kitchens,laboratoriesandmechanicalroomsunlessafloodcontainment
systemhasbeenputinplace.
3.2.3 Thebuildingmusthaveadedicatedloadingareatohandle allanticipateddeliveriesof
suppliesandequipment.
3.2.4 ThebuildingmaterialsmustbenonͲcombustible e.g.brick,concrete,plasterboardetc.
3.2.5 Thecomputerroomheightmustallow0.4mabovetheequipmentrackonasolid
floorand0.4maboveandbelowarackforaraisedfloor
3.2.6 Allfloorsmustbecapableofsupporting750kgequipmentracks
3.2.7 Ifaraisedfloorisfittedthenitmustbeatleast400mmhighwithairflowsnot
substantiallyblocked




Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 6 of 18

3.3 Securityandaccesscontrolrequirements


 Minimumrequirements
3.3.1 Anywindowsatgroundfloorlevel,thatallowdirectaccessintothedatacentre,must
bebarred
3.3.2 Securitystaffpershifttobeatleastonepersonper3000m2ofthedatacentrearea
24hoursperday.
3.3.3 Motiondetectoralarmsincomputerrooms,M&Eplantrooms,telecommunication
entranceroomsand,officeareas
3.3.4 Doorsintocomputerroomstobecontrolledbypin,swipecardorbiometricaccess
3.3.5 Maindoorintothedatacentretobecontrolledbypin,swipecardorbiometricaccess
3.3.6 MannedsecuritycounterwithsigningͲinandbadgeallocationtovisitors
3.3.7 DigitalCCTVmonitoringandrecordingofallexternaldoorstothecomputerrooms
3.3.8 Nounauthorisedpersonsmustbeabletoenterthecomputerrooms,M&Eplant
rooms,telecommunicationentranceroomsandofficeareas



3.4 Fireprecautions,smokedetectionandfiresuppression


 Minimumrequirements
3.4.1 Numberandlocationoffireexitsaccordingtolocalcodes
3.4.2 Fire exits to be unblocked, opening in the right direction and with 900 mm wide
approachroutes
3.4.3 EmergencylightingtobeinplaceaccordingtolocalcodesorEUDirectives
3.4.4 ExitsignagetobeinplaceaccordingtolocalcodesorEUDirectives
3.4.5 Manualcallpointsmustbefittedaccordingtolocalcodes
3.4.6 Walls and doors to the computer room, telecommunications rooms, M&E plant
roomsandothercriticalareastobefireratedtoDutchnationalstandards
3.4.7 Thecomputerroomtobeprotectedbytwocoincident(doubleͲknock)smoke
detectionsystems.Atleastonesystemmustbeanaspiratingsmokedetector(ASD)
monitoringreturnairflows.ThesecondsystemmustbeanotherASDdeviceora
combinationoptical/ionisationdetector
3.4.8 Other parts of the building must be monitored by an ASD or a combination optical
/ionisationdetector
3.4.9 Allofthebuilding’’ssmokedetectionsystemsmustbeintegratedintoonecommon
alarm
3.4.10 Settingoffofthealarmmustbetransmittedtoathirdpartymonitoringagencyorthe
firebrigade
3.4.11 Amaintenancelogcoveringallitemsrelatedto firedetectionandsuppressionmust
beavailableforinspection
3.4.12 Evidenceofstafftrainingintheeventofafireeventmustbeavailable
3.4.13 Firemarshalswhowilloverseeevacuationofthebuildingandotheragreedactivities
mustbeidentifiedforeveryshift
3.4.14 Analgorithmshowingthecauseandeffectstatusofallfirerelatedequipment,e.g.
HVAC,UPS,firedampers,smokedetection,gasreleaseetc.,mustbeavailablefor
inspectionandapprovalbyAMSͲIX.
3.4.15 Thecomputerroommustbeprotectedwithagaseousfiresuppressionsystem
3.4.16 ThegasfiresuppressionsystemmustbeanEUapprovedcleanagentwithlowozone
depletionandglobalwarmingpotential

Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 7 of 18

 3.4.17 Thegasfiresuppressionsystemmustonlybeactivatedbytwoseparateand
concurringfiredetectionsystemsorbymanualactivation.Thesystemmustalso
haveamanualoverride.Thesystemmustbeleftinautomaticmode
3.4.18 Areasprotectedbythegasfiresuppressionsystemmustbefittedwithadequate
overpressureventilationplates
3.4.19 Ameansofpurgingtheareaofgasafteradischargemustbeinplace
3.4.20 Manualfireextinguishersmustbeplacedatdoorwaysorotherrelevantpoints
containingCO2orotherapprovedfiresuppressant.
3.4.21 Allfirerelatedsignagemustbeinplace,e.g.gassuppressionsystemwarningson
doors,‘‘actionsintheeventoffire’’noticesetc
3.4.22 EveryroomprotectedbyagassuppressionsystemmustbepressuretestedtoISO
14520andacertificatetothateffecttobeavailableprovingtheroomhaspassedthe
pressuretestwithinthelast12months
3.4.23 Allfiredetectionandsuppressionsystemsmustbedesignedandinstalledby
qualifiedpersonnelanddesignedtonationalorinternationalstandards,e.g.NFPA75,
BS6266,VdS2095etc
3.4.24 Ifthecomputerroom,telecommunicationsroomorM&Eplantroomshavea
sprinklersystemthenitmustbeofthepreͲactiontype
3.4.25 Adjoiningroomsorbuildingstothemaindatacentrelocationshouldbeprotected
withasprinklersystem.Iftheyarenotthenthefireriskoftheadjoiningbuilding
mustbeassessedandmethodstolimitfirespreaddescribed.


3.5 Provisionofpowersupplies


 Minimumrequirements
3.5.1 Thesitemustbefedfroma transformersubstation dedicatedtothedatacentreand
itsassociatedfacilities
3.5.2 Transformersmustbeinasecuredarea
3.5.3 Power cables must enter the building and be terminated in a dedicated electrical
switchroomcontainingallnecessaryswitchingandmetering
3.5.4 Thepoweravailabletothesitefromthemainsutilitymustbeatleast20%morethan
thepeakprojectedload
3.5.5 Atleasttwostandbygenerators,inanN+1configuration,arerequiredtoprovide
powerintheeventofamainsfailure
3.5.6 Thestandbygeneratormustbeavailablewithatleast24hoursoffuel
3.5.7 Ifmorethanonegeneratorisavailablethenanautomaticsynchronizingdevicemust
befittedwithamanualoverrideoperationpossible
3.5.8 Operationofthegeneratorwillbebyanautomatictransferswitchthatwilldetect
mainsfailure
3.5.9 Thegeneratorwillautomaticallyswitchoffwhenmainspowerisrestored
3.5.10 Thegeneratorwillhavemanualcontrolsbutwillbeleftsettotheautomaticsetting
undernormalcircumstances
3.5.11 Amaintenanceandtestregimewillbeinplaceandthesystemwillhavebeentested
withaloadbankandthenwitha‘‘live’’load
3.5.12 Aconnectionwillbemadeavailablesothatanexternalgeneratorcanbeplugged
intothemainswitchboard
3.5.13 Thegeneratorandcablingwillbeinasecurelocationadjacentorwithinthedata
centre
3.5.14 Thepowercapacityofthegeneratorwillbe atleast20%morethanthepeak
projectedload
3.5.15 TheICTequipmentwillbeprotectedbyanUninterruptiblePowerSupply,UPS,inan
N+1manner

Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 8 of 18

 3.5.16 TheenergystoragemechanismoftheUPSwillbesufficientthattherewillbe
adequatetimeforthestandbygeneratortostartandcomeonline
3.5.17 EachUPSwillhaveitsowndedicatedenergystoragemedium,e.g.batterystring,
kineticenergysystemetc.
3.5.18 TheUPSwillbecentralised
3.5.19 ThecapacityoftheUPSwillbegreaterthanthepeakprojectedload.Theaverage
runningloadcapacitywillnotbemorethan80%oftheUPScapability
3.5.20 TheUPSwillhaveasoftwarebasedmonitoring,reportingandalarmsystem
3.5.21 TheUPSwillbecapableofanautomaticandmanualbypassontorawmainsfeedin
theeventofmaintenancerequirementsorcompleteUPSfailure
3.5.22 UPSunitconnectedinparallelwillhavebeenfullytestedunderloadandwillbe
guaranteedbythemanufacturerforthatconfiguration
3.5.23 TheUPSwilldistributepowertotheICTequipmentviaPowerDistributionUnit
(PDU)switchgear
3.5.24 TwopowercableswillbeavailabletoeachracklocationfedfromtwoseparatePDUs
3.5.25 Arackdesignatedasacommunicationsrackwillbeabletobesuppliedwithatleast
two32ampsinglephasefeeds
3.5.26 Racksrequiringmorepowerwillbesuppliedwithmore32ampfeedsbutpreferably
withathreeͲphasedistributionsystem
3.5.27 Threephasepowerdistributionsystemsthroughoutthedatacentrewillbeevenly
loadedtowithin5%ofeachotherandthismustbeeasilymeasurableatallsignificant
pointse.g.UPSoutput
3.5.28 TwopowerstripswillbeavailableineachrackeachfedfromaseparatePDU.Each
powerstripmusthavealocalcurrentmonitoringfacility
3.5.29 Thebuildingwillbeprotectedwithalightningprotectionsystem
3.5.30 Externalplantsuchasgeneratorsandcondenserswillhaveallexternalmetalwork
earthedsothatsurgecurrentscannotenterthebuilding
3.5.31 Incomingcoppercommunicationscableswillbeprotectedwithsurgesuppressors
beforetheyenterthecomputerroom
3.5.32 Thecomputerroomwillhaveanexposedcoppermainearthterminalthatwillbethe
earthreferencepointfortheroom
3.5.33 Thefollowingitemswillconnectedtotheearthingsystem;allequipmentracks,at
leastevery6thlegofanyfloorpedestalarrangement,allmetalliccablecontainment,
allelectricalswitchgear,HVACunitsandmetallicducting,allmetallicpipes,the
exposedstructuralsteelofthebuilding
3.5.34 Ifpowerislosttoarackorracksthenthesystemmustbeabletorestart
automaticallywithoutcausingcircuitbreakerstotripoutduetohighinrushcurrents.
Thismaybeachievedbyusing'softͲstart'equipment,automaticsequentialstartup
proceduresorbyhavingsufficientovercapacityinthepowersupplytrain.















Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 9 of 18








 Notethattransferswitches
 andsynchronisation
 devicesmaybeseenasa
 singlepointoffailurebut
 theprobabilityoffailureis
 regardedassufficientlylow


































 Figure1
 Minimumacceptablepowersupplyarrangement.GenerallyinanN+1format



Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 10 of 18

3.6 Provisionofcoolingandventilation

 Minimumrequirements
3.6.1 Thecomputerroommustmaintaintemperatureandhumiditycontrolaccordingto
theASHRAE2008requirementslistedbelow.Batteryroomsmustbelimitedto250C.
Thesystemmustmaintainthiscapabilityatfullloadandwithanexternal
temperatureof360C
3.6.2 HVACequipmentshallbeconnectedtothemainsinputandbackedupwithastandby
generatorasdescribedinsection3.5
3.6.3 HVACequipmentshallsupplycoolingcapacityinanN+1format
3.6.4 HVACequipmentshallberatedtoatleast20%morecapacitythanthemaximum
expectedsteadystateloadofthecomputerroomevenafteroneCRACunithasfailed
orbeenturnedoff
3.6.5 Humiditycontrolmustbeprovided
3.6.6 Aventilationsystemmustbeinplacetochangetheairinthecomputer,telecoms
room,plantroomatapproximatelyoneairchangeperhour
3.6.7 AllincomingairmustbefilteredaccordingtoEN77960%standard.Smokedetectors
mustbefittedtotheincomingair,afterfiltration,thatwillshutofftheairintakesif
smokeisdetected
3.6.8 AmeansofpurginggasͲprotectedroomsmustbeinplaceforremovaloffire
suppressiongasafteradischargeevent


ASHRAE2008computerroomenvironmentalrequirements

Lowendtemperature   180C
Highendtemperature   270C
Lowendmoisture 5.50Cdewpoint(note,thismaybetakenasaminimumRH
of35%forsimplemeasuring)
Highendmoisture 60%RHand150Cdewpoint

Inaddition

Minimumcoldaisletemperature 180C
Maximumcoldaisletemperature 240C
Maximumhotaisletemperature 350C

Ifthereisnodefined‘‘hotaisleͲcoldaisle’’layoutinplacethenmeasurementswillbetakenattheair
inputandairexhaustsectionsofallappropriateracks.


3.7 ProvisionofCablingandcablemanagement


 Minimumrequirements
3.7.1 AstandardsͲbasedstructuredcablingsystemistobeinplacetoprovideconnectivity
toeveryrack
3.7.2 Allinteriorcablingshallbeofalowflammabilitytype.Anynewcablingmustbe
specifiedaslowflammabilityandzerohalogentype
3.7.3 Externalcables,i.e.flammableconstruction,shallnotbetakenmorethan5minto
thebuildingbeforetheyareterminatedorenclosedinanonflammableconduit
system

Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 11 of 18

 3.7.4 Everyrackwillhaveaccesstoacablecontainmentsystem,aboveandorbelowit,
thatwillallowcablestobeproperlymanagedbetweenracks
3.7.5 Dataandsinglephasepowercableswillbeseparatedbyatleast200mm
3.7.6 DataandthreeͲphasepowercableswillbeseparatedbyatleast600mm
3.7.7 Nocablewillbetrappedindoors,laidoutonthefloor,strungbetweenracksor
forcedintorightanglebends
3.7.8 Everycablewillhaveauniqueidentitylabelatbothendsdenotingorigin,destination
andifnecessary,owner
3.7.9 Everyequipmentrackwillhaveauniqueidentityandifnecessary,owner’’slabel
3.7.10 Therewillbeaseparatetelecommunicationsroomordedicated‘‘comms’’areainthe
computerroomforconnectingseparateexternaltelecommunicationscablesinputs.
3.7.11 Telecommunicationsroomsorareas,thatcontainactiveequipment,willprovide;air
conditioningandfireprotectiontothesamestandardasthecomputerroom,change
fromexternalgradetointernalgradecable,provideovervoltageprotectiononall
coppercables,provideacleardemarcationbetweenownershipofcablesand
equipment
3.7.12 Allcablespassingthroughfirewallswillbesealedagainstfireandsmoketonational
orEUregulations
3.7.13 Allmetalliccablecontainmentitemstobecorrectlyearthed
3.7.14 Eachequipmentrackwillprovideadequateverticalandhorizontalcable
managementitemstopreventpatchcordsfromobscuringanyitemofequipment


3.8 BuildingManagementSystemsandmonitoring


 Minimumrequirements
3.8.1 Thecomputerroomwillhaveonegeneraltemperaturesensorandonegeneral
humiditysensorfortheroom
3.8.2 Telecommunicationsandplantroomswillhaveageneraltemperatureandhumidity
sensor
3.8.3 Therewillbeawatersensorundereveryraisedfloor,everyCRACunitandevery
watercooledrackandalsoineveryplantroom
3.8.4 EverysensorwillbewiredbacktoadedicatedBMSpanelinoneofthe
telecommunicationsrooms
3.8.5 AsoftwareͲbasedBMSwillmonitor
Ͳeverytemperature,humidityandwatersensor
Ͳthestatusofthefirealarmpanel
Ͳthestatusofthesecurity/accesscontrolsystem
Ͳthestatusoftheincomingmainsandautomatictransferswitch
Ͳthestatusofthegenerators
ͲthestatusofeachUPSandCRACunit
Ͳthestatusofanyfire/smokecontrolorHVACdampers
Ͳitsownstatus
3.8.6 ThisinformationwillbedisplayedonadedicatedBMSterminalwithgraphical
interface
3.8.7 TheessentialitemsoftheBMSsystemwillhaveadedicatedUPSsupplywithatleast
onehourofbatterytimeorwithotherpowerbackupmethods.
3.8.8 Amanagementstrategywillbeinplacethatwilldescribethereactiontoeachalarm,
whowillrespondandresponseoutsideofnormalworkinghours
3.8.9 Essentialalarmswillconnectviaanexternaltelecommunicationscircuittoanagreed
listofstakeholders
3.8.10 TheBMSwillstorehistoricaltrendsandgenerateincidentlogsandreports

Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 12 of 18

 3.8.11 TheBMSwillbefailsafeinthatifitshouldfailitwillnotaffecttheoperationofany
systemthatitpurelymonitors

4 O p e r a t i o n a l  R e q ui r e m e n ts 

Management techniques must be put in place that give operating procedures and protocols that
ensurethecontinuingefficientandreliableoperationofthedatacentreandtimelycommunication
toallstakeholders.


4.1 Goodworkpracticesinthebuilding


 Minimumrequirements
4.1.1 Thedatacentrewillhaveonecentralmannedpedestrianentranceandonegeneral
loadingbayarea
4.1.2 Allequipmentwillbeunpackedandassembledinadedicated‘‘build’’areaandnotin
alivecomputerroom
4.1.3 Therewillbeadedicatedcontrol roomorarea.Allcontrolandmonitoringfunctions
willbeconnectedintoanddisplayedinthecontrolroom


4.2 OperationalManagementDocumentationrequired


 Minimumrequirements
4.2.1 Ageneraloperationsmanualisrequiredthatbringstogetheralltheoperating
requirementsandsettingsofthedatacentre
4.2.2 Allmajoritemsofequipment,e.g.UPS,HVAC,generatoretcshallbeincludedinan
assetregisterwhichwillincludeasettingsandoperationalproceduretoinclude
x Location
x Serialnumber
x Dataofprocurement
x Manufacturercontactdetails
x Maintenance/calloutcontactdetails
x Servicehistory
x Datesofcalibrationifrequired
x Settingsandsetpointstobeobservede.g.RHandreturntemperatureonCRAC
units
4.2.3 Achecklistisrequiredofallitemsthatneedtobeleftinan‘‘automatic’’statusand
wherethesecontrolsarelocated,e.g.firecontrolpanelandgassuppressionsystem,
activationofstandbygeneratoretc
4.2.4 Sitedocumentationinformationtoinclude
x Buildingandfloorplans
x Locationsofracksandothermajoritemsofequipment
x Racklayout
x Logicalandphysicalinterconnectivityofequipment
4.2.5 A‘‘calllist’’istobeavailablewhichwilllistallprinciplestaffofthedatacentre,their
functionandcontactdetailsplusanyappropriatesuppliers,maintenancecompanies
Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 13 of 18

 andemergencyservices
4.2.6 Thedatacentrewillhaveasafetyoperationsmanualthatwilldetail
x Safeworkingatheight
x Safeuseofelectricity
x Useoflasertransmissionequipment
x Liftingheavyloads
x Monitoringofnoiseintheworkplace
x Useofbarriersaroundworkareassuchasmissingfloortilesetc
4.2.7 Analgorithmchartisrequiredthatwilldetailthecauseandeffectofallrelated
x Firedetection
x Alarmstates
x Activationofgassuppression
x Controlofdampers
x Poweringoffofpowersupplyequipment
x PoweringoffofHVACunits
x Restartprocedures
4.2.8 Amethodstatementisrequireddetailinghowvisitorswillbeidentifiedandescorted
aroundthesite
4.2.9 Amethodstatementisrequireddetailinghowexternalcontractorsmayenterand
workonthesiteandhowtheirworkismonitored,controlledandrecorded
4.2.10 Anaudittrailisrequiredthatdemonstrateshowchangestothedesignandfabricof
thedatacentrearemade.ThiscouldbebyaChangeControlDocumentandWork
Ordermethod
4.2.11 Awrittenplanmustbeinplacethatdetailshowallalarmsstatesandsystemfailures
willbehandledbythedatacentrestaff


4.3 Maintenanceprocedures


 Minimumrequirements
4.3.1 Anyinternalstaffinvolvedinmaintenancemustbetrainedandqualifiedtodoso.A
personalstatementmustdemonstratehowthiscompetencehasbeenachievede.g.,
professional/tradequalifications,attendanceatmanufacturers’’trainingcoursesetc
4.3.2 Anyexternalcontractorinvolvedinmaintenancemustalsobeabletodemonstrate
competence
4.3.3 Allequipmentrequiringmaintenanceshallhaveamaintenancelogdetailingthe
equipment,maintenancedates,outcomes,contactdetailsetc.See4.2.2
4.3.4 Maintenanceoncriticalitemsmusttakeplaceatagreedtimes.Thesewillbeduring
07h00Ͳ12h00(GMT+1h).IncidentManagement,i.e.,tobringacomponentbackinto
production after a failure, should be done between 00h00 and 07h00Maintenance
willbepreͲannouncedtostakeholdersatleastoneweekinadvance.Therewillbeno
routinemaintenanceworkoncriticalitemsoutsideofthesetimewindows.



5 B us in es s C o nt i nu i ty r equi r em ents 


Recoveryplansmustbeinplacethathasanestablishedandrehearsedprocedurefordealingwith
any incident that impairs the operation of the data centre and puts in to place a recovery
programme.


Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 14 of 18

AnICTcontinuityprogrammemust

x Understandthethreatsto,andvulnerabilitiesof,ICTservices
x IdentifythepotentialimpactsofdisruptiontoICTservices
x Providecollaborationbetweendatacentrestaff,internalandexternalserviceprovidersand
stakeholders
x DemonstratecompetencebycredibleresponsesthroughexercisingICTcontinuityplansand
testICTcontinuityarrangements

Thedatacentremustbeableto

x ProtecttheICTenvironmentthroughresilientdesignandbestpracticeengineering
x Detectincidentsattheearliestopportunity
x Reacttoanincidentinanappropriatemanner
x Recoverthedataprocessingandcommunicationssystemsinaprioritisedmanner
x Operateinadisasterrecoverymodeuntilnormaloperationsarepossible
x Returntoanormalbusinessposition



5.1 Incidenthandlinganddisasterrecovery



 Minimumrequirements
5.1.1 Thedatacentremusthaveariskassessmentanalysisthatdetailsrisks,theimpactof
theseriskandthestrategytoobviatetheserisks,e.g.
x Site:fire,flooding
x Operations:powersupplyfailure
x Personnel:majorepidemicetc
x Communications:maincabledamage
5.1.2 AllcriticalsystemssuchasUPSstatus,fireconditionsetc,mustbeautomatically
monitored.Thesearedetailedinsection3.8
5.1.3 AnycriticalfailureorlossofservicemustbecommunicatedtoAMSͲIXbytelephone
withinfiveminutes
5.1.4 Anycriticalfailureorlossofservicemustbeautomaticallycommunicatedtoall
relevantpersonnel,managementandotherstakeholderswithinfiveminutes
5.1.5 Themethodofcommunicationtoexternalagenciesandstakeholdersmustbe
independentofthesamecriticalfacilitiesthathavefailede.g.essentialmonitoring
andcommunicationsequipmentmustbepoweredbyanindependentbatteryͲbacked
supply
5.1.6 Awrittenplanmustbeinplacethatdetailshowallalarmsstatesandsystemfailures
willbehandledbythedatacentrestaff
5.1.7 Allmajorincidentswillbesubjecttoamanagementreviewthatwilldetail
x Whattheincidentwas
x Whereithappened
x Whenithappened
x Whatwastheimpactonserviceprovision
x Howwasitdealtwith
x Whatchangescanbemadetopreventasimilarincidenthappeningagain
5.1.8 Astatementisrequiredthatwilldetailtheimpactofasuddenandcompletelossof
powerontheICTequipmentandhowarestartwillbemanaged

Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 15 of 18

 5.1.9 A‘‘calllist’’istobeavailablewhichwilllistallprinciplestaffofthedatacentre,their
functionandcontactdetailsplusanyappropriatesuppliers,maintenancecompanies
andemergencyservices.Contactdetailsmustalsoinclude‘‘outͲofͲhours’’contact
detailsanddefineresponsibilitiesofalltheparties.
5.1.10 TheeffectsofpowercutsmustbesimulatedtoprovetheoperationofUPSand
generatorstartuproutines.Thesesimulationsmustbetreatedasmaintenance
episodesandtakeplaceandbecommunicatedas4.3.3




References


ANSI/TIAͲ942:2005TelecommunicationsInfrastructureStandardsforDataCenters
ASHRAE2008computerroomenvironmentalrequirements
TierClassificationsdefinesiteinfrastructureperformance.TheUpTimeInstitute,2008
BS6266:2002Codeofpracticeforfireprotectionforelectronicequipmentinstallations
EN 779:2002 Particulate air filters for general ventilation. Determination of the filtration
performance
EN50173Ͳ5:Informationtechnology:GenericCablingforDataCentres
ISO 14520Ͳ1:2006 Gaseous fireͲextinguishing systems. Physical properties and system design.
Generalrequirements
NFPA75:StandardfortheProtectionofInformationTechnologyEquipment,2009Edition
VdS2095:2005Ͳ02VdSguidelinesforautomaticfiredetectionandfirealarmsystemsͲPlanningand
installation



Bibliography


Accessforthedisabled

Council Directive 2000/78/EC of 27 November 2000 establishing a general framework for equal
treatmentinemploymentandoccupation
BS8300:2008Designofbuildingsandtheirapproachestomeettheneedsofdisabledpeople——Code
ofpractice

BusinessContinuity

BS7799Ͳ3:2006Informationsecuritymanagementsystems.Guidelinesforinformationsecurityrisk
management
BSISO/IEC20000Ͳ1:2005Informationtechnology.Servicemanagement.Specification
BS ISO/IEC 27001:2005 Information technology. Security techniques. Information security
managementsystems.Requirements
BS ISO/IEC 27002:2005, Information technology. Security techniques. Code of practice for
informationsecuritymanagement
BS25999Ͳ1:2006Businesscontinuitymanagement,Part1:Codeofpractice
BS25777:2008InformationandCommunicationsTechnology––Continuitymanagement,Codeof
Practice

Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 16 of 18

BSISO/IEC17799:2005Codeofpracticeforinformationsecuritymanagement
ISO/PAS 22399:2007 Societal security Ͳ Guideline for incident preparedness and operational
continuitymanagement
NFPA1600;2007StandardonDisaster/EmergencyManagementandBusinessContinuityPrograms

Designstominimizefirerisk

BritishBuildingRegulations:2000PartB,FireͲstopping
TheConstructionProductsDirective(CouncilDirective89/106/EEC)
BS5839Ͳ6:2004Firedetectionandfirealarmsystemsforbuildings.Codeofpracticeforthedesign,
installationandmaintenanceoffiredetectionandfirealarmsystemsindwellings
BS6266:2002Codeofpracticeforfireprotectionforelectronicequipmentinstallations
ISO 14520Ͳ1:2006 Gaseous fireͲextinguishing systems. Physical properties and system design.
Generalrequirements
BS7273Ͳ1:2006Codeofpracticefortheoperationoffireprotectionmeasures.Electricalactuation
ofgaseoustotalfloodingextinguishingsystems
EN54Ͳ20:2006Firedetectionandfirealarmsystems.Aspiratingsmokedetectors
EN 779:2002 Particulate air filters for general ventilation. Determination of the filtration
performance
BFPSACodeofPracticeforDesign,Installation,Commissioning&MaintenanceofAspiratingSmoke
Detector(ASD)Systems2005
BFPSAGuidanceonECRegulationNo842/2006onCertainFluorinated
GreenhouseGases2006
Regulation (EC) No 842/2006 of the European Parliament and of the Council of 17 May 2006 on
certainfluorinatedgreenhousegases
NFPA72:NATIONALFIREALARMCODE:2007
NFPA75:StandardfortheProtectionofInformationTechnologyEquipment,2009Edition

ElectricalSafety

BS7671(IEE17thEdition)––RequirementsForElectricalInstallations
BS6701––TelecommunicationsEquipmentandcabling
EN50310––Bonding&EarthinginBuildingsWithITequipment
EN60950Ͳ1:2002,InformationtechnologyequipmentͲSafety~Generalrequirements
NEN1010:2007Dénormvoorlaagspanningsinstallaties

EmergencyLightingandsignage

BS5266:Part1:1999ͲCodeofPracticeforEmergencyLighting
BS 5499Ͳ5:2002 Graphical symbols and signs ~ Safety signs, including fire safety signs Ͳ Signs with
specificsafetymeanings
StatutoryInstrument2005No.1541TheRegulatoryReform(FireSafety)Order2005

Energyconsumptionandoptimisation

EUEnergyperformanceofBuildingsDirective2002/91/EC
EUEnergyServicesDirective2006/32/EC
Energy savings at Data Hotels, The Environmental and building department of the Municipality of
Amsterdam,ECN,April2008

Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 17 of 18

EU Code of Conduct on Data Centres Energy Efficiency Version 1.0 EUROPEAN COMMISSION
DIRECTORATEͲGENERALJRCJOINTRESEARCHCENTREInstituteforEnergyRenewableEnergiesUnit,
October2008

Healthandsafety

TheConstruction(DesignandManagement)Regulations2007
Workplace(Health,SafetyandWelfare)Regulations1992
ANSIZ136.2ͲSafeUseofLasersinOpticalFiberCommunicationSystemsUtilizingLaserDiodeand
LEDSources
IEC60825Ͳ2.SafetyofLaserProductsͲPart2:Safetyofopticalfibrecommunicationsystems(OFCS).

HVAC

ASHRAEThermalGuidelinesforDataProcessingEnvironments2004
CIBSEGuideB.Heating,ventilation,airconditioningandrefrigeration
TheBuildingRegulationsApprovedDocumentFͲVentilation
TIA942TelecommunicationsInfrastructureStandardforDataCenters
VDI2054Airconditioningforcomputerareas

RaisedAccessFloors

EN12825:2001Raisedaccessfloors
Property Services Agency (PSA) Method of Building Performance Specification 'Platform Floors
(RaisedAccessFloors)',MOBPF2PS
RIBANBSK41Raisedaccessfloors

SystemDesign

EN50173Ͳ2:ITGenericITCablingForCustomersPremises
EN50172Ͳ3:ITGenericCablingforIndustrialPremises
EN50173Ͳ4:ITGenericCablingforHomes
EN50173Ͳ5:ITGenericCablingforDataCentres
ISO11801:2001ITGenericCabling
TIA942TelecommunicationsInfrastructureStandardforDataCenters

SystemInstallationandqualityassurance

EN50174Ͳ1(2000)Informationtechnology––cablinginstallation ––Part1:Specificationandquality
assurance
EN50174Ͳ2(2000)Informationtechnology––Cablinginstallation––Part2:Installationandplanning
practicesinsidebuildings,
ISO/IECTR14763Ͳ2––Informationtechnology––Implementationandoperationofcustomerpremises
––part2:Planningandinstallation

SystemTest

EN50346:2002InformationtechnologyͲCablinginstallationͲTestingofinstalledcabling
IEC61935Ͳ1 Generic cabling systems –– Specification for the testing of balanced communication
cablinginaccordancewithISO/IEC11801––Part1:Installedcabling

Amsterdam Internet Exchange Doc ref CPTL 0908-09 Copyright licensed to AMS-IX
AMS-09-001 Issue 005 19-10-09 BJE Page 18 of 18