IOH CR MOP Format V1.

758265790.docx

Prepared by: Yudi Arfianto Imam Date: 21 May 2023

M. Kharis Hartadi /
Reviewer: Date: dd mmm yyyy
Dhimas Pradityo
Hery Nugraha / I P
Approver: Date: dd mmm yyyy
Astrawan
 IOH CR MOP Format V1.7

Contents

1 Description of Change..................................................................................................................1
1.1 Description of Site......................................................................................................................................................1
1.2 Purpose and Requirements.........................................................................................................................................2
1.3 Description of Change and Change Influence............................................................................................................2

2 Preparations for Change...............................................................................................................3

2.1 Composition of Change Team and Responsibility of Team Members.......................................................................3
2.1.1 Customer Change Team.....................................................................................................................................3
2.1.2 Huawei On-site Change Team............................................................................................................................3
2.1.3 Huawei Support & Guarantee Team...................................................................................................................3
2.2 Remote Access+ Platform and Software Preparation................................................................................................4
2.3 Check of Equipment Running....................................................................................................................................4
2.4 Change Risks and Countermeasures..........................................................................................................................5
2.5 Check Before Change.................................................................................................................................................5

3 Operation Steps for Change........................................................................................................7

3.1 Overall Description of Change Steps.........................................................................................................................7
3.2 Operation Steps for Change.......................................................................................................................................7
3.3 Test and Verification...................................................................................................................................................8
3.4 Solution for Changeback In the Case of Failure......................................................................................................11
3.4.1 Definition of Change Failure.............................................................................................................................11
3.4.2 Changeback Steps.............................................................................................................................................11
3.4.3 Tests After Changeback.....................................................................................................................................11
3.4.4 Changeback Risk Analysis................................................................................................................................12
3.5 Change of Spare Parts and Emergency Workstation................................................................................................13

4 Work After Change.....................................................................................................................14

4.1 Observation..............................................................................................................................................................14

Issue ()
 IOH CR MOP Format V1.7

1 Description of Change

1.1 Description of Site

Network topology Please refer to the topology below;
diagram

Key features and Modify and adjust QoS level as mitigation if link congest occurred
business
Related business Traffic IP Access
amount

Issue ()
 IOH CR MOP Format V1.7

1.2 Purpose and Requirements

Change purpose* Modify and adjust QoS level as mitigation if link congest occurred
Change scope Change QoS level
Change time* yyyy-mm-dd 00:00 - yyyy-mm-dd 04:00

1.3 Description of Change and Change Influence

No.* NE Type* NE ID* Q Current version* Target version*
T
Y*

NE Name NE Type NE IP Address Software Version Patch Version

(MPU Type) List

No. Related NE name QTY Version

1 NA

Change Influence
Impact Risk Description
Performance Low, if wrong value on QoS level is configured will impact performance
degradation

Issue ()
 IOH CR MOP Format V1.7

2 Preparations for Change

2.1 Change Periode

The change period is dd mmm yyyy, 00:00 – 04:00 (Indonesia GMT+7.00)

2.2 Composition of Change Team and Responsibility of Team

Members
2.2.1 Customer Change Team
Name Responsibility Telephone No.
Sakim IOH VP NE Owner +628990233337
Muchlis Prasetyawan IOH Network Planner +628990969333
Bony Suryono IOH Network Operation +628159208233

2.2.2 Huawei On-site Change Team

Name Responsibility Telephone No.
Shawei TD MS +6285780121230
Roby Wibisono BO Datacom Manager +6289603002009
On Duty BO BO Datacom +62xxxxx
Dian Fawaik IM East Area (CJ, EJBN, SUL) +628997978175
Andrew Tosima FO Team / DNOC +6289654425451
Chandro Fresli Parulian Implementation Team +628999767888

2.2.3 Huawei Support & Guarantee Team

Name Responsibility Telephone No.
GTAC GTAC PSE 0078036015242 / 08001-080808 / 0062-21-5711378
(accessible through mobile call only) (Indonesia)

Issue ()
 IOH CR MOP Format V1.7

2.3 Remote Access+ Platform and Software Preparation

Type Content Owner
Category Detail Description Yudi Arfianto Imam
Remote access Login NE using customer network from NOC office for backup On Duty BO
solution last configuration
Remote access Remote access authorization : as BO datacom engineer we have On Duty BO
authorization authorization for login device
Remote access test Remote access test : day before will onsite team will try login On Duty BO
using console,
Platform, software SecureCRT for login NE, U2000 for Alarm monitor, On Duty BO
and license performance monitor, Cacti for monitor graph traffic, inspector
for backup configuration, Notepat++/ultra edit for compare
configuration.
Site permission Onsite team will apply enterance permit to TOC engineer and On Duty BO
IOH customer region
NE login BO datacomm / executor have authorization and privillege to On Duty BO
permission excute the change
Tools and SecureCRT, Notepad++ / ultraedit, inspector, eSpace, NCE On Duty BO
Softwares

2.4 Check of Equipment Running

Check Item* Check result or steps*
Health check Check all current existing connectivity -> existing connections status maintain
Check all existing device / hardware -> status alarm clear and all existing board
registered
Check all existing point to point service connections by do ping test -> all existing
service ping test result maintain no packet drop, no fo cut before execution.
Check all existing interface that already service -> no service interface down before
activity
Check all existing network protocol -> all running bgp peer at established state no
idle state,all running ospf peer occur and in full state without idle state,all running
mpls ldp peer occur in normal state,all running bfd peer in up status with no alarm
Check all service under related NE using NCE -> no traffic drop or degrade before
activity under related NE
Check all alarm appear under related NE using NCE -> no major alarm occur
activity under related NE

Issue ()
 IOH CR MOP Format V1.7

2.5 Change Risks and Countermeasures

No Main Issues and Risks Possibility Impact Countermeasure
. Description
1 Configuration lost/failure Low Medium - Call and info to Team Leader to explain
detail step what have done previously.
- Reconfigure port, or do
replacement/change port.

2.6 Check Before Change

No. Preparations Before Change Owner Completion Date
1 Change back solution has been prepared, On Duty BO 1 Day before
confirmed and authorized by customer * implementation
2 Change solution and change back solution have On Duty BO 1 Day before
been verified* implementation
3 Spare parts are ready for use within x minutes; if On Duty BO 1 Day before
no spare part, contingency plan is available * implementation
4 As per change scenario, collect data for recovery On Duty BO 1 Day before
following <XX product data collection guide>, implementation
and upload to iCare/eTrance *
5 Confirm has perform main/standby switchover test On Duty BO 1 Day before
implementation
6 Readiness of getting access permission to the site On Duty BO 1 Day before
implementation
7 Readiness of remote log-in environment On Duty BO 1 Day before
implementation
8 Confirm change risks and emergency measures On Duty BO 1 Day before
implementation
9 Confirm data and configuration are backup On Duty BO 1 Day before
implementation
10 Confirm network performance is tested and recorded On Duty BO 1 Day before
implementation
11 Readiness of change test personnel Chandro Fresli 1 Day before
Parulian implementation
12 Readiness of relevant equipment maintenance On Duty BO 1 Day before
personnel implementation
13 Readiness of support & guarantee team Chandro Fresli 1 Day before
Parulian implementation

Issue ()
 IOH CR MOP Format V1.7

No. Preparations Before Change Owner Completion Date

14 Readiness of mobile phone test card Chandro Fresli 1 Day before
Parulian implementation
15 Readiness of boards, tools, and instruments Chandro Fresli 1 Day before
Parulian implementation

Issue ()
 IOH CR MOP Format V1.7

3 Operation Steps for Change

3.1 Overall Description of Change Steps

No. Main steps Start time End time
1 Preparation before Operation 23:00 23:45
2 Service Confirmation 23:45 00:00
3 Change Procedure 00:00 02:00
4 Test Verification 02:00 02:30
5 Briefly Report After the Operation 02:30 03:00
6 Rollback Steps 03:00 03:30
7 Service Verification And Test 03:30 04:00

3.2 Operation Steps for Change

No. Change Step* Operation* Implementation Time*
* Person*
1 Log in to the BO Datacom remote login to NE using SecureCRT On Duty BO 23:00 --
NE by remote from NOC office +62xxxxx 23:45

2 Health check Datacom Health check performed 1 hour before On Duty BO 23:00 --
implementation : +62xxxxx 23:45
term len 0
admin show run
admin show install summary
admin show install active summary
admin show install inactive summary
admin show install committed summary
admin show platform
admin show environment fans
admin show environment led
admin show environment power-supply
admin show environment all

Issue ()
 IOH CR MOP Format V1.7

No. Change Step* Operation* Implementation Time*

* Person*
admin show diag details
admin show inventory all
admin show context location all
admin show redundancy
show run
show platform
show version
show version brief
show clock
show led
show redundancy
show diag details
show inventory
show hw-module fpd
show environment temperatures
show filesystem
show filesystem location. all
show media location all
show pfm location all
show process cpu
show watchdog memory-state location all
show logging
show alarms brief system active
show ipv4 interface brief
show ipv4 vrf all interface brief
show ipv4 vrf all interface bri | i Up
show interface | i "(line protocol| rate |
Description)"
show interface | i "(line protocol| error|
Description)"
show interfaces
show cdp neighbor
show bfd session
show bfd session detail
show ospf interface
show ospf neighbor
show mpls ldp neighbor
show mpls ldp bindings

Issue ()
 IOH CR MOP Format V1.7

No. Change Step* Operation* Implementation Time*

* Person*
show mpls ldp int brief
show mpls interfaces
show rsvp interface
show mpls traffic-eng tunnels
Show running-config
Show ospf neighbor
Show mpls ldp neighbor
Show mpls traffic-eng tunnel tabular
Show mpls traffic-eng tunnel role head
Show mpls traffic-eng tunnel role head detail
Show mpls traffic-eng tunnel role mid
Show mpls traffic-eng tunnel role mid detail
Show mpls traffic-eng tunnel role tail
Show mpls traffic-eng tunnel role tail detail
show bgp vpnv4 unicast summ
show bgp all all sum
show l2vpn xconnect
show route summary
show route vrf all summ
show route
show ipv4 vrf all interface
show run
check NMS Cisco, cacti
inspector tools
3 backup Save logging of current configuration On Duty BO 23:00 --
configuration Cisco#term length 0 +62xxxxx 23:45

4 Voice and Call NOC to check service voice and data before On Duty BO 23:45 --
data implementation +62xxxxx 00:00

5 Start verification : On Duty BO 00:00 --

Execution +62xxxxx 02:00
Script
show policy-map pmap-name qos-set-af4
show policy-map pmap-name qos-set-af3

adjust qos to level 4

policy-map qos-set-af4

Issue ()
 IOH CR MOP Format V1.7

No. Change Step* Operation* Implementation Time*

* Person*
class class-default
set qos-group 4
set mpls experimental imposition 4
!
end-policy-map
!
commit

adjust qos to level 3

policy-map qos-set-af3
class class-default
set qos-group 3
set mpls experimental imposition 3
!
end-policy-map
!
commit

binding to interace

interface x/y/z
service-policy input qos-set-af4
commit

3.3 Test and Verification

Change Validation
No. Test Item* Operation Method* Owner* Time*
*
1 Check Nokia Router Datacom need check after implementation : On Duty BO 02:00 --
show time +62xxxxx 02:30
show version
show chassis
show card

Issue ()
 IOH CR MOP Format V1.7

Change Validation
No. Test Item* Operation Method* Owner* Time*
*
show mda detail
term len 0
admin show run
admin show install summary
admin show install active summary
admin show install inactive summary
admin show install committed summary
admin show platform
admin show environment fans
admin show environment led
admin show environment power-supply
admin show environment all
admin show diag details
admin show inventory all
admin show context location all
admin show redundancy
show run
show platform
show version
show version brief
show clock
show led
show redundancy
show diag details
show inventory
show hw-module fpd
show environment temperatures
show filesystem
show filesystem location. all
show media location all
show pfm location all
show process cpu
show watchdog memory-state location all
show logging
show alarms brief system active
show ipv4 interface brief
show ipv4 vrf all interface brief

Issue ()
 IOH CR MOP Format V1.7

Change Validation
No. Test Item* Operation Method* Owner* Time*
*
show ipv4 vrf all interface bri | i Up
show interface | i "(line protocol| rate |
Description)"
show interface | i "(line protocol| error|
Description)"
show interfaces
show cdp neighbor
show bfd session
show bfd session detail
show ospf interface
show ospf neighbor
show mpls ldp neighbor
show mpls ldp bindings
show mpls ldp int brief
show mpls interfaces
show rsvp interface
show mpls traffic-eng tunnels
Show running-config
Show ospf neighbor
Show mpls ldp neighbor
Show mpls traffic-eng tunnel tabular
Show mpls traffic-eng tunnel role head
Show mpls traffic-eng tunnel role head
detail
Show mpls traffic-eng tunnel role mid
Show mpls traffic-eng tunnel role mid
detail
Show mpls traffic-eng tunnel role tail
Show mpls traffic-eng tunnel role tail detail
show bgp vpnv4 unicast summ
show bgp all all sum
show l2vpn xconnect
show route summary
show route vrf all summ
show route
show ipv4 vrf all interface
show run
check NMS Cisco, cacti

Issue ()
 IOH CR MOP Format V1.7

Change Validation
No. Test Item* Operation Method* Owner* Time*
*
inspector tools
inspector tools
and compare configuration before and after
using notepad++ / ultraedit
2 Voice and data (bts) Call NOC to check service voice and data On Duty BO 02:00 --
before implementation +62xxxxx 02:30
3 Collect operation logs Collect the operation logs of the redirection On Duty BO 02:00 --
storage that night, UI operation logs, and +62xxxxx 02:30
upload
4 Briefly report after the Briefly report after the operation On Duty BO 02:30 --
operation +62xxxxx 03:00

Issue ()
 IOH CR MOP Format V1.7

3.4 Solution for Changeback In the Case of Failure

3.4.1 Definition of Change Failure
Changeback time* failure shall be recognized if the following major problems cannot be resolved
before 02.30 am (local time)
Changeback decision TD Shawei (+6285780121230), project manager IOH : Sakim
maker* (+628990233337)
Changeback condition A change failure shall be recognized if the following major problems (port and
device alarm) and impact to service voice and data more than 15 minutes

3.4.2 Changeback Steps

No Changeback Operation content* Owner* Time*
.* Steps*
1 Get Rollback Call TD and Customer to get approval for On Duty BO 03:00 -- 03:05
approval rollback +62xxxxx
1 Data Collection / Backup alarm by command and from EPNM On Duty BO 03:05 -- 03:15
Backup config +62xxxxx
2 Rollback Rollback Script On Duty BO 03:15 -- 03:30
Configuration +62xxxxx
verification :

show policy-map pmap-name qos-set-af4

show policy-map pmap-name qos-set-af3

release config qos level 4 from interace

interface x/y/z
no service-policy input qos-set-af4
commit

release config qos level 3 from interace

interface x/y/z
no service-policy input qos-set-af3
commit

Issue ()
 IOH CR MOP Format V1.7

3.4.3 Tests After Changeback

Rollback Validation
No. Test Item* Operation Method* Owner* Time*
*
1 Check Nokia term len 0 On Duty BO 03:30 -- 04:00
Router admin show run +62xxxxx
admin show install summary
admin show install active summary
admin show install inactive summary
admin show install committed summary
admin show platform
admin show environment fans
admin show environment led
admin show environment power-supply
admin show environment all
admin show diag details
admin show inventory all
admin show context location all
admin show redundancy
show run
show platform
show version
show version brief
show clock
show led
show redundancy
show diag details
show inventory
show hw-module fpd
show environment temperatures
show filesystem
show filesystem location. all
show media location all
show pfm location all
show process cpu

Issue ()
 IOH CR MOP Format V1.7

Rollback Validation
No. Test Item* Operation Method* Owner* Time*
*
show watchdog memory-state location
all
show logging
show alarms brief system active
show ipv4 interface brief
show ipv4 vrf all interface brief
show ipv4 vrf all interface bri | i Up
show interface | i "(line protocol| rate |
Description)"
show interface | i "(line protocol| error|
Description)"
show interfaces
show cdp neighbor
show bfd session
show bfd session detail
show ospf interface
show ospf neighbor
show mpls ldp neighbor
show mpls ldp bindings
show mpls ldp int brief
show mpls interfaces
show rsvp interface
show mpls traffic-eng tunnels
Show running-config
Show ospf neighbor
Show mpls ldp neighbor
Show mpls traffic-eng tunnel tabular
Show mpls traffic-eng tunnel role head
Show mpls traffic-eng tunnel role head
detail
Show mpls traffic-eng tunnel role mid
Show mpls traffic-eng tunnel role mid
detail
Show mpls traffic-eng tunnel role tail
Show mpls traffic-eng tunnel role tail
detail
show bgp vpnv4 unicast summ
show bgp all all sum

Issue ()
 IOH CR MOP Format V1.7

Rollback Validation
No. Test Item* Operation Method* Owner* Time*
*
show l2vpn xconnect
show route summary
show route vrf all summ
show route
show ipv4 vrf all interface
show run
check NMS Cisco, cacti
inspector tools
2 voice and data Call NOC to check service voice and On Duty BO 03:30 -- 04:00
data before perform rollback steps +62xxxxx

3.4.4 Changeback Risk Analysis

No.* Main Issues and Risks Possibility* Impact* Countermeasure*
Description*
1 Change description cause low medium 1. Escalated to project TD (00631929,
missed configuration on Shawei +6285780121230) and
port. Supervisor less than 5minutes to
explain detail situation.

2. Parallel open SR emergency or

call GTAC standby (On-Duty GTAC
or call GTAC hotline
00862981770999) to get support.

3. Get approval from TD,

Management and Customer before
perform rollback/solution on step4.

4. Execute emergency plan for the

disaster recovery scenario following
GTAC guidance.

Issue ()
 IOH CR MOP Format V1.7

3.5 Change of Spare Parts and Emergency Workstation

Type Content Planned Time
Spare parts Any board fail, will directly request from warehouse Implementation Time

Issue ()
 IOH CR MOP Format V1.7

4 Work After Change

4.1 Observation
No. Operation Items: Operation method Owner Time Expected
Result
1 Keep monitoring Through NMS NOKIA FO Datacom dd mmm yyyy No new
alarm check alarm at router Andrew Tosima 02.00 – 04.00 alarm
Sulawesi
+6289654425451
2 Check Throughput Through Cacti Check On Duty BO dd mmm yyyy Throughput
throughput and traffic +62xxxxx 02.00 – 04.00 performance
pattern IP Sulawesi maintain
3 Monitor KPI Area Through NMS NOKIA NPM/ Dian dd mmm yyyy KPI is
and DCN monitor KPI Fawaik - 02.00 – 04.00 maintain.
after activity +628997978175 and
For next 24
hour

Issue ()