Huawei Optical Network Maintenance Reference-WDM ASON-20140826-C

8/9/2019 Huawei Optical Network Maintenance Reference-WDM ASON-20140826-C

1/116

Huawei Optical NetworkMaintenance Reference - WDMASON

Issue 02

Date 2014-08-26

Huawei Technologies Co., Ltd.


2/116

Issue 02 (2014-08-26) Huawei Proprietary and Confidential

Copyright © Huawei Technologies Co., Ltd

i

Huawei Technologies Co., Ltd. provides customers with comprehensive technical support and service.

Contact our local office or company headquarters.

Huawei Technologies Co., Ltd.

Address: Huawei Industrial Base

Bantian, Longgang

Shenzhen 518129

People's Republic of China

Website: http://www.huawei.com

Email: [email protected]

Copyright © Huawei Technologies Co., Ltd. 2011. All rights reserved.

No part of this document may be reproduced or transmitted in any form or by any means without prior

written consent of Huawei Technologies Co., Ltd.

Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.

All other trademarks and trade names mentioned in this document are the property of their respective

holders.

Notice

The information in this document is subject to change without notice. Every effort has been made in the

preparation of this document to ensure accuracy of the contents, but all statements, information, and

recommendations in this document do not constitute a warranty of any kind, expressed or implied.

Acknowledgement

This document is prepared and reviewed by the ASON R&D Maintenance Team, Information Development

Dept, Customer Support Dept, and Technical Support Dept together.

Editor:

Feng Junjie, Zhu Fei, Feng Haoyu, Wang Chaokai, Feng Chao, Liu Yuan, Zhang Meng, Zhou Yuxing,Zheng Fan, Li Weiping

Others:

ASON R&D Maintenance Team: Jiang Yi, Bai Zhongqiang, Li Qingsong

Information Development Dept: Fan Xiaoke, Pei Xin

Technical Support Dept: Dou Yongtan, Xie Bing

Customer Support Dept: Zhang Junguang, Fu Ming, Ma Qingquan

Quality Assurance Dept: Xue Xiuhua

Special acknowledgements to Jin Yuzhi, Mu Jianhong, Feng Zhigang, Wu Gang, Niu Shouchang, and

Chen Bin

http://www.huawei.com/http://www.huawei.com/mailto:[email protected]:[email protected]:[email protected]://www.huawei.com/


3/116

Huawei Optical Network Maintenance Reference -

WDM ASON About this Document



ii

About this Document

Overview This document mainly covers four aspects: preventive maintenance, quick service recovery

guide, fault diagnosis, and maintenance cases. It consists of the following chapters:

SOP for Maintaining NG WDM ASON DevicesThis chapter provides the inspection items and operation methods for preventivemaintenance of NG WDM ASON devices. The standard operating procedure (SOP)helps maintenance personnel discover and eliminate network risks during preventive

maintenance, therefore ensuring network stability and security.

Quick Recovery Guide for NG WDM ASON Services

This chapter provides handling measures for quickly recovering ASON services on NGWDM devices.

Fault Diagnosis for ASON Service Interruptions on NG WDM Devices

This chapter provides major methods and measures for diagnosing ASON service faults

on NG WDM devices.

Typical ASON Service Troubleshooting Cases

This chapter provides the typical ASON service troubleshooting cases for reference toguide maintenance personnel through troubleshooting of NG WDM devices.

FAQs

This chapter provides frequently asked questions about NG WDM ASON operation and

maintenance.

Change History

Issue Date Description

02 2011-08-26 Added “Precautions” section in “1 SOP for Maintaining NG

WDM ASON Devices”.

01 2011-09-29 This issue is the first official release.


4/116


WDM ASON Contents



iii

Contents

About this Document ..................................................................................................................... ii

1 SOP for Maintaining NG WDM ASON Devices .................................................................... 1

1.1 Introduction to SOP for ASON Maintenance ........................................................... ........................................ 2

1.2 ASON Device SOP Checklist ............................................................. ............................................................ 12

2 Quick Recovery Guide for NG WDM ASON Services ....................................................... 13

2.1 Overview ............................................................ .............................................................. .............................. 13

2.2 ASON Service Recovery Process ....................................................... ............................................................ 13

2.2.1 Intended Audience ................................................................................................................................ 13

2.2.2 Standard Operations for Routine Maintenance ............................................ ......................................... 13

2.2.3 Classification of Fault Causes .............................................................. ................................................. 14

2.2.4 Rerouting and Non-rerouting Conditions ........................................................ ...................................... 14

2.2.5 ASON Alarms ........................................................................... ............................................................ 15

2.2.6 General Principles for Recovering ASON Services .............................................................................. 16

2.2.7 Key Information for Recovering ASON Services .............................................................. ................... 16

2.2.8 General Flowchart for ASON Service Recovery ................................ .................................................. 17

2.3 AQuick Recovery Process for ASON Services ...................................................................................... ........ 17

2.3.1 Quick Recovery Process for ASON Services (Optical-Layer Service) ................................................. 17

2.3.2 Quick Recovery Process for ASON Services (OTN Electrical-Layer Service) .................................... 20

2.3.3 Quick Recovery Process for ASON Services (VC-4 Services) ..................................................... ........ 24

2.4 Troubleshooting Processes for Typical ASON Service Faults ..................................................... ................... 25

2.4.1 Troubleshooting Process for Route Computation Failures ............................................................ ........ 25

2.4.2 Troubleshooting Process for Rerouting Failures ................................................................................... 27

2.4.3 Troubleshooting Process for Unreachable NEs ....................................................... .............................. 28

2.4.4 Troubleshooting Process for Interruption of a Single-wavelength length ASON Service .................... 30

2.4.5 Troubleshooting Process for Interruption of Multi-Wavelength Services ............................................. 31

2.4.6 Troubleshooting Process for Malfunctioning SCC Boards ..................................... .............................. 33

2.4.7 Troubleshooting Process for a Failure to Recover Interrupted ASON Services Because of Add/Drop

Channel Faults ............................................................................................................................................... 34

2.5 Appendix: ASON Node Troubleshooting Process ............................................................ .............................. 35

2.5.1 General Principles for Handling ASON Node Faults ......................................................... ................... 35

2.5.2 Fault Description ............................................................ ............................................................... ........ 35

2.5.3 Procedure for Restoring Databases Using the Real-Time Database Backup Function ......................... 35


5/116


WDM ASON Contents



iv

2.5.4 Restoring the Configurations Manually .................................................................. .............................. 38

2.5.5 Restoration Process If No Backup Database Is Available ..................................................................... 40

3 Fault Diagnosis for ASON Service Interruptions on NG WDM Devices ....................... 41

3.1 General Fault Diagnosing Procedure ................................................. ............................................................ 41

3.1.1 Overview .............................................................. .............................................................. ................... 41

3.1.2 Flowchart .............................................................................................................................................. 42

3.1.3 Troubleshooting Procedure ........................................................................................................... ........ 42

3.1.4 Exception Handling Process ................................................................................................................. 46

3.2 Diagnosing the Fault that a Service Is Unavailable After Being Successfully Rerouted ............................... 46

3.2.1 Overview .............................................................. .............................................................. ................... 46

3.2.2 Flowchart .............................................................................................................................................. 46



3.3 Diagnosing the Fault that Service Rerouting Fails ........................................................... .............................. 48

3.3.1 Overview .............................................................. .............................................................. ................... 48

3.3.2 Flowchart .............................................................................................................................................. 49



3.4 Diagnosing the Fault that a Service Is Not Rerouted ................................. .................................................... 50

3.4.1 Overview .............................................................. .............................................................. ................... 50

3.4.2 Flowchart .............................................................................................................................................. 51



3.5 Common Operations Involved in Fault Diagnosis ......................................................................................... 52

3.5.1 Diagnosing OPA Adjust Failures .......................................................... ................................................. 52

3.5.2 Diagnosing Incorrect Attenuation Delivery of OPA ............................................................................. 54

3.5.3 Confirming Information About a Faulty Service ........... ............................................................... ........ 55

3.5.4 Obtaining Performance Data ................................................................ ................................................. 57

3.6 Information to Be Collected ........................................................................................................ ................... 57

3.7 References .......................................................... .............................................................. .............................. 58

4 Typical ASON Service Troubleshooting Cases .................................................................... 59

4.1 ASON-Specific Operations and Configurations ................................................................ ............................. 59

4.1.1 Case 1: Creating an ASON Service Fails Because the Wavelength for the Service Is Reserved .......... 59

4.1.2 Case 2: ASON NEs Reset Because of a Duplicated OSPF IP Address ................................................. 61

4.1.3 Case 3: An Attempt to Start the TE Link Management Window Times Out ......................................... 61

4.1.4 Case 4: Service Protection Level Fails to Change After a Line Board Participating in the Protection Is

Moved to Another Slot ............................................................ ............................................................... ........ 63

4.1.5 Case 5: Deleting Inter-NE Fiber Connections Fails When Separate Optical and Electrical NEs Are

Configured ..................................................................................................................................................... 65

4.1.6 Case 6: Route Computing Fails During Creation, Optimization, or Rerouting of an ASON Service or

During an Upgrade of a Static Service to an ASON Service ........................ ................................................. 66


6/116


WDM ASON Contents



v

4.1.7 Case 7: A Newly Created ASON Service Fails to Traverse a Node or an Existing ASON Service Fails

to Traverse a Node During Trail Optimization or Rerouting ........................ ................................................. 68

4.1.8 Case 8: LMP Protocol Check Fails Due to DCN Errors and Consequently Service Deployment Fails 69

4.1.9 Case 9: ASON Services Fail to Be Deployed Because Line Attenuation Is Excessively High ............. 71

4.1.10 Case 10: An Error Message Is Displayed When Users Attempt to Create Virtual TE Links .... .......... 72

4.1.11 Case 11: Route Computation Fails After Explicit Resources Are Specified to Create or Optimize an

ASON Service ............................................................... .............................................................. ................... 75

4.2 ASON Service Restoration .......................................................................................................... ................... 77

4.2.1 Case 1: An ASON Service Is Interrupted Because OPA Fails ....................................................... ........ 77

4.2.2 Case 2: An ASON Service Is Interrupted Because Protection Switching Fails After a Second Fiber Cut

....................................................................................................................................................................... 78

4.2.3 Case 3: An ASON Service Is Interrupted After Being Rerouted Because of Incorrect Fiber Connections

....................................................................................................................................................................... 79

4.2.4 Case 4: An ASON OCh Trail in a Slave Subrack Is Interrupted but Not Rerouted After the Slave

Subrack Is Powered Off ................................................................................................................................. 81

4.2.5 Case 5: An ASON Service Is Frequently Rerouted Among Multiple Trails.......................................... 82

4.2.6 Case 6: An Optical ASON Service Fails to Be Automatically Restored in Case of a Wavelength-Level

Fault ............................................................................................................................................................... 83

4.2.7 Case 7: An ASON Service Enabled with Scheduled Reversion Fails to Be Reverted to Its Original

Trail After the Scheduled Reversion Time Elapses ........................................................................................ 84

4.2.8 Case 8: Route Computation Fails When an ASON OCh Service Traverses a Regeneration Board ...... 84

4.2.9 Case 9: An ASON OCh Service Is Interrupted but Not Rerouted ......................................................... 86

5 FAQs .............................................................................................................................................. 90

5.1 Operations on the NMS ........................................................... ............................................................... ........ 90

5.1.1 How to Distinguish Between ASON Services and Traditional Services on the NMS? ......................... 90

5.1.2 How to Identify the First and Last Nodes of an ASON Service? .......................................................... 91

5.1.3 How to Identify the Original Trail and Preset Restoration Trail of an ASON Service? ........................ 92

5.1.4 How to Manually Optimize an ASON Service? ...................................................... .............................. 93

5.1.5 How to Manually Optimize ASON Services in Batches? ....................................... .............................. 94

5.1.6 How to Change a Wavelength to Optimize Trails? ..................................................................... .......... 95

5.1.7 How to Obtain Fiber Connections of Boards that an ASON Service Traverses? .................................. 95

5.1.8 How to Quickly Locate the Board Where an OCH_SER_INT Alarm Is Generated? ........................... 95

5.1.9 How to Quickly Query the Current Trail or Preset Restoration Trail of an ASON Service that Traverses

a Specific NE or Board? ................................................................................................................................ 96

5.1.10 How to Check Whether a Preset Restoration Trail Is Available? ........................................................ 96

5.1.11 How to Check Whether an Optical Cross-Connection Is Successfully Created for an ASON Service?

....................................................................................................................................................................... 96

5.1.12 How to Quickly Restore an Interrupted Service? ............................................................. ................... 96

5.1.13 How to Quickly Create Fiber Connections Between Sites? ........................................................ ........ 97

5.2 Configuration Rules ....................................................................................................................................... 97

5.2.1 What Are Common Attributes and Recommended Configurations for ASON Services? ..................... 97

5.2.2 What Are the Risks if ODU0, ODU1, and ODU2 ASON Services Are Concurrently Configured? ..... 97

5.2.3 What Are the Basic Rules for Configuring Preset Restoration Trails? .................................................. 97


7/116


WDM ASON Contents



vi

5.2.4 What Are the Recommended Configurations for the Preset Restoration Trails and Revertive Attributes

of SDH/OTN/WDM ASON Services? .......................................................... ................................................. 98

5.2.5 What Are the Restrictions on Regeneration Boards When Optical NEs and Electrical NEs Are

Separated and Why?....................................................................................................................................... 98

5.2.6 How to Ensure the Rerouting Function When No OA Board Is Configured Between an FIU Board anda WSS Board? ............................................................... .............................................................. ................... 99

5.2.7 Why Are TN52SCC Boards Instead of TN51SCC or TN11SCC Boards Recommended for ASON NEs?

....................................................................................................................................................................... 99

5.2.8 What Are the Rules for Configuring Node IDs for ASON NEs? .......................................................... 99

5.2.9 Why Must the Node ID and IP Address of an NE Be in Different Network Segments? ..................... 100

5.2.10 Why Does the LMP or OSPF Protocol Need to Be Disabled on Electrical Links on OTU Boards

Adding or Dropping WDM ASON Services? .............................................................................................. 100

5.2.11 What Are the Application Scenarios and Configuration Method for Resource Reservation? ........... 102

5.2.12 Why Does the LMP Protocol Need to Be Disabled for Optical Ports on Tributary Boards? ............ 103

5.2.13 How to Disable the LMP Protocol for the Optical Ports that Are Not Used by ASON Services? .... 103

5.2.14 How to Disable the OSPF Protocol for the Optical Ports that Are Not Used by ASON Services? ... 103

5.2.15 How to Split a Large DCN Subnet into Smaller DCN Subnets? ....................................................... 104

5.2.16 Do Diamond, Gold, Silver, and Copper ASON Services Support Hitless Conversion? ................... 105

5.3 ASON Principles ........................................................... .............................................................. ................. 105

5.3.1 What Overheads Are Used by Control Channels on the Control Plane? ............................................. 105

5.3.2 What Is the Difference Between the Menu Items "Revert To Port" and "Revert to Channel"? .......... 105

5.3.3 What Is the Difference Between Trail Overlap and Trail Sharing? ..................................................... 106

5.3.4 What Is Associated Sharing? ................................................................ ............................................... 106

5.3.5 What Is the Relationship Between SRLGs and Associated Services? .......................................... ...... 106

5.3.6 Why Is a Revertive Service Reverted to the Original Trail 5 Minutes After Rerouting and How to

Revert the Service to the Original Trail Within 5 Minutes? ........................................................ ................. 106

5.3.7 Does an OPA Adjust Failure Affect Rerouting of ASON Services? .................................................... 106

5.3.8 Why Cannot Revertive Services Be Downgraded After Rerouting? ................................................... 107

5.3.9 What Is the Difference Between the Function of Downgrading ASON Services in an NE Explorer and

the Function of Downgrading ASON Services in the Trail Management Window? .................................... 107

5.3.10 Do Service Optimization Must Be Performed at the First Node? ..................................................... 107

5.3.11 Why Do Services Fail to Be Reverted to the Original Trail? ............................................................ 107

5.3.12 Why Does Synchronization Between the NE and NMS Need to Be Performed During Each Query of

ASON Service Information? .................................................................................................................. ...... 108

5.3.13 Does a CPW_XXX_INT Alarm on the Control Plane Mean Service Interruption? ......................... 108

5.3.14 What Is the Difference in Database Backup and Restoration Between an ASON Network and a

Non-ASON Network? ............................................................. ............................................................... ...... 108

5.3.15 Why Do I Need to Periodically Check Whether Rerouted ASON Services Are Reverted to the

Original Trails? ............................................................................................................................................ 108

5.3.16 What Are Residual Cross-Connections and How to Delete Residual Cross-Connections? .............. 108

5.3.17 What Do the CPW_OCH_SER_INT and CPW_ODUk_SER_INT Alarms Mean?.......................... 109


8/116


WDM ASON 1 SOP for Maintaining NG WDM ASON Devices



1

1 SOP for Maintaining NG WDM ASONDevices

Precautions

ASON networks must be appropriately planned and designed so that the network isrobust and services can survive against multiple fiber cuts. For the detailed process, seeFigure 1-1.

The expansion of links, subracks, wavelengths, or ODUk channels of an ASON networkmust be planned, designed, and deployed according to the related creation procedure (seeFigure 1-1). Otherwise, the robustness of new services cannot be ensured, and even thesecurity of the existing services may be affected.

Normal ASON network running and service security depend on meticulous routine

maintenance and periodical comprehensive assessment and optimization.


9/116





2

Figure 1-1 ASON network delivery and maintenance process

Purpose

This chapter provides the SOP for preventive maintenance personnel so that they can discover

and eliminate network risks, thereby ensuring network stability and security.

Intended Audience

System maintenance personnel

ApplicationDevice maintenance personnel perform the standard operations and activities for preventive

maintenance at suggested intervals.

1.1 Introduction to SOP for ASON Maintenance

Table 1-1 lists the SOP for preventive maintenance of ASON.


10/116





3

Table 1-1 SOP for preventive maintenance of ASON

PreventiveMaintenance Item

TimeRequired

Frequency

InspectionMethod

Procedure Priority Purpose

Item Sub-Item

ASON

databases

Checking

backupdatabasesfor ASON

NEs

5 min/NE NE

databasesmust be backed up

once

networkchanges

occur.

Manual 1. Verify that

the NMS cansuccessfully back up NE

databases at

specifiedintervals.

It is

recommendedthat the NMS

start backingup databasesfor

network-wide

NEs at 2 a.m.every day.

2. After a

networkchange

occurs, for

example,many services

are deployedor reroutingoccurs,

request thecustomer to

arrange a time

window formanually

backing updebases for all

ASON NEs

on thenetwork.

Minor To ensure

that userscandownload

the NE

database torecover the

node once a

node faultoccurs onthe

network.

ASON

resources

Checking

the status

of control

links

5

min/100

links

Monthly Manual 1. On the

NMS client,

navigate to

the ASONcontrol link

managementwindow andsynchronize

the controllink

information

network-wide.

Major To ensure

that control

links are in

normalstate. If the

control linktopology isincorrect,

you maynot be able

to create,

optimize,


11/116





4


TimeRequired

Frequency

InspectionMethod


Item Sub-Item

2. Checkwhether

isolated NEsare present in

the controllink topology

view. If there

are isolated NEs, pinpoint

the cause andrestore the NEs to

normal state.In addition,

check for

alarms on theisolated NEs

and clearthem one by

one.

3. Check

whetherabnormal

alarms are

generated onthe controllinks. If thereare abnormal

alarms, locate

the boardsthat report the

alarms andclear thealarms one by

one.

4. Export the

control linkinformationinto an excel

file. Comparethe controllink

informationwith this

information

next time you perform the preventive

maintenance,

and check

delete, orreroute an

ASONservice.


12/116





5


TimeRequired

Frequency

InspectionMethod


Item Sub-Item

whether thecontrol links

are the samein the two

inspections.

Checkingvirtual TE

links

2 min/onelink

Monthly Manual 1. On the NMS client,

navigate tothe ASON TE

linkmanagement

window and

synchronizethe TE linkinformationnetwork-wide.

2. Check the

value of

Extend Type of TE links. Ifthe value isnot

Automaticall

y Discovered,check for

alarms on the

source andsink boards ofthe TE links.If alarms have

been

generated onthe source and

sink boards,

clear thealarms

according tothe NMS

online help.

Major To ensurethat virtual

TE links areavailable.

Checking

status of

TE links

5

min/100

links

Monthly Manual 1. On the

NMS client,

navigate tothe ASON TE

link

managementwindow andsynchronizethe TE link

information

Major To ensure

that TE

links are up.If alarms

are

generatedon TE links,recovery ofASON

services


13/116





6


TimeRequired

Frequency

InspectionMethod


Item Sub-Item

network-wide.2. Check

whetherabnormal

alarms are

generated onthe TE links.

If there are

abnormalalarms, locate

the boardsthat report the

alarms andclear thealarms one by

one.

3. Check thevalue of Link

Status of the

TE links. Ifthe value is

not Up, check

for alarms onthe boards

where the TElinks aredown. Then

clear thealarms.

4. Export the

TE linkinformationinto an excel

file. Comparethe control

link

informationwith this

informationnext time you perform the

preventivemaintenance,

and checkwhether the

TE links are

the same inthe two

inspections.

will beaffected.


14/116





7


TimeRequired

Frequency

InspectionMethod


Item Sub-Item

Checkingfor

residualcross-conn

ections

5min/onecross-con

nection

Monthly Manual Checkwhether a

CPW_XXXX _TEL_PATH

MIS alarm is

generatedthroughout

the network.

If there is aCPW_XXXX

_TEL_PATHMIS alarm,

clear itaccording tothe NMS

online help.

Major To ensurethat there

are noresidual

cross-conne

ctions. TheCPW_XXX

X_TEL_PA

THMISalarm may

affectrecovery of

ASONservices.

ASON

services

Checking

ASON

services

10 min

/network

Daily Manual 1. On the

NMS client,

navigate tothe ASONtrailmanagement

window; then

synchronizeASON trail

information.

2. Check

whetherASON trails

are activated.If an ASON

trail is

displayed asInactive,

check whethera clientservice is sent

to the ASONtrail. If no

client service

is sent to theASON trail,

no furtheraction is

required. If a

client serviceis sent to the

ASON trail,

contact the

Critical To ensure

that ASON

services arenormal.


15/116





8


TimeRequired

Frequency

InspectionMethod


Item Sub-Item

customer forfurther

confirmationand take

records of theconfirmation

result.

3. Check

whetherrerouting

lockout isdisabled for

ASON trails.If it isdisabled,

contact thecustomer to

check why itis disabledand take

records.

4. Check

whetheralarms are

generated onthe ASONtrails. If yes,

clear themaccording to

the NMSonline help.

5. Export theASON

informationinto an Excel

file and save

it for futurereference.

Alarms Clearing

control

planealarms

5 min/one

alarmMonthly Manual Check

whether an

alarm startingwith "CP" or

"CPW" isgeneratedthroughout

the network.

If there issuch an alarm,

clear it

Major To ensure

that there is

no control plane alarm.

A control plane alarmmay disable

ASON

servicesfrom

running


16/116





9


TimeRequired

Frequency

InspectionMethod


Item Sub-Item

according tothe NMS

online help.

normallyand may

evendirectly

affectASON

trails.


17/116





10


TimeRequired

Frequency

InspectionMethod


Item Sub-Item

ASONevents

Checkingfor

abnormalASON

events

20min/50

ASONservices

Monthly Manual 1. Checkwhether an

ASON servicererouting

failure has

occurredlately. (To

verify this

information,click new

events icon inthe main

topology ofthe NMS,then browse

the events inthe Browse

Events Logs

– [New

Events]

window.)

2. Check

whether anASON service

re-creationfailure hasoccurred

lately. (Toverify this

information,click new

events icon inthe maintopology of

the NMS,

then browse

the events inthe Browse

Events Logs

– [New

Events]window.)

Critical To monitornetwork

operation. Ifan ASON

rerouting or

re-creationfailure is

reported

frequentlyduring a

time period,identify the

cause andtakecorrespondi

ngmeasures.


18/116





11


TimeRequired

Frequency

InspectionMethod


Item Sub-Item

Inspectionof potential

risksaccording to

pre-warning

notices

Checking potential

risks based on

pre-warni

ng notices

30 min

/network

Monthly Manual Download pre-warning

notices fromthe

http://support.

huawei.comwebsite and

perform the

workarounds, preventive

measures, orsolutions

provided inthe notices.

Major To remove potential

risksaccording to

officially

released pre-warning

notices.

Preventive

maintenance inspection

Preventive

maintenance

inspection

60

min/NEMonthly Tool +

manual

1. Perform

preventivemaintenance

inspection

(PMI) ofASONnetworksusing a PMI

tool

(download thelatest tool

from the

http://support.huawei.comwebsite) and provide the

PMI result to

Huawei HQfor filing.

2. Analyze the

PMI resultaccording tothe PMI

guide. If thereare any

problems,

rectify themimmediately.

Major To ensure

that theASON

resources

and serviceson anASON NEare in good

condition.

http://support.huawei.com/http://support.huawei.com/http://support.huawei.com/http://support.huawei.com/http://support.huawei.com/http://support.huawei.com/http://support.huawei.com/http://support.huawei.com/


19/116





12

1.2 ASON Device SOP Checklist

Site: ____________ Version: ___________ Maintenance Owner: __________ Time: _____________

PMI Item PMI Sub-Item Result Remarks

ASON databases Checking backup databases for

ASON NEs□ OK □ NOK □ POK

ASON resources Checking the status of control

links□ OK □ NOK □ POK

Checking virtual TE links □ OK □ NOK □ POK

Checking status of TE links □ OK □ NOK □ POK

Checking for residual

cross-connections□ OK □ NOK □ POK

ASON services Checking ASON services □ OK □ NOK □ POK

Alarms Clearing control plane alarms □ OK □ NOK □ POK

ASON events Checking for abnormal ASON

events□ OK □ NOK □ POK

Checking potential

risks based on pre-warning notices

Checking potential risks based

on pre-warning notices□ OK □ NOK □ POK

Preventive maintenance

inspection

Preventive maintenance

inspection□ OK □ NOK □ POK


20/116


WDM ASON 2 Quick Recovery Guide for NG WDM ASON Services



13

2 Quick Recovery Guide for NG WDMASON Services

2.1 OverviewThis chapter provides guidelines for recovering ASON services from service interruption ornode faults on an ASON network by assuming that the ASON network has been in operation,

aiming to guide the network maintenance personnel through the service recovery and faultdiagnosis processes.

This chapter assumes that the network maintenance personnel are skilled in fault diagnosis for

traditional WDM services on transport equipment and therefore focuses on the differences for

maintaining the ASON-capable WDM transport equipment and the traditional WDM transportequipment.

2.2 ASON Service Recovery Process

2.2.1 Intended Audience

This chapter is intended for the following personnel:

Customer personnel: members of the operator's maintenance team. They are responsiblefor routine maintenance of the ASON and handling of common faults.

Customer service personnel: members of Huawei's GTS team. They provide technical

support for the customer, and assist the customer in handling network faults. R&D personnel: members of Huawei's R&D team. They assist the customer service

personnel in handling network faults.

2.2.2 Standard Operations for Routine Maintenance

To ensure high survivability of the ASON, find the potential faults and problems occurring onthe ASON as early as possible, and promptly recover ASON services from faults, the network

maintenance personnel must maintain the ASON in a routine manner according to Routine

Maintenance Operation Guide. Particularly, they must at least take the following actions:

1. Perform preventive maintenance regularly using the SmartKit Inspector.

2. Back up the databases regularly.


21/116


22/116





15

Domain Rerouting Occurs When No Rerouting Occurs When

OCH optical

layer

MUT_LOS or BD_STATUS

alarms are generated on FIU boards.

LOS alarms are generated only on SC2

boards.

MUT_LOS alarms are generated only onOA boards.

LOS, SSF, DEG, or EXC alarms aregenerated only on OTU boards.

The service is locked.

No other trails are available (the fiber is broken) for the affected service.

TE link check fails (for example, the TE

link is degraded or broken).

There are discrete cross-connections.

The control link is unreachable and a CL

DOWN alarm is reported for the route.

Note: After a service is rerouted, it will not be rerouted again in case of a channel alarm(an alarm on an OA or OTU board).

2.2.5 ASON Alarms

Among all ASON alarms, the following alarms have direct impact on ASON services:

1. Service interruption alarms: CPW_OCH_SER_INT, and CPW_ODUk_SER_INT

2. Link down alarms: CPW_OCH_TEL_DOWN and CPW_ODUk_TEL_DOWN

An OTU board reports an R_LOS, OTUk_SSF, DEG, or EXC alarm or an FIU board reports aMUT_LOS or BD_STATUS alarm when one or multiple services are abnormal.

The following ASON alarms have no direct impact on ASON services:

3. Link control alarms: CPC_CC_DOWN, CPC_OSPF_CL_DOWN, andCPW_OMS_TEL_DEG

These ASON alarms are generated when an SC2 board reports an R_LOS alarm or theRES overhead of an OTU board is unavailable. When the alarms are generated,verification of control links or TE links will fail.

4. Resource alarm: CPW_XXX_TEL_PATHMISThe alarm is usually generated when there are discrete cross-connections. In other words,

the alarm is generated when a cross-connection is configured only at one end for a fiberlink between two sites or the cross-connection at one end is in use while the

cross-connection at the other end is not in use.

5. The root causes for other ASON alarms (for example, CPW_CLNT_SER_NOTOR,CPW_OCH_SER_SLADEG, CPC_NODE_ID_CONFLICT, andCPC_RSVP_NB_DOWN) are not related to traditional alarms, nor have direct

relationship with interruption of ASON services. For details about the causes andmethods for handling them, refer to the alarm reference manual.


23/116





16

2.2.6 General Principles for Recovering ASON Services

The symptoms and causes for interruption of different ASON services vary greatly. To reduce

the downtime and promptly restore an interrupted service, the following steps are generallyused:

1. Locate the interrupted service, and optimize the service onto the preset restoration trail orrevert the service back to the original trail (or switch the service to the protection trail)

after confirming the trail is available.

2. If the preceding step fails, deactivate the service. Then create the service as a traditionalservice.

3. Check for traditional alarms that have caused the service interruption, and clear them.

The preceding steps are three essential actions for promptly recovering ASON services.

For the detailed service recovery procedure, see the next section.

To determine whether a trail is available, users can check whether an end-to-end trail is

available based on the network topology, fiber interruption symptoms, ASON topology

(TE link and control link), and service source or sink node.

NOTE

While restoring an interrupted service, confirm the following information:

Check whether any configurations are changed before or after the service is interrupted.

These configurations include protocol parameter settings (OSPF, LMP, and RSVP parameters) that

affect the functions of the control plane, port attribute settings (port loopback, FEC/AFEC mode, and port rate mode) that are related to the service, and service configurations (for example, service trail

configurations on the client side).

Check whether there is any service-affecting major alarm after the service is interrupted.

For example, check whether there is a new hardware damage alarm (HARD_BAD), fiber breakalarm (R_LOS and MUT_LOS), traditional service interruption alarm (LOF, and AIS), or ASON

service interruption (XXX_SER_INT).

2.2.7 Key Information for Recovering ASON Services

After an ASON service is interrupted, troubleshooting personnel must obtain the key

information about the service fault, because the key information is very important for preliminary fault diagnosis and prompt service recovery. The key information includes:

1. When the ASON service is interrupted?

Method: The troubleshooting personnel can determine the time either from the customerfeedback or by checking the alarm occurring time.

2. Whether a fiber is broken on the link for the interrupted service?

Method: In the main topology or signal flow diagram of the NMS, check whether a fiberis broken. If the fiber is red, then the fiber has been broken.

3. What alarms (ASON or traditional alarm) are reported for the interrupted service?

Method: Check whether there are any new major alarms.

4. Does the fault occur in an ASON or traditional service?

Method: Check for service alarms directly in the WDM ASON Trail Management and

WDM Trail Management windows of the NMS. The trail for the interrupted service is

red.

5. If multiple services are interrupted, do they all carry client services? Which services arethe most important? Are all interrupted services in the same direction?

Method: Directly confirm this information with the customer.


24/116





17

6. Which services are interrupted (source/sink nodes and boards)? What are the service protection levels? Which trails does the service traverse? Do the interrupted services pass

any regenerators?

Method: View the information directly in the WDM ASON Trail Management window.

Click Original trail and Current trail in the lower left part of the window, and take thescreenshots for the required information.

7. Has rerouting been triggered for the interrupted service? Has the rerouting succeeded orfailed?

Method: Directly click Event to view the detailed event information (alternatively, selectthe specific event screenshot).

8. Has any network configurations been modified, has any services been added or deleted,or has service cutover been performed before the fault occurs?

Method: Directly confirm this information with the customer or maintenance personnel.

2.2.8 General Flowchart for ASON Service Recovery

Figure 2-1 General flowchart for ASON service recovery

2.3 AQuick Recovery Process for ASON Services

2.3.1 Quick Recovery Process for ASON Services (Optical-LayerService)

Step 1 Locate the trail for interrupted services.


25/116





18

1. According to the customer feedback, determine when service interruption occurs, howmany services are interrupted in addition to the source and sink nodes and fiber status

(for example, fiber break) of the network.

2. Choose Configuration > WDM ASON > WDM Trail Management to display the

WDM ASON trail management window. Identify the interrupted ASON service trailaccording to the service source or sink node information and ASON service alarms. If

the CPW_OCH_SER_INT alarm is generated, clear the alarm to recover the services.

3. If no ASON service interruption alarm is reported, click in the lowerright of the Browse Current Alarms window. After alarm synchronization is complete,check whether an ASON service interruption alarm is reported. If there is such an alarm,clear the alarm to recover the services.

4. If no service interruption alarm is reported after alarm synchronization is performed,

filter the interrupted services according to their source or sink nodes. Check whether


26/116





19

there are inactive services. If there are inactive services, check whether they are thefaulty services. If yes, activate the services to recover them.

5. If no ASON service interruption alarm has ever been reported, check for traditionalalarms that are generated on the source or sink NE during the service interruption time period as specified by the customer. Then determine the affected boards and

corresponding ASON service trails according to the alarm information.

NOTE

For optical-layer services on WDM devices, only BD_STATUS or MUT_LOS alarms on FIU boards can

trigger rerouting. In other words, alarms on OTU boards cannot trigger rerouting.

Step 2 Restore the interrupted services.

Route computation for optical-layer services is very complex because it must consider manyfactors such as optical parameter constraints, regenerators, and optical power. Therefore,Huawei uses preset restoration trails to recover ASON services at the optical layer of NGWDM equipment. A maximum of two preset restoration trails can be configured for each

silver service. When a silver service at the optical layer is interrupted, the best choice to

quickly recover the service is to optimize the service onto one of the preset restoration trails,or revert the service back to its original trail.

The following describes how to quickly recover an interrupted service:

1. In the WDM Trail Management window, select the service and click .Then save the service information (for example, the service attributes, current trail, and

original trail) as an excel file. The information will be subsequently used as reference forfault diagnosis and service recovery.

2. In the WDM Trail Management window, select the service. Check whether presetrestorations are configured for the service. If they are configured, take the screenshotsfor the two preset restoration trails, because this information can be used as reference for

reconfiguring preset restoration trails for the service in future fault diagnosis. In addition,

check whether the current trail of the service is the same as one of the preset restorationtrails. Ensure that the preset restoration trails are explicit trails during service

optimization.

The following figure shows a screenshot for one of the preset restoration trails.


27/116





20

3. If no restoration trail is configured for the service, optimize the trail for the service.During trail optimization, try to select a trail that has common nodes as the original trailand specify it as the explicit trail for the service. Then check whether the service is

recovered. If a failure message is displayed or if a success message is displayed but the

service is not recovered, go to the next step.

4. If the service fails to be optimized or reverted back to the original trail, check forconfiguration errors (such as rerouting lockout and incorrect optical parameter settings,

link verification failure/unavailability of control links not triggered by alarms). Then

assess the current network topology, fiber status (for example, fiber breaks), and servicesource or sink node. If idle trails are available for the service, deactivate the service and

then try to recover the service as a traditional service by creating a WDM trail for it. Ifthe service still fails to be recovered, go to the next step.

5. Query the traditional alarms corresponding to all ASON service trails. If there are sometraditional alarms (for example, R_LOS, MUT_LOS, VOADATA_MIS, andOPA_FAILED), clear the traditional alarms according to the traditional alarm handling

process. Then check whether the service is recovered. If not, go to the next step.

6. Contact Huawei for troubleshooting.

NOTE

For WDM ASON services, disable the function of computing optical parameters if the optical

parameter check fails when you are optimizing the service. Then optimize the service again. To disable the optical parameters, right-click the service in the ASON trail management window, and

choose Disable Optical Parameters from the shortcut menu.

----End

2.3.2 Quick Recovery Process for ASON Services (OTNElectrical-Layer Service)

Step 1 Identify the interrupted services.


28/116





21

1. According to the customer feedback, determine when service interruption occurs, howmany services are interrupted in addition to the source and sink nodes and fiber status

(for example, fiber break) of the network.

2. Choose Configuration > WDM ASON > WDM Trail Management to display the

WDM ASON trail management window. Identify the interrupted ASON service trailaccording to the service source or sink node information and ASON service alarms. If a

CPW_ODU3_SER_INT, CPW_ODU2_SER_INT, CPW_ODU1_SER_INT, or CPW_ODU0_SER_INT is generated, clear the alarm to recover the services.

3. If no ASON service interruption alarm is reported, click in the lowerright of the Browse Current Alarms window. After alarm synchronization is complete,

check whether an ASON service interruption alarm is reported. If there is such an alarm,

clear the alarm to recover the services.

4. If no service interruption alarm is reported after alarm synchronization is performed,select and right-click all services and choose Synchronize from the shortcut menu.Check whether there are inactive services. If there are inactive services, then NMS datais inconsistent with the NE data. Activate these services if required. If activating these

services fails, refer to Step 2.


29/116





22

5. If no ASON service interruption alarm has ever been reported, ensure that the staticcross-connections are configured correctly for the client side of the NE. If alarms have

been generated on the client side, ensure that the client-side boards are free of any faults

or are online. Then, check for alarms and performance events on OTU boards. If thereare alarms, locate the affected ASON services and rectify the faults.

Step 2 Restore the interrupted services.

1. In the WDM Trail Management window, click Maintenance to sequentially refreshthe current trail, original trail, and preset restoration trails. Then select the service and

click . Then save the service information (for example, the service attributes,current trail, and original trail) as an excel file. The information will be subsequentlyused as reference for fault diagnosis and service recovery.

2. In the WDM Trail Management window, select the service. Check whether presetrestorations are configured for the service. If they are configured, take the screenshots

for the two preset restoration trails, because this information can be used as reference forreconfiguring preset restoration trails for the service in future fault diagnosis. In addition,

check whether the current trail of the service is the same as one of the preset restorationtrails. Ensure that the preset restoration trails are explicit trails during service

optimization.

The following figure shows a screenshot for one of the preset restoration trails.


30/116





23

3. If no restoration trail is configured for the service, optimize the trail for the service.During trail optimization, try to select a trail that has common nodes as the original trail

and specify it as the explicit trail for the service. Then check whether the service is

recovered. If a failure message is displayed or if a success message is displayed but theservice is not recovered, go to the next step.

4. If the service fails to be optimized or reverted back to the original trail, check forconfiguration errors (such as rerouting lockout, link verification failure/unavailability ofcontrol links not triggered by alarms). Then assess the current network topology, fiber

status (for example, fiber breaks), and service source or sink node. If idle trails are

available for the service, deactivate the service and then try to recover the service as atraditional service by creating a trail for it. If the service still fails to be recovered, go to

the next step.

5. Query the traditional alarms corresponding to all ASON service trails. If there are sometraditional alarms (for example, R_LOF, R_LOS, LOM, and BUS_ERR), clear thetraditional alarms according to the traditional alarm handling process. Then check

whether the service is recovered. If not, go to the next step.


31/116





24

6. Ty to deactivate/activate the ASON service. If the service is recovered after thedeactivation/activation, reconfigure the service attributes and the preset restoration trails

for the service based on the service information and screenshots saved previously.

----End

2.3.3 Quick Recovery Process for ASON Services (VC-4 Services)Locate the interrupted VC-4 ASON service according to the control plane alarms. Determinewhether the alarms are an internal network alarm (CP_SER_INT) or an external networkalarm (CP_SRV_INT_OUT).

1. For an external network alarm, locate and handle the service faults on the client side. Forexample, check whether the service receiving boards are faulty or offline and whether

errors have been generated in client signals.

2. For an internal network alarm, handle the alarm as instructed in section 2.3.2 "QuickRecovery Process for ASON Services (OTN Electrical-Layer Service)".


32/116





25

If no control plane alarm indicating service interruption is reported, check for correspondingtraditional alarms according to the service interruption time, and clear them to recover the

services.

2.4 Troubleshooting Processes for Typical ASON ServiceFaults

This section provides the troubleshooting processes for typical ASON service faults (for

example, fiber breaks, device software or hardware faults that cause ASON serviceinterruption) so that maintenance personnel can promptly recover the services by isolating the

faults according to the troubleshooting processes.

2.4.1 Troubleshooting Process for Route Computation Failures

Fault Description

An ASON service (OCH, ODUK, or SDH) is interrupted, and the following alarms arereported for the end-to-end ASON service trail and the port on the WDM side:

End-to-end ASON service alarms:

CPW_OCH_SER_INT, CPW_ODUk_SER_INT, and CP_SER_INT

Port alarms on the WDM side of affected boards:

R_LOS, OTUk_LOF, OTUk_SSF, OTUk_LOF, OTUk_LOM, OUTk_DEG, ODUk_PM_SSF,

ODUk_PM_OCI, ODUk_PM_DEG, IN_PWR_HIGH, IN_PWR_LOW, R_LOF, AU_AIS,

BD_STATUS, MS_AISEnd-to-end ASON service event: route computation failure event

Workarounds

According to the fault occurrence time and the customer feedback, identify the interrupted

services and promptly recover the services by, for example, optimizing the services orreverting them to their original trails. In events of a route computation failure or optical

parameter check failure, preferentially check whether there are sufficient network resources. Normally, many factors can cause a shortage of network resources. They include:

configuration errors, fiber breaks, residual cross-connections, device hardware faults, node

faults, and software bugs. Currently, rerouting protection is provided for optical-layer services

using preset restoration trails. If a service is interrupted and cannot be automatically recovered because of a route computation failure, manually recover them using the following methods:

1. Find the current ASON service trail (including the preset restoration trail) of the service,and deactivate the service.

2. Create a static service over a traditional WDM trail and recover the service usingtraditional commissioning methods. For an optical-layer service, try to use the original

trail or configure two preset restoration trails.

NOTE

If the preceding methods cannot recover the service, isolate the fault according to the symptoms (for

example, fiber breaks, residual cross-connections, hardware faults, node faults, and software bugs) andthen recover the service. For details about the troubleshooting procedure, refer to XXX Fault Location

and Handling Procedure.


33/116


34/116





27

2.4.2 Troubleshooting Process for Rerouting Failures

Fault Description

An ASON service (OCH, ODUK, or SDH) service is interrupted. For details about the servicealarms of the end-to-end ASON service trail and WDM port, see section 2.4.1

"Troubleshooting Process for Route Computation Failures."

End-to-end ASON service event: rerouting failure event

Workarounds For "Creation timeout": If timeout is caused by traffic congestion or a communication

failure at the source or sink node, the service cannot be recovered by just optimizing the

service to another trail. Instead, you have to deactivate/activate the service, or re-createan end-to-end traditional service.

For "RSVP egress or ingress port is down", explicitly specify different egress ports tooptimize the service onto another trail so as to quickly recover the service.

The system returns the error prompt "Failure of label allocation". If the faulty node is the

source or sink node of the ASON service, attempt to recover the interrupted service byoptimizing the tunable wavelength for the optical layer. For the non-tunable NEconfigurations, identify the resources that conflict with the cross-connections or

wavelengths of the current service and delete the conflicting resources to recover theinterrupted service.

Troubleshooting Procedure

Step 1 Review the ASON service events that are generated during the time period when the fault

occurs, and check whether the "Creation timeout", "RSVP egress or ingress port is down" or"Label allocation failure" message is displayed.

Step 2 If the "Creation timeout" message is displayed, check whether network nodes successfullycommunicate with each other. The nodes fail in control plane communication if the

CPC_OSPF_AUTH_ERR, CPC_OSPF_NB_DOWN, CPC_RSVP_AUTH_ERR, or

CPC_RSVP_NB_DOWN alarm is generated on the control plane. To restore the control planecommunication, clear the alarm. If the network nodes communicate successfully with each

other, check whether traffic congestion has occurred at the source node, sink node, or anintermediate node of the service using the following methods:

1. Run the nbb-set-debug-mode:open command to the node to enable the debug functionof the node.

2. Run the mpls-query-rsvp-tcevent:0.0.0.0,0.0.0.0,0,1,2 command five times at intervalsof 3 seconds.

If the same triplet information is displayed in the command output every time, then traffic

congestion has occurred at the node.

If traffic congestion has occurred at the source or sink node, deactivate/activate the service or

perform a soft reset on the node to rectify the fault. If traffic congestion occurs at anintermediate node, recover the service by optimizing it to an available trail, then perform a

warm reset on the node to rectify the fault. At this point, the "creation timeout" fault is

rectified.

Step 3 If the "RSVP egress or ingress interface Down" error message is displayed, first determine the

faulty node according to the rerouting failure event. Then check whether the interface control


35/116





28

block of the control plane is in normal state and whether the RSVP interface information iscomplete using the following methods:

1. Run the mpls-get-if:all command to query the interface information. Check whetherthere is an interface corresponding to the interrupted service and whether the interface

information is correct (DEFINED=0x7, ADMIN=UP, OPER=UP; the remoteinformation is not null).

2. If the interface information is incomplete, runthe :mpls-debug-command:get_lim_if,0,0,0,0,0; command to recover the interface

information or perform a warm reset on the SCC board of the NE.

At this point, the "RSVP egress or ingress interface Down" fault is rectified.

Step 4 If the "Failure of label allocation" error message is displayed, check whether there areconflicting resources on the network. Handle the fault using the following methods:

1. Check whether there are any reserved resources or residual cross-connections on the preset restoration trails or specified explicit trails.

2. Check whether there are any resource inconsistency alarms (for example,CPW_XXX_TEL_PATHMIS).

If there are some reserved resources or residual cross-connections, manually release the

reserved resources or delete the residual cross-connections.

At this point, the label allocation failure is resolved.

----End

2.4.3 Troubleshooting Process for Unreachable NEs

Fault Description

An ASON service (OCH, ODUK, or SDH) is interrupted, and the corresponding NE is

dimmed in the main topology of the NMS) and an NE unreachable alarm is reported for the

NE. As a result, users cannot operate and configure the NE. For details about the possible

service alarms generated by the end-to-end ASON service trail and port on the WDM side, seesection 2.4.1 "Troubleshooting Process for Route Computation Failures."

Workarounds

After the ASON service is interrupted and the NE is unreachable by the NMS, identify the

cause. If the NE is unreachable by the NMS because the gateway NE is faulty, check whether

the secondary gateway is available or reconfigure another gateway NE for the unreachable NE.Then try to recover the service using the Navigator if the NE can be managed by the

Navigator. The recovery steps are as follows:

Step 1 Query alarms on the add/drop boards and the source/sink nodes of the affected services.

Step 2 According to the source or sink node of the interrupted service, run a command to query anddetermine the corresponding affected ASON service (determine the affected ASON service

according to the add/drop service board and ASON service alarm).

Step 3 Recover the interrupted service quickly be optimizing the service (or reverting the serviceover to the original trail) manually.


36/116





29

NOTE Run the following command to query service alarms:

:alm-get-curdata-ext:0,ALL,0

Run the following commands to query ASON services:

:mpls-get-och-smlsp:src (optical layer)

:mpls-get-otn-smlsp:src,all (ODUk electrical layer)

:mpls-get-smlsp:src (SDH)

Run the following commands to optimize ASON services:

:mpls-optim-och-lsp (optical layer), including the wavelength tuning command:

mpls-adjust-och-lsp

:mpls-optim-otn-lsp (ODUk electrical layer)

:mpls-optim-lsp (SDH)

mpls-restore-och-lsp (manually revert optical layer ASON services), mpls-restore-och-origrt (revert network-wide optical layer ASON services)

mpls-restore-otn-lsp (manually revert ODUk services), mpls-restore-otn-origrt (revert

network-wide ODUk services)

mpls-restore-lsp (manually revert SDH services), mpls-restore-original-rt (revert network-wideSDH services)

Recover the service quickly by optimizing the service or reverting the service over to the original

trail.

Step 4 If neither the NMS nor the Navigator is available but trails are available for the service, youcan disable the SC2 and OA laser of the reachable node on an available trail to trigger service

switching, or perform a cold reset on the FIU and remove the line fiber to trigger servicererouting, finally restoring the interrupted service.

----End


An NE may become unreachable by the NMS for many reasons: IP address conflict, subnetmask errors, OSC/ESC communication fault between NEs, SCC hardware or software faultsof the NE, ECC storm, and faults of the customer's outband DCN. You can simply

troubleshoot this fault by determining whether the NE becomes unreachable by the NMS

because of the fault of the gateway NE. If yes, replace the faulty gateway NE. For other typesof faults, refer to the DCN Recovery Guide.

NOTE

To check for network communication faults, run the following commands:

:cm-get-iproute (query the routing table):cm-set-ip (modify the IP address)

:cm-set-submask (modify the subnet mask)

:cm-get-gateway (modify the gateway NE)


37/116





30

2.4.4 Troubleshooting Process for Interruption of aSingle-wavelength length ASON Service

Fault DescriptionAn ASON service (OCH, ODUK, or SDH) service is interrupted and alarms are reported. For

details about the service alarms of the end-to-end ASON service trail and WDM port, see

section 2.4.1 "Troubleshooting Process for Route Computation Failures."

In the ASON service management window, only a single ASON service is interrupted. You

can determine whether the service initiates the rerouting process according to thecorresponding performance event (rerouting success or failure event) and service alarm

(ASON service interruption alarm, "not over the original trail" alarm, and rerouting lockoutalarm). The possible symptom of service interruption is that the rerouting process is not

initiated or the ASON service is still interrupted even after the rerouting process is initiated.

Workarounds

For details about the workarounds, see section 2.4.1 "Troubleshooting Process for Route

Computation Failures."

NOTE

If the interrupted single-wavelength ASON service does not initiate the rerouting process, it is possible

that no trail meets the requirement or a component (for example, board or fiber) is faulty (the currentASON service only supports the rerouting switchover triggered by a line-class fault rather than achannel-class fault).


Step 1 According to the fault information (for example, fault time, faulty node, and current alarm) provided by the customer, determine whether a single ASON service or multiple ASONservices are interrupted through the MUT_LOS alarm and the R_LOS alarm of multiple

boards. If a single ASON service is interrupted, proceed to the next step.

Step 2 If the ASON service has the protection capability (traditional protection plus ASONassociated service), check whether the service is locked to an abnormal channel forcibly. Ifyes, unlock the service and check whether the service is recovered. If a normal channel is

available, switch the service toward the normal channel forcibly. Alternatively, check whetherthe switching conditions are configured correctly (SD). If not, correct the switching condition

configurations and check whether the service is recovered. If the service is still not recovered,

proceed to the next step.

Step 3 Check the interrupted ASON service. If rerouting is locked for the ASON service, unlockrerouting. The ASON service may still fail to initiate the rerouting process (it is possible that a

channel alarm causes the service interruption; if the service has an available trail, optimize theservice manually), or the ASON service is still interrupted even after the ASON serviceinitiates the rerouting process (by default, an optical-layer alarm does not trigger service

rerouting). In this case, proceed to the next step.

Step 4 According to the interrupted service information, check whether the service trail must traversea repeater, and check whether the current trail has traversed a repeater. If the current trail does

not traverse a repeater, you can specify a repeater to optimize the service for restoring theservice. If the service trail does not need to traverse a repeater or the current trail has traverseda repeater, but the ASON service is still interrupted, proceed to the next step.


38/116





31

Step 5 Determine whether a service of the same wavelength causes any crosstalk. You can disablethe OTU at the source or sink node of the interrupted service or disable the laser of the trunk

board to determine whether another service board generates any same-wavelength crosstalksection by section. Alternatively, ask the customer's maintenance personnel to check whether

the wavelength of a board is adjusted at the OTU of the source or sink node, on a trunk board,

or in an intermediate site of the service, or whether any service board is inserted. According tothe check results, adjust the conflicting wavelength to recover the interrupted service. If theASON service is still interrupted, or there is no conflicting wavelength, proceed to the nextstep.

Step 6 Check whether the OTU configurations at the source node are consistent with those at the sinknode and whether the trunk board configurations (for example, wavelength, rate, FEC, andTDC) between two ends are consistent with each other. If some configurations are

inconsistent, correct the configurations immediately. If the configurations are all consistent

but the ASON service is still interrupted, proceed to the next step.

Step 7 Check whether a power adjustment alarm is generated for the service trail, for example, anOPA adjustment failure alarm (OPA_FAIL_INDI). If yes, check the corresponding

configuration information (including the WSS board, optical amplifier, and preset insertionloss) of the OPA reference section. If there are some configuration errors, correct the

configurations as instructed by the System Commissioning Guide. If no adjustment failure

alarm is generated, or if the corresponding configurations are correct but the ASON service isstill interrupted, proceed to the next step.

Step 8 Determine the current trail for the interrupted service. Check the power of the OTU (includingthe trunk board) at the source or sink node of the single-wavelength service. Then query thecurrent and historical performance of the OTU at the receiving or transmitting end. If the

current network is configured with an MCA board, use the MCA board to scan the

single-wavelength power in the service trail and check whether any power is abnormal in theservice trail. If the current network is not configured with any MCA board, check whether the

receiving power of the corresponding port is normal (performing a hardware loopback) on site.If a faulty fiber or board is identified, replace the faulty fiber or board. Then, the ASON

service is recovered.

NOTE

When checking the faulty trail, note the following alarms and configuration information:

Multiplexer or demultiplexer board alarm and optical amplifier board alarm: RLOS, MUT-LOS,

BD_STATUS, MODULE_ADJUST_FAIL, MOD_COM_FAIL, WAVEDATA_MIS, IN_PWR_LOW,IN_PWR_HIGH, OUT_PWR_HIGH, and OUT_PWR_LOW

Multiplexer or demultiplexer board and optical amplifier board configurations: insertion loss,

attenuation, rated power, gain, and dispersion compensation.

Rectify a board fault (if available) as instructed by the Board Replacement Guide, and rectify a fiber

fault (if available) as instructed by the Fiber Repair Guide.

----End

2.4.5 Troubleshooting Process for Interruption ofMulti-Wavelength Services

Fault Description

ASON services (OCH, ODUK, or SDH) are interrupted and alarms are reported. For detailsabout the service alarms of the end-to-end ASON service trail and WDM port, see section

2.4.1 "Troubleshooting Process for Route Computation Failures."


39/116





32

In the ASON service management window, multiple ASON services are interrupted. You candetermine whether the service initiates the rerouting process according to the corresponding

performance event (rerouting success or failure event) and service alarm (ASON service

interruption alarm, "not over the original trail" alarm, and rerouting lockout alarm). The possible symptom of service interruption is that the rerouting process is not initiated or the

ASON service is still interrupted even after the rerouting process is initiated.

Workarounds

For details about the workarounds, see section 2.4.1 "Troubleshooting Process for Route

Computation Failures."


Step 1 According to the fault information (for example, fault time, faulty node, and current alarm) provided by the customer, determine whether a single ASON service or multiple ASON

services are interrupted by checking the related alarms, for example, the MUT_LOS alarmand the R_LOS, IN_PWR_HIGH, or IN_PWR_LOW alarm of multiple OTUs. If multipleASON services are interrupted, proceed to the next step.

Step 2 Check each interrupted ASON service. If rerouting is locked for the ASON service, unlockrerouting. if the ASON services initiate the rerouting process and are recovered, the fiber onthe line side is faulty. Therefore, rectify the line fault. If the ASON services still fail to initiate

the rerouting process or the ASON services are still interrupted even after the ASON servicesinitiate the rerouting process, proceed to the next step.

Step 3 If the ASON services are still interrupted even after the rerouting or switching process isinitiate

Huawei Optical Network Maintenance Reference-WDM ASON-20140826-C

Documents

Transcript of Huawei Optical Network Maintenance Reference-WDM ASON-20140826-C