© Nokia Siemens Networks Company Confidential 1 (7)

Technical Support Note

TS-RNC-SW-162

RNW plan download failures which can block further configuration management operations Radio Control lers

WCDMA RNC

RN4.0, RN5.0, RN6.0

This document contains following type of information

Informative

Preventive

Corrective X

Urgent

Security

Information is classified as

Internal

Public

Customer Specific

© Nokia Siemens Networks Company Confidential 2 (7)

Table of Contents 1. Keywords........................................................................................................................................3 2. Summary ........................................................................................................................................3 3. Detailed description........................................................................................................................3

3.1 RAKTOR hand gets stuck during RNW Plan activation ....................3 3.2 RNW Plan operation times out during download/pre-activation ........4

4. Solution / Instructions .....................................................................................................................5 4.1 Releasing RAKTOR’s hanging plan hands .......................................5 4.2 Splitting large RNW Plans into smaller parts.....................................6

5. Note................................................................................................................................................6 6. References .....................................................................................................................................6

Contact:

Contact your local Nokia Siemens Networks support.

Summary of changes:

10-Feb-2011 1.0 First version 24-Mar-2011 2.0 Clarifications to Chapter 4.1

© Nokia Siemens Networks Company Confidential 3 (7)

Purpose This document contains generic information about products. These can be instructions that explain problem situations in the field, instructions on how to prevent or how to recover from problem situations, announcements about changes or preliminary information as requirements for new features or releases.

1. KEYWORDS

RNC, RNW Plan, WCEL, RAKTOR, modify

2. SUMMARY

Two kinds of problems have been observed to happen occasionally in RNC when operator makes modifications to RNW configuration by using an RNW Plan,

1. RNW configuration Manager’s (RAKTOR) hand might get stuck during Plan activation and further attempts to plan operation fail in NetAct with reason “RNW PLAN DOWNLOAD FAILED: NO FREE HANDS AVAILABLE”

2. RAKTOR hand may get killed by the RNC recovery system due to too high CPU usage and because of this the RNW Plan operation (download/pre-activation) may fail in NetAct due to reason “NO REPLY FROM RNW MANAGER PROGRAM BLOCK (RAKTOR, RNC OMU) TO RNW PLAN MANAGER”

The problem appears most likely with large RNW configurations or when RNW Plan includes invalid data. Incorrect workaround for the failure situation where RAKTOR hand gets stuck has been given at least in List of Generic Faults report in December. That workaround causes in the worst case RNC outage.

3. DETAILED DESCRIPTION

3.1 RAKTOR hand gets stuck during RNW Plan activation

When operator tried to make modifications to RNW Plan by using a RNW Plan, RNW configuration Manager’s hand (RAKTOR, Family ID: 0x4EE and Process ID e.g. 0x010 in RN5.0) might get stuck. The problem appears most likely with large RNW configurations or when RNW Plan includes invalid data. While deleting WCELs by using RNW Plan, first RNW Plan Manager locks WCELs which are not blocked by user (means in Administrative unlocked state), and after that WCELs and their adjacencies are deleted from acting database. During WCELs locking, RNW configuration Manager’s hand got stuck. In the error situation “RAK no free hand” log writing appears to OMU computer log. . In NetAct the plan operation is seen as failed due to reason “RNW PLAN DOWNLOAD FAILED: NO FREE HANDS AVAILABLE”.

Incorrect workaround for that failure situation has been given at least in List of Generic Faults report in December. [LGF-RNC-2010-12.pdf, Chapter 1.7 After NetAct plan download 100% cells as BL(SM)] and Resolve case workarounds. That workaround causes in the worst case RNC outage.

© Nokia Siemens Networks Company Confidential 4 (7)

Wrong workaround which is forbidden to be used in any circumstances:

Solution / Workaround: Restarting of RAK process will free reserved resources and recovers it back to normal situation. Performed RAKTOR program block restart as follows. 1. Destroy RAK family: ZOD:4EE 2. Rebuild RAK family: ZOBT:RAK 3. Restart RAK family: ZLP:Q,SQQ then ZQRR:4EE

Following is the re-commended order of creation/modification and deletion used for RNW Configuration Management.

3.2 RNW Plan operation times out during download/pre-activation

RNC performs RNW database consistency check during Plan download/pre-activation phase. In case the RNW Plan contains very large amounts of modifications (e.g. over 10000 WCEL+ADJx objects) the consistency check may take quite a long time and consume heavily RNC OMU unit CPU resources. In this case RNC unit level supervision may terminate the RAKTOR Plan hand that is performing the consistency check.

This case does not disturb RNC behavior or prevent further RNW Plan operation attempts, but the ongoing download/pre-activation gets timed out and NetAct shows the status as “NO REPLY FROM RNW MANAGER PROGRAM BLOCK (RAKTOR, RNC OMU) TO RNW PLAN MANAGER”.

© Nokia Siemens Networks Company Confidential 5 (7)

4. SOLUTION / INSTRUCTIONS

While performing the workaround for the releasing of RAKTOR plan hand, please don’t use any simultaneous operation for same WBTS/WCEL and their adjacencies either by using Object Browser (Application Launcher/Parameter Editor of OMS) or by direct activation which are going to be modified by using RNW Plan.

4.1 Releasing RAKTOR’s hanging plan hands

If RNW Configuration Manager’s Plan hand creation fails during plan operation start due to “NO FREE HANDS AVAILABLE” reason; following work-around can be used to release the hanging hand. Notice that these instructions are valid only for RAKTOR’s group 2 hand, not for other hands.

a) Start remote session to WO-EX OMU: ZDDS:OMU;

b) Load FAMUTI extension:

ZLP:1,FAM

c) Check RAKTOR group 2 hand information: Z1H:4EE

Example output from RN5.0: HAND FO:PREV NEXT TIME GR STATE STABITS JBUFFER RCOMP FAM PROC FO ... 0010 00 0010 0010 00001980 02 Reserved -------- 0000.00000000 0000 0000 0000 00

Check the hand ID (HAND) from the line where GR value is 02. STATE column may indicate hand to be “Reserved” or “Free”. (In RN4.0 the hand ID for group 02 is 000B and in RN5.0 & RN6.0 the hand ID is 0010.)

d) Send "hand process frozen" message to RAKTOR master in the name of the plan hand so

that master recovers it. ZOSD:C000,4EE,<hand>,0,*,7002,4EE,,,1,,8515,,, (in general) E.g. ZOSD:C000,4EE,B,0,*,7002,4EE,,,1,,8515,,, (in RN4.0) E.g. ZOSD:C000,4EE,10,0,*,7002,4EE,,,1,,8515,,,(in RN5.0 and RN6.0)

e) In case the plan hand was in “Reserved” state, verify using FAMUTI that the hand is now free. Otherwise continue to step (f).

Z1H:4EE

Example output from RN5.0: HAND FO:PREV NEXT TIME GR STATE STABITS JBUFFER RCOMP FAM PROC FO ... 0010 00 0010 0010 0002CDCE 02 Free ---W-L-- 0000.00000000 0000 0000 0000 00

f) Return to MML: ZE

After this the RAKTOR plan hand is no longer reserved, and plan activation operation can be reattempted.

© Nokia Siemens Networks Company Confidential 6 (7)

4.2 Splitting large RNW Plans into smaller parts

Before RN5.0 MP3 installation the RNW Plan can be split into e.g. 3 smaller RNW Plans so that WCELs and their adjacencies are modified in parts. With this solution the RNW Configuration Manager’s plan hand getting killed because of too high CPU usage may be avoided during RNW plan download/pre-activation phase. Further splitting of the RNW Plan parts may be required if the same problem is repeated.

This workaround is applicable if operator is facing the RNW Plan download/pre-activation timeout problem while using big RNW Plan.

5. NOTE

Final correction for the root cause (RAKTOR plan hand getting stuck) is part of RN5.0 MP3. Before that following workarounds are available.

6. REFERENCES

NA04777824: RAK data error NA04815379: RNC failure when trying to activate plans from NetAct NA04838614: OMU-0 can not work NA04733173: Planactivation for LAC-change takes much to long NA04858956: RNW PLAN DOWNLOAD FAILED: NO FREE HANDS AVAILABLE NA04855712: Impossibility to run plans with CM Operation Manager in RNC NA04824845: OMU 0 faulty.

© Nokia Siemens Networks Company Confidential 7 (7)

Disclaimer

The information in this document is subject to change without notice and describes only the product defined in the introduction of this documentation. This documentation is intended for the use of Nokia Siemens Networks customers only for the purposes of the agreement under which the document is submitted, and no part of it may be used, reproduced, modified or transmitted in any form or means without the prior written permission of Nokia Siemens Networks. The documentation has been prepared to be used by professional and properly trained personnel, and the customer assumes full responsibility when using it. Nokia Siemens Networks welcomes customer comments as part of the process of continuous development and improvement of the documentation.

The information or statements given in this documentation concerning the suitability, capacity, or performance of the mentioned hardware or software products are given “as is” and all liability arising in connection with such hardware or software products shall be defined conclusively and finally in a separate agreement between Nokia Siemens Networks and the customer. However, Nokia Siemens Networks has made all reasonable efforts to ensure that the instructions contained in the document are adequate and free of material errors and omissions. Nokia Siemens Networks will, if deemed necessary by Nokia Siemens Networks, explain issues which may not be covered by the document.

Nokia Siemens Networks will correct errors in this documentation as soon as possible. IN NO EVENT WILL NOKIA SIEMENS NETWORKS BE LIABLE FOR ERRORS IN THIS DOCUMENTATION OR FOR ANY DAMAGES, INCLUDING BUT NOT LIMITED TO SPECIAL, DIRECT, INDIRECT, INCIDENTAL OR CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITED TO LOSS OF PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY OR DATA, THAT MAY ARISE FROM THE USE OF THIS DOCUMENT OR THE INFORMATION IN IT.

This documentation and the product it describes are considered protected by copyrights and other intellectual property rights according to the applicable laws.

The wave logo is a trademark of Nokia Siemens Networks Oy. Nokia is a registered trademark of Nokia Corporation. Siemens is a registered trademark of Siemens AG.

Other product names mentioned in this document may be trademarks of their respective owners, and they are mentioned for identification purposes only.

Copyright © Nokia Siemens Networks 2011. All rights reserved.