Controlling Shared Memory Contention on a iLIultiprocessor-based

Telephone Switch

Naveenta Verma, B.Eng.

A thesis stibniitted to the Faculty of Graduatr Studies and Resexch

in partial fulfillnlent of the rrqiiirenirnts for the degree of

Master of Engineering

Ottawn-Carleton Institute for Electrical Engineering C

Füculty of Engineering Department of Systems and Cornpliter Engineering

Carleton University Ottawa. Ontario. canada K 1 s 5B6

January ZOO0

@ Copyright 2000. Naveenta Vema.

National Library I*I of Canada Bibliothèque nationale du Canada

Acquisitions and Acquisitions et Bibliographie Services services bibliographiques

395 Wellington Street 395. rue Wellington Ottawa ON K 1 A ON4 Ottawa ON K I A ON4 Canada Canada

Your file Votre relerenu,

Our fi& Notre relérence

The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive permettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or sel1 reproduire, prêter, distribuer ou copies of this thesis in microfonn, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/film, de

reproduction sur papier ou sur format électronique.

The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fiom it Ni la thèse ni des extraits substantiels may be printed or othewise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation.

Abstract

This thesis is concemrd uith the performance of a sccilablc mu l t iprocessor based

tclephone sa1itch. Simulation rnodcling is an effective technique for understanding the

pcrhrnliinceh of large and cornples systems suçh as multiprocessor systrms. A simulation

progrmi for the clil 1 control module of a mu1 tiproccssor-basrd sysrrm htis bern designrd.

dewlopsti. and tested for this tlirsis study. Ciipocity anrilysiz of thc multiproctl.;sor rystsni

i \ Jonc and the performance of the systcm itndcr iliffcrent rvorltload\ is premmd. The

multiproceisor systrm comprises of thrce iliffercnt types of procehm rhüt a n run in

pariillrl: piiyloüd i thrit d o e cal1 proçessing fiinctions i rind tivo non-payloxi types i ont:

h;indline maintenance related açiivitich and the othrr hanillin- opcrüiiond functionsi.

Contention occurs whcn multipk processes try to accrss the same rliartxi rnemory line. .At

thu tinir. one proce33 conilnues rsecution and the other one rolls h x k deprnding iipon

the ~lcci'ion made hy a rollh~ick alyoriihni. Thrw rollbxk writcgies: Buundcd-Priority

Sclicnir. Age-Bascd Rollhack Schenir and R«llb:ick-Payload Schcnie ai-e sirnullitsci and

their iinpxth on the systern cripacity art2 an:ilyzrd. The simtil;itur spstrrn is writtrn in C++

usin? PROPHET ( a Nortrl proprictiiry simulation tool). One of the major contributions of

the thrsis is the observation that the combination olchnrging rollbacks to class and the

rollback strritegies trstrd in the system breaks the CPU shares dlocüted by the sch~dulrr.

.\ho. out of the thrre rollback stratrgies. the boundrd-priori ty rollbxk schemct exhibits

the bsst sçalability. A tool for estimiiting rnernory contention drscribed in this thesis %as

highly ussful to Nonel Networks for capncity riniilysis.

Acknowledgements

This reseiuch has brrn succrssful due to partial funding from ClTO (Communication and

Information Technology of On tario 1.

1 would likr to thank Professor Shikharesh Majumdx for his continued suidance and

super\%ion of this thesis. 1 would also l i k rd tnkc an opportunii) i o thana Brian Carroil

and Dale Streibr.! from Nonrl Networks who hliw alwti's bwn 3 wurce of esçellcnt hclp

and inspiration for me and rny work throughout the course ol'thih thesir. The challen~rt

and mthusiiihrn of workine on this rrsrarch uould not h n c been ho much fiin without the

support ot'staff and Pxilities at Sorte1 Nr.r~vr.orks.

My hcmfclt ihlinks ro rny dad. mom. hihtcr and her ftiniily i r i Otiai~ii who hiive alwciys

becn thrrt. h r me. encourriying nie üt evc?. singlc siep and puhhing mc a -h rnwsr 1 1211

brumblrd or disappointcd wiih anythin" at a11 during this çoiirse of tirné. 1 wouidn'i have

been able to n i a k i t to [hi' day nithout thém 311.

1 u.ould likc. to thank rny lovinp husband. Vishril. I thmk him for k i n g estrrnirly

understandin2 and patient whrn 1 couldn't spcind our nr.\vly marrird da!.; with him in Son

Francisco. due to my thcsis uork in Ottawa. Also I would like tci give the crrdit of

succrssfiil completion of my thesis to him sincr hr made my life simpler by moving to

Ottawa making sure 1 don't =ive up. Thiinks for bclievin_o in me t i l l the 1;ist moment.

Finally. 1 wouid like to dedicaie rny thesis to my guru: my rnothcr. Chündsr Vrmri. 'You

have bren my id01 since my childhood and the best telicher I have swr corne across in my

lik. l'ou are the best!'

Table of Contents

1 . Inrroduction ................................................................................................................... 1 1.1 Motivation .............................................................................................................. 1 1.2 Related Work .......................... .. ........................................................................ 3 1.3 Contributions .......................................................................................................... 5 1 . 4 Outline of the thesis ............................................................................................... 7

7 A Multiprocessor based Telephone Switch .................................................................. 9 2.1 Ciipücity nrrds on cxisting node ............................................................................ 9 2.2 Objectives of rnovine to a Multiprocessor-bascd Architecture ............................ 10 2.3 XA-Corekchitecturt: .......................................................................................... 1 1 7 - -.A 1 Types of processes ......................................................................................... 13

2.4 Mrthod of Opçration ............................................................................................ 14 2.5 Mrmory Oumrship ............................................................................................. 16

2.5.1 Blocking ....................................................................................................... 17 2.5.2 RoIlbaciks and Cornmits ................................................................................ 1S 2.5.3 Schedulingofproccssrs ................................................................................ 21

......................................................................................................... 3 Sirnulritor Mocit-l 24 3.1 Client-Server klodsl ................ ..... ................................................................. 2 1 3 .2 iModelin_o with PROPHET ................................................................................... 26 3.3 Scheduler .............................................................................................................. 29 3.4 Inputs o f the Sysrcn~ ............................................................................................. 31

3.4. 1 PROPHET Confiyrrition .............................................................................. 3 1 .............................................................................................. 3.4.2 Mcmory Trîççs 34

.......................................................................................... 3.5 Outputs of the System 34 1 - ........................................................................................................... 3.6 Architecture 7s

....................................................................................... 3.7 Programrninp Pxtdigin 37 ............................................................................................ 3 .Y S in~uIrition Sotiwcir e. 39

4 Rollback Policies ........................................................................................................ 40 .......................................................................................................... -1.1 Introduction JO

4.2 Age-Bwd Rollback Sçhemc ............................................................................... 41 4.3 Boundcd-priority Rollbnck Policy ....................................................................... 42

..................................................................... 4.4 Rollbück-Payload Priority Schrmr 4 3 ................................................................................................ 4.5 RollbacksCharges 44

.............................................................................. 1.6 Rollbacks Chxged to Systern 44 ................................................................................. 4.7 Rollbiicks Charsed ro Class 45

4.8 C h q e to Class chosrn for this study ............................................................... 46 ............................................................................ 5 Results of Simulaciun Experiments 47

............................................................................................ 5.1 Performance Metrics 47 5.2 Mode1 Validation .............................................................................................. 50

5.2.1 Validation Technique 1 ................................................................................ 50 ................................................................................. 5 2 . 2 Validation Technique 2 51

5.2.3 Validation Technique 3 ................................................................................. 52 5.23.1 Sensitivity Andysis 1 ................................................................................. 57

................................................................................. 5.2.3.2 Sensitivity Analysis 2 57

5.3 Results of Esprrimznts ........................................................................................ 62 5.3.1 Scenario 1 ................................................................................................... 63

.............................................................................................. 5.3.1.1 Discussion 65 5.3.2 Scenario 2 ...................................................................................................... 65

5.3 2 . 1 Discussion ........................ ..... ........ ,,. ................... 6 9 5.3.3 Scenario 3 .................... ..., ..................................................................... 70

.................................................................................................. 5.3.3.1 Discussion 74 ......................................................................... 6 Payload and Non-Payloiid contention 77

.................................................... 6.1 Contention brrwesn paylorid and non-payload 77 6.2 The qnp hctor ...................................................................................................... 78

................................................................................ 6.2.1 Algorithm for ?rio script 79 - 6 - 2 2 1 h r "qnp" ~Mrawrinp Tool ............................................................................ 81

.................................................................................. 6.2.3 Rrcalculatcd qcp value 82 7 Conclusions ................................................................................................................ 84

........................................................................................................... 7 . 1 Discussion S-I ............................................................................ 7.1. 1 Rollbacks Chiirging Policy 34

7 . 1 . 1 Sclilahility of the sysiem ................................................................................ 86 7.1.3 Cornparison of Rollhack Pulicics .................................................................. S7

7.7 Scopci of Fut~irc Reselirch ..................................................................................... 88 .................................................................................. 7.3 M y ideris ro future research 89

................................................................................................... 7.3.1 Algorirhml Y9

................................................................................................... . 7.3.2 Algorithni 2 90 ........................................................................................................................... .Append ix 94

List of Figures

Figure 1 : Shared memory architecture of a multiproçessor-based zystem ........................ 12 Fisure 1: Illustration of bloc king in the Shared mrmory ................................................. 20

7 - Figure 3: Illustration of Commit and Rollback in the Shared Memory architecture ......... -z 3 - Figure 1: Clisnt/Server Mode1 ........................................................................................... -3

Figure 5: Systrm Modcling Priradigm in t r m s of PROPHET .......................................... 17 Figure 6: Graphical Repressntation of a Scheduler ........................................................... 29 Figure 7: CPU shares by Scheduler Classes ...................................................................... 30

. .. Figure 8: The PROPHET Configuration tile ..................................................................... 32

............ Figure 9: Simulation resulrs for sensitivity anitlysis I showino - ( a ) UPP ( b ) R A P 5 5 Figure 10: Simulation results for srnsitivity analysis ?. showino_ ( a ) UPP ( b ) RAP .......... 56 Figure 1 1 : Simulation rrsults for Scenario 1 shouing (<i l RAP . (b) UPP ......................... 67 Figure 12: Simulation Rrsults for scrnario 3 showhg i a 1 RAP (h UPP ......................... 71 Figure 13: Siniulation Rrsiilts for sccnario 3 showing i a ) R.AP ( b 1 CPP ......................... 76 Figure 14: O\wliip of d m domain of priyloüd S: non-pavload proccss running in

piiraIlel ........................................................................................................................ 79 Figure 15: The rffect of deçrease in value of qnp on R.AP ............................................... 8-3 Figure 16: Class Diiigritm of object cllisses used in the zimuliiror ................................. 94

vii

List of Tables

Table I : Decision Table For the Ase-Brised algorithm ...................................................... 41 Table 7: Drcision Table for the Bounded-Priority Al=onrhm .......................................... 43 Table 3: Decision table for the Rollback-Payload cilgorithm ............................................ 45 Table 4: Results of Validation Technique 7 ................................................................... 51 Table 3: Simulation Rrsults for Srnsitivity Analysis 2 for 3 PEs ..................................... 39 Table 6: Simulation results for Sensitivity Andysis 7 for 5 PEs ...................................... 59

.................. Table 7: 9lr;isured CPU shüres for diffrrent rollback policies for xrnario 1 63

.................. Table 9: .M rasured CPU shares for different rollhacl; policie tor cenar io 3 72

1

Chapter 1

Introduction

Ptrforniiince rinalysis ;inci resourcc nimagernenr o l niulriproicwv- sptcinr :irc iiriport;int

arecis ro b r csplorecl i n cichirving a high capaiity sy,tem. Al1 cimlpi~tiiig qstt'nih arc

continually ch;illenged to deliver tiighsr procttssin~ c;ipticii> 10s iliiproving the rcsponse

of existing applicriiions cind rctiiizing nciv application.\ of Iiiplicr simplcsiiy. In [hi';

e

rcht.;trch ive cire deriling wiih the crill cunirol niodulr. of ;i i&pli~~tic i i i . i t ~ h ihat is bascd

on ?;liard nicinoiy inultiproct'ssor mhitrctiisr.. .\lcinor! cimimion i . ed t \ ti.uiii niuliiple

processcs accrssing the sams variable storeci in rhc '~h;irt.cl nicriior>. of thc 3ysrt.m. Shrired

niemory contention ia ;i source of ciipacity limitation on riic su iicii. Difi'sicnt tccliniqiie~

1iwx5;itc.d witli 1 tic sc"~~1i1tio~i 01- >ilch contentions ;1rc thc t;ni\ 01' .irtcrir ion h r thih thcsii;.

PertOiiii~n~c c i r i ; i l p i \ 01' thc t~chniqiicz iz i i ~ l i i ~ \ cd thsci~ipli :I I ~ . ; I ~ c - ~ I . I \ s ~ I \inid;itor ;i.\ ;i

p m ni'rhi. z i u ~ l ~ . Ttiis ch;tprci. prc5tnt.i ;iri o \ w \ ~ i c n of die z y . m i . r h ~ iiioti\.;iiion\ or [tic

stx;irc1i. fi~!lo\i*c.d hy I hr. br id Jr.iripiion 01' dis o\~s;iil ;isctiirc.~~iiii~c 01' r tic. + - w i i mo~irl

and contribution3 tmde 10 thc the&.

1.1 %lotivation

This thesi'; is concernsd with the sirntil~ition ot' the criIl conti-d nioclulc of A shrised-

mrmory rnuliiprocrssor-based telrphone switch system The main componrnts of a

telrphone suritch incliidr line iritrrfrices. the switching fabric. and the cal1 conrrol module.

In our study. our primary focus is on the cal1 control module. The muliiprocessor-based

call control module is responsiblc for ixrious types of functions like cal1 connection and

setup. call maintenance. billing etc. Thrre are two catrgorics of processrs that perforrn the

abow functions. The first ccitrgory comprises of pqloot l processrs. which are

rèsponsible for processing a cal 1. The othrr catrgory consists of ~ r o q - x ~ f o t r d processes

which are responsihlr for operational. üdministrcit ive and maintenance funcrions of the

switch. Cd1 procrssing is an example of paylolid process and maintenance. audit are

examplex of non-pliyload processes. Our sinidation m d y involve.; malysis ot'thr

multiprocessor systeni in ~vhiçh three types of processes nin in parriIlel. Ca11 motirls exist

thai are based on the skeleton o l k a l c4 l mixes that c m run on a telephone wirch. For

the purposes of simulations. 3 mis ofçails is hroken clown inro s~ibsr ts ca1lr.J .siihctril.y.

Each subciill handles the processing of an rvent such as srndinp dialrd digits. cal1 rrlease.

and ça11 hookup oççurring on the su-itch. Erircution of a subçcill is callrd it tiwi.swrim.

Based on the inçorning messages to the iystrm. thcre ir one subclill per inçoming

rnersiiy. It is detcnnined by the system archi lecture as to how man'. periphcrd3 srnd

mrssiigrs to the cd1 çonrrol module. Exrc~iting subcallz m:iy nred ro ciçcrbs the süme

addresses in the sharrd mcmory at some point of execiition. During the lifrtimc of a call.

a systrrn maintenance (non-piiyload) procrss mtiy collide a411 call procrssing process

(paylotid). At the point of collision. a dccision has to bc made as to which procrss should

rollback. Thrse drcisions are made with the help of a rollback algorithm. The rollback

algorithms can poten~ially have a strong impact on system performance. Wr have

analyzcd thrcr rollbûck dgorithms in our study: Boundçd-Priority algorithm. Ase-Bued

algorithm and Rollbück-Pay load algorithm. A drtailed description of t hesr algori t hms is

providrd in Chapter 4. A s imuhor tool written in C++ using the PROPHET simulation

rnginr (Nonel Networks proprietary) is used in this thesis (details in Chapter 3). The ciim

of this thesis is to undersrand the brhavior of the rnultiprocrssor system with threr

diffcrent types of processes running in parallrl. compare the performance of differrnt

rollback striitegies under diffcrent workload and system parmeters. computs the amount

of scnlability thrit can be lichievrd out of the systrm. Scalability is an important katurr in

this systsm sincr more processor cards can br addrd to the switch i with no cost

in~ol\remrnr o t' rcdesiging ) i f sccihbility is assured.

1.2 Related Work

Other studics have illustrütd \wrk thüt exisis in the üretis of mrmory contrntion. rollbiicli

al2orithrn.r. 3c hsduli ng algorithni', ;inri capxity and p i s for shrired rnsmory

rnultiproçr\sor sytcmb as w l l ris disrributrd nystms.

.A çonsider;ihle Iimount of work csists in tlic arci of techniques for the control ot' rcwurcr

contention. X reprcscntriiivt. subset of the cisisting work is presrnted. Most work rrl;itt.J

tu resource contention on multiprocrssor systrms hris focusseci on the çontrol of shared

msmory contention [Miij99.1]. Memory hotspots and their impact on spstem psrform;iiice

for binnry hyprrcube networks are discussed in [Dand92]. The problern of saturation of

the nrtwork connecring the processors to memory duc to hotspots and a request

combination technique ro reducr its effect on performance are discussed in [Pfis95].

Management of procrssing resourcrs on multiprocessor systems has spûnnrd two

different domains: the dlocrition of processors to the threads in a given pardlrl

computation and the scheduling of processors to multiple parallei applications mnning

simultanrously on the system. One of the early works in the xea of mappin, a thrads in a

sin& application to processors proposed a heuristic strategy ciillcd ..rnultifit" rhat

considercd t h r allocation of processors to applications w i th inter-comrnunicating t hrelids

is presented in [Coff78].

Majurndlir talks about controlling shiired rnrrnon contention rhrough scheduling in

[Miij97]. He proposes diffrrent scheduling policies such as Scrial < whrre all non-payloiid

processes runs serial1 y 1. Inde pendent i dl p;iyload processes run in pariille1 1. Gang

(processes are gangcd together) and Hybrid (rnginsrsed mix of tuSo or more o f the

prrvious policies 1.

Using l o c h ; and rollbacks for controlling üçcsss to shtirrd resourcrs on _oentiral purpose

computin_o sysiems is dcscribed in (Sil b9-I 1. Di fferent aspects of rollbtick siratqies h ü v r

been reserirched.

Koo and Toueg [KooS7] have proposed a rollhxk rrcovrry algorithm for distributecl

sjPstrms io tiike c3re of hilures during rsecution mi also to prsvent concurrent

invocation of rollback rccowry algorithms. This cilgorithm tiike~ n consistm set of

checkpoints and avoids livelock probiems durin2 recovery. The algorithm's iipprocich 13

said to be sync hronous as the processes involveci çoordinatc thsir local chcckpointing

actions such that the set of al1 reccnt checkpoints in the systrm is puariintecd to he

consistent.

In most rollback recovery schrmes. a failrd procrss is restored to a checkpoint. and the

process receives messages in the exact samr ordrr as it received them before failing. To

recover messages lost dur to hilure. loeginp is the most commonly used technique. Jnlote

[Jd89] shows the above approach to recover messages is stricter than nrcessary to save

the rollback overhead on the system. He proposes new techniques in [Ja189] to implerncnt

fault-tolerant processes that ctin handle multiple process Fÿilurss due to collisions. Thcse

references are token from [Sing94].

Anothsr algorithm is proposrd by Juang and Vcnkatesrin (Juan9 1 ] is for recovery in a

y t e m t hat ernplys asynchronous chsçkpointing. The algorithm assumes thrit ii proccssor

upon rrswting broadcasts a rnrssqe that i t Iiad hiled. Under the llrynchronous ripprorich.

chcçkpoinrs rit each procrssor are taken indrprndcntly u~thoui ;in' \ynchron izatiun

among ilie procrisors. .Asynchronous chrckpointing ha3 i r h d iuivrini t ip agliinst

s>nchronous çheckpoiniing but ctlso ü\*oids the svnchroniz~ition o\.erhrad cost in t h i h

met hod.

Previi~us work in Nonri Network.\ ha3 addressrd the mcdeling of shrired rnemory

contention on thc multiproceb~or-baxrd telephone witch. Di.urei~a [Dr\v96.1] describes a

niet hod of calcultiiing the capaçity impact of mernon contcniion hetuwn pct).loÿd and

non-payload processes in SA-Corr < Norrsl Networkn I . He dewloped Lin analyticiil mode1

of hounded-priority rollbxk policy (de finrd in Section -1.7 i. From this model. ti set of

rquüiions is solwd to find the probiibility of rollbxk and the processing power used by

rollbacks cüuscd. A mrthod of èstimating the contention probübility is proposrd.

1.3 Con tributions

Previous work rxisted thût simulated a multiprocessor systrm that could handle only

payload clliss of input data [Stre97]. This work was extendsd to simulate a multiprocrssor

system that could simulate more than one class thiit can nin parallel ro each other. The

sirnulator devcloprd in this thesis is for a rnultiprocessor systrrn thot handles three classes

of processes and c m be extended easily to handle more number of classes as wcll. The

three process classes studisd in this thesis are Cali Processing t ChLLP ). Maintenance

(MAINT). and Audit (AUDIT). The Cal1 Processing procrsbrs belong to payload catqor)

of procrss class wherelis MAINT and AUDIT bclong to the non-payload classes.

PROPHET (Nonel Networks) l i b r q is the backbone of the simulator modcl. The

sim~~lütor uses PROPHET ( Nonel Nrtworks proprietac sirn~iliition tool routines. I t

provides the rules. concepts and sttindtird funçtions. to simuliiir the reril-timc mode1 of a

telephone wiich. The multiprocrssor system has a schedulèr thai is rssponsihle for

schediiling the Jiffrrrnt types of procrssrs running in the sysrrim. A partial design of the

simulator scheduler for a single class cxisted. It is re-written in this thesis to hiindlr ihrw

c lasm. The 9imiillitor code is wrirtrn in progriirnming languaee C++. The esisting inglt:

class systrm implrmentrd ihe sirategp of chorging the roll b x k s io the qstrm/clus~.

Being a single closs spicm. both chargins policies producr the samc results. The

sirnuliitor drssribed in this thrsis implcments the rollbiicks char~ed to cltiss policier. Two

rollback srrategies rxistrd brforr for charge to system: .-\se-Basrd tilprithm and

Bounded-Priority nigorithm. A charge to class version is implemrntrd for each of the

stratrgies. Another roilhack strategy callrd Rollback-Payload is dcvrloped lis a pari of

this study. A description of this strategy is included in [kIaj99.2]. The simulation results

are illustrated with thcir explanations in chapter 5 . The analysis of the results obtainrd out

of the system have provsn that the stntegy of charging rollbacks to class (refrrred to as

charge to class) breaks the scheduler shares allocatsd to the processes running in the

system. Thus the processes are not able to achiew their actual CPL' shares. This study

wiis useful ro Nonel Networks althou,oh prrvious resuits bascd on analytical models

indicarrd this this thesis provided an experimrntçd cont'irmation. The rrsuits of the

simulation experiments dsscribed in the thcsis provide valuable insights into system

scaiiibil iry and the relative prrformmces of the rollbück stratcgiss.

Paylolid and non-priyload clnssrs are squally important for mriintenlince of reawntihle

capaçi~y on the rrlrphonr switch. Piiyload is responsiblc for processing the cal1 ihar p e s

through the switch and non-piiyload is rrsponsihlc for the mliinrcniincti and othrr

ubsidiiin üctivirirs of the nuitch. Thus contention be twen ~a!~loild and non-paylo;id

processr hiis besn ii major conccrn. This thrsis includrs the developrnm of ;i trace-

bascd tool ~ h a t corn putes the level of conren tion b e t w r n püyloüd and non-pq+mi. The

tool identilics rhr addrcsscs that contrnd the most. -4s ;i piin of this study. a fciitiirc in the

ton1 kvas irnplernented whiçh ciicluded ihc highly contcntivus ~iddrcsscs !-rom the w of

meniop addresses :icccsscd hy the application. By re-sn$wering the wdr to nvoid

contention for thssr mernon. locations. rhc blocking Irvel coiild be lowrrrd and ii higher

siipaciry cii~ild be achiciwd.

1.4 Outline of the thesis

Chaptrr 2 describes the architecture of the multiprocessor basrd telephone switch

involveci in the study. Brief description of the architecture and rnodrling of the systcni

under study is presentrd. Chüprer 3 describes the simulator mode1 designrd and

drveloprd in rhis thesis. It describes the simulation techniques applied to sirnulate the

multiprocessing fsatures of the tekphone switch with the help of PROPHET. Chiiptrr 4

prese nts the roliback strategies implemented in the s ystem. Chapter 5 describes the

simulation results obtained wirh three differrnt sets of input traces for different rollback

schrmrs. Chaptrr 6 describes another tool that calculates the lrvel of contention betwen

paylond and non-ptiyload processes. Conclusions and directions for future reserirch are

desçribed in Chaptrr 7.

t

Chapter 2

A iMultiprocessor based Telephone Switch

and data and personal communication services are processing intensive. and thcir

n u s deployrnent will require für greatrr computing p o w r than beforr.

Software Evolution: The economics of software drwlopment. production and

support. as wrll as market responsivrness. Lire constant drivers for improving

software-propmming rnrthodolo_oies of product 's arc hi tecture frrinirworks. For

example. Object Orientcd programming u n mnkc sotiwarc more manîgeliblr.

more modifiable and casier to debug for shorter tirne to cïolve new featurrs. Bui

in most cars.; [lis irnde off is thlit thc software progrun3 consume more p r o c e s ~ i n ~

poarr. and this trend is liksly to continue uith the ~rowing ch;illt.nge of

compiiting systcms ro drliver h i z h and hi2hr.r procrssin~ potver.

7.2 Objectives of moving to a .\Iultiprocessur-based Architecture

High Capacity: The n r u multiproçcssor-based product aims to re-ürchitrct the

cxistinz nodr for sustriinhg siyific~int growr h in n.orkloiid.

I High Reliribility: The new systcm rmploys new standards through the design of

strite-of-the-art architectural frameworks for d i ab le computing.

Scdability: This implies flexibiiity to incremcnt capacity without Ion, ~-trm

planning and complicated hardware surgery. The new product hased on

multiprocessor architecture provides the nbility to rstend processing copricity.

mrmoy. message throughput. and mass srorage by simple and straight additions

to the existing one.

Cornpatibility: The goal is to run al1 applications that are currently ninning on the

rxistine Uniprocessor-basrd application architecture and to hidc the changes in

the procrssine environment from othrr system compontnts.

The new systcrn XA-Core is proposed that prrsenr the opportunity of rnertin_o the ahove

requirement'; [Ben97]. XA-Core adopts the sharrd-msmory multiprocessor circhitecrurr

brcaiise i t offers compatibility ~ v i t h lcgacy softwcirc thÿt ran on a unipri~crssor biimi

system. A description of rhr XA-Core architecture is prewntrd neut.

NA-Core has a shnred-niemory architsct~ire that attüins nor only Iiigher ctipliciiy. but also

higher rrliühility than is possible with the duplcs proçessor design of the rxihtinp w i t ç h

prodiict. The merno- cnpücity. the cornpuring capacity and the input/outpui siipit~it>. are

scÿlnhle hy inscrt ing niorc rnrmory çcird.; and procesms. A high-lewl d iqram of the

Muitiproçrssor-Based teIrphone suitch is @en in Fisure 1. The architecture consisi3 o f a

redundtintly irnplrmcinted Shared ih4rrnoI-y. a serirs of Procesing Elrrncnts i PEs i to

provide cornpuring capricity. and n nurnber of VO processors ( IOPs 1 to provide input

/output interfaces and m m storqe. The intemal communication nrtwork that connrçts

al1 of thrse componrnts together is cÿlled the Interconnect.

1. Shared Memory 1

S I W I - ~ . \ l t~ \~ i i ) ,? : consists of independen! memory cards that prevent the conflicting use of memory iocattons by multiple processors. It operates in duplex for fault tolerance.

PE (Pr,~~.cls.vl~i,i .~ Elrr~ruriri: iç a simplex processor. All are functionally ident~cal. executing processes in parallel for fully symmetric operation.

An IOP r //O prr~ i~c* .ssor~: is a simplex processor responsible for mass-storage or for communication with the outside world Iike disk. tape etc.

The I , r r c , r . t . o , i i i c * t . f : 1s an interna1 point-to-multipoint network that allows any PE or IOP to access any memory card.

Figure 1 : Shared memory architecture of a multiprocessor-based system.

The Shared Mcmory supports a concept of dynxnic mernory ownership. which üilows a

PE to operate on data structures in merno- while pi-evsnting any other PE h m reading

or u-riring the samè memory locations and ca~isinp contlicts. This gives comparihility

with Iegcicy software. including paj load activiiir~ in ri cüll. However sornr tÏntxuning is

requirsd in order to rwch capacity tarsets. The rnr.mo1-y allociition donc ro the procssxx

can bs static or dynaniic.

Sharrd blemory contain3 Dütn Store i DS niernory. which is iully addrrsslihle by r v e n

proctxor. ShmJ Xlcmor). al30 conrains a miister copy of P r o p i n Store iPS msmi>ry. .A

local copy of the PS is milde awilablr ;i priori ;ind is used b! PEs to cnsure ni;isimum

eseçution specd. .A Procr..rhing Eleiiient i PE is a simplex procrssor. The ;meral-piirposr

çapribility is provided by the PE\. Each PE exeutes a procrss in piirnllt.1 with othcr PEs.

A11 PEs arc functiondly qiiivrileni and esrcLitr the scime software load during normal

operition. The input/ourpur interLice is providrd hy an IOP procrssor. a siinples

procehror supporthg disk and tape units for mas\-stora~e cüpability or for

communicrition n-ith the outside wrlcf.

The Interconnect is an interna1 point-to-multipoint nrtwork that allows any PE or IOP to

üccrss any memory card. The main advantrige of an intrrconncction nrtwork is ils ahilit~,

to cause communication in the archi trcturc and cstendibility. i.e.. the possibility of

adding or removin: functioncil units rui ly.

2.3.1 Types of processes

Therr are mainly two types of processes in the multiprocessor systern.

Payloüd: This refers to application software tlint implemsnts services that a Trlco

sells to its customers. The püyloüd software inclildes functionlilities such as cal1

procrssing. responsible for actulilly rxrcutin= a crill made on rhr suitch and billin?.

Non-püyloüd: This refers to switch support softuxe. consisting prirnarily of

operation. administration. and maintenance feriturrs. For rnamplc. the non Call-

procrssing such as .bt\INT. AUDIT cllisstts ;ire re5ponsible for maintrnmx and

operational f~~nctionalities of a cal1 made on swi tch.

Paylond md Non-payload tire coniplt.mcnt;iry tcrniz fo r c;itegorizing proçcszc?, or general

soRuxe Fiinctionaliry esclusive of the operating systeiii.

4 Method of Operation

5 l u l i i p r s s i n in SA-Core is achicwi with the same multiple proçrshcs as in thc prc-

rsisring architrcrure by enccuting thrm in pardiel ~onsurrcntlyi on srparcite PEs. The

ob.\tticlr. in t h i i str;iiegy is that parüllcl ihreadr ofesccution violate histi>rical iiiirnpiionz

ahoiit niutual exclusion. Rekrence of mntcrial in this bection is ohtüincd froni [Ben97 1.

Vtilities thüt inçliidr the control of preemprihility i for rsample %1CTEXOi; 1. mcrcly

prevent a process from beinp preempted inside ;i criticül secrion. Alli'TEXON is ü

srmaphore used to ensure that only one procsss cün enter rh r critical section at one timr.

It is used for müinly disabling the inierrupt so that the processor is not t a k a awriy in the

middle of exrcution. They were not designed to stop other processors from accessinp

memory at the s a m r time. In order to achieve correct operation on rhr switch. it is

required that the implied rnutual exclusion of ri non-prssmptrible process be presrrved

when tme concurrency is creatrd. There must be a way of ensuring that a non-

prrcmptable process retiiins cornpiete control of al1 data i t rerids or writes unril i t is once

again preernptable. Other processes must be locked out [rom using the same data. but i t

should not be prevçnted from accessin~ othsr xeas of mernory in pxallrl. XA-Core

provides precisely this cripabiiity. The Shared Mrmory hardware implrmcnts an

ownership mcchanism that riutomaticdly daims ownrrship of al1 merno- thüt a proceis

accesscs when runninp. That mrmory rrrniiins inaçcèssible ro other processes until thc

orvning process finishés its work and suspends. Me;in\~hilr. iinothrr procchs thüt uses a

dift'rrent set of data is ahle to accrss the Shnred 4 l rmop in pnrülld. With this memon,-

uçctx strcltep in mind. se\wril principlrs of operation ernergr. t'rom rhe SA-Core

iirchitt.ctiirt.:

O Multiple PEs cxrcutrs processes in paaillsl.

O ShareJ M e m q is fully addrsswble by e w r y PE and coniinues to br the d e

rrpositon for riIl data presrrvcd w h e n 3 procrss suspends. cornpleit.~ its

e u u t i o n .

O The Sh~ired Mrmory supports dynrimiç rnenioi? o ~ v n r r h h i p . \\.hich preveni.

desrruçtivc interference between parallrl processes rittemptin, LT to ;tcct'sl;i cornmm

datri.

PEs are symmetric: that is dl PEs are functionallp equivalsnt and rxecute the

same software load. A Software program achirws the slime result no matter

which PE it runs on.

The systrm schrduler runs on al1 PEs. As before. when the current procrss is

tinished the scheduler is invoked (within a critical section) to select the nest

process to execute.

Any single procrss will mn on only one PE at a tirne. but the process can run on a

dit'fsrrnt PE wen: iimr. As cnlls are processed cit the sarnc tirne. cal1 processing

can ext'wt~1 on 41 PEs sirnultaneouslp.

The work riccomplishrd by process does noi becorne visible io ihr rest of the

system iintil i t conimitc;.

A sh~1rc.d-memory mu1 tiprocessor s!-steiii miisr prevenr pnr;il Ir1 processes from contl iciing

use of çhared mrmory loclitims. The shlirrd memon supports a concept of rnrmory

ownership thiit provides rhis ~ibiliiy. The undcrlying principlc of operation is thai two

procrsses ciin nin in piiriil1t.l providrd that the) rlon't rrad or wits any cornmon memon.

Icications. To implrrnenr this. r x h process thiit is exccuting in 3 PE is a\signed ii unique

identifier. and this identifier is used as a ta; of daim ownrrship of dl lociirions in Shared

Memon thitt are rend or writirn by the proccss. Any orher procrss atiemptin, = to acces'i ii

rnrmory location thlit is already owned is blockrd frorn proceeding. A hlockcd procrss

nborts whai i t was doing and rcst'arts from the beginning at a future point in timc. The

outcorne is thrit thosr procrsses requinng the same Shored Mrmory locations are. in

effect. seridized by the system. whereas independent processes are allowed to execute in

pnnllel. A blocked process is able to start over becnuse the Shrired Mernory nrver allows

a procrss to perrnünrnrly u p d m any memory location until i t commits. in other words.

signals its succrssful complction. hl1 updates are recordrd in the Shared merno? as

pending changes only. and the pendinp changes do not becornr officiai unril commit timr.

the tirne whrn the job is succrssfully iiccornplishtcl. If the procrss blocks. the pending

chiinses are discarded and rhr process is reschrdulcd for another ritternpt to run tci

complction Iüter. after the blocking process had committed and cleared its ownership

tags. This ribility to discard pending changes and return to prrvious staie is callrd a

rollhock The ro1lh;ick ts ;t vsry powrfi i l capabilitu that provideb data intrgrity and i b

wcll huitrd to hiilt-tolc.r;int procrsbins. In the évent of ri PE failurr. i t is possible for thc

procesr thÿt u.35 cuccuting ;it the time of failure to roll back and restart in anorhcr PE.

Software functionh ;ire now ci'fecriwly immune w (1 procrhsor crash. Frerdorn of

srm+ation is cnsurcd on the systcm throiyh rollback policirs describeci i n Srciion 4.1

through 4.4.

2.51 Blocking

The Shared mrtmory prwidcs owwrship t a p intçmnlly ter a11 mrmory lineh ( a menior!,

linr is 32 bytes long). The t q s cillow the Shrirrd Memory to kzep track of which procesr

is accessins which mrrnory locations. Whenrvrr a process esrçutss. 311 memory lineh thüt

it reads or writes us lockrd. on the first accrss. A memory line is t q e d as being

temporxily owned by that procrss. The ownership tags remain in effect until the proçcaa

finishes its work and commits. At that time. ownership is reieaïed and other procrssrs

c m daim the memory lines. Only one process c m own a memory iine at a tims. The

S hared memory will hlock any other procrss from axcs s in~_ a mrmory lins that is alreitdy

tasgrd. When blockrd. a process rolls back: that is. it abons whiit i t was doing. al1 work

performrd up to the point of the block is discürded. and it is given iinother chance some

time larer to execute without blocking. stÿrting over from the point of the last commit.

The terms "proccss contention" and "process collisions" are used synonymously with the

trrm "process blocking". Evrry time two processes contend for the same memory linr.

one of thçm must roll h c k and start over. The. are rffectivclp forced to executè serially

so rhüi mutual ctecurion is preserved. Ssr illustrririon of blockine brtureen tuo processes

in Figure 2. In Fisiire 3. A and B are acronyms iised for procsss .A and B respcctively.

2.52 Rollbaclis and Commi ts

There are rwo copies of evcry nieitiory lin<: in Shüred .\.lemur>*. The Commit copy i3 the

ofîïciiil copy ihiit records the mos1 rrcriitly çommiitcd updm. The Active copy is whrre :i

proc tx tcmpimrily records an updüic iiniil i t cornmith. Whrn proces3 cornmirs. the two

copiri eschüngr roles such that the Active copy in sach t a g p i merno. line becorne3 the

new Commit çopy and the old Commit çopy becorne?; the nrw Active copy to be updiited

ihr nrxt time. Whcn a process rolls back. changes made to Active copies are just igorrd.

rffcctively erasing any trace of the proçess's esecution. In sither case. ownrnh ip ta, ai are

rrleased so other processes can use the memory lines. Commits and rollbacka are not

system widr but rather funciion on a prr process basis - many processes can be enecutinp

ai any one tirne and commit or rollbxk independrntly of one anorher.

Commit and rollback (Figure 3 ) rnight apprar to he intriclite but in practice the? are

simple. They are activated by ü mere state change of the tap identifier in the Shared

blernory. Thzrt is no irnmediate need to walk throush the memory to update a11 the

rnrmory lines that cire affectrd. A rnernory line will bc adlusrrd on its next ricccss.

The cornmir proceduri: is associatrd with the normal completion of a process's threcld of

exrcuticm. The process stops running on the PE and al1 c x h r d information ir flushrd

from the PE biick to the Shared Mrrnoq. The rolihack procedure. on the other hiind. is

a s m k n x i with the ahnormal termination of the rhreiid. The procrss stops niiininl 0 on the

PE. the cliched informrition ih invdidatrd. and the pi-occss is reschcduled for anothrr

chtince ai eseciitin- to complrtion. Thus. a procer5 thiit rolls brick must behwc 3s i f i t

newr escci~ted since its I N cornniit. Thére is aIw no glarmter thar the rolled-bxk

procrhs once rescht.du1c.d. will rsrcute esx t ly [lie sarnr seqiirncc ul' inxrniction.; ils

behirc: ii rnirhr brünch onro ti dit'fmnt p t h of cs~cutii~ri than hefore.

L

Shüred \lernory

I . 1

1

Proccss B time

Proccss B time

Figure 2: Illustration of blocking in the Shared mernory.

- Process A and process B can execute in parailei

writes are tagged as being owned by that since they don*[ read or write any of the same j memory tines. l I

1

f Process 0 h10cA.r tty~ng to access memory aiready lagged by process A and nills hiwli.

That 1s. process B abOflS what it was rfotng, releases tagged memory. and mes again later.

In shared rnernory systems. several processes cooperatr by exchanging information from

shared mrmory to accomplish a task. In such concurrent systems. ail cooperating

processes need to estüblish recovery points. where one process has to rollback when a

hilures occurs. A procrss in XA-Core executes a numbrr of subcalls. XA-Corr

implements ci concept of Commit and Continue to cnsurs the recovey of the system in

case of mrmory contentions. Here a process commiis at the end of e r q subctill

(trünsxtioni it sxscurcs bcforr starting the next one. The procrss releascs the rnemory

addresses i t owncd in the previous transaction and does n contrxt switcli to the nrxt

trtinsiction. This wny. whrn the proçess filils (rnrmory contention occurs and has i« roll

back. i t hns w e d all the processing pouer ciccomplished t i l l ihe liist cornmit mi only the

trcinsitstion i t tvüs currc.ntl>. proçcssing gors to wnstr.

2.3 Scheduling of' proïesses

The 'cheduler b~indles prosesses into classes for nianaging rht. h i r alloccition ot' CPL'

tirne ricrosh piocrsscx. Class shareh are rngineerrd to n rnüsirnurn percenruge of tord CPC

tirne when the hystcm is running at full capcicitu. A prinçiple of this system is thai dl PEh

are rqual. A schedulrr in the t o m of a schedulrr remplrire delines the ordrr in which the

procrssrs cire schrduled. Each catrgory of procrss is relcrred to as a clnss in the schsdulsr

templliie. So. for example i f there are six diffsrent types of processes running in the

systrrn. there are six classes in the system. When the process at hand finishcs and

suspends. the scheduler is invoked to select the next process to nin. using the class

specific ready queues and other scheduling informarion residing in the scheduler trmplritr

at the scheduler levrl. The system has a cripability of guarding agliinst two PEs invoking

the schedulrr at the samc instant and picking the samr procrss to mn. The schcduler has a

smrill critical section thüt can run in only one PE at a tirne. h schrduler template exisrs

w hrre rach class. out of the n classes. is dlotted some percemage of the total share. The

schcduler t h r n picks up the panicular clnss according to the scheduler shars and

schrdiilts a procsss of thiit class CO mn. The tact thlit a proccss can roll back and start oLPctr

means that it rnight suffer sonie dclay before completion. especially if i t roll3 back more

rhnn onw in a row. The scheduler cnsures that srcady progress is made and th< '!stem ix

rngineerrd to meet grade of service standards. The schsdulcr trmplatc is drs iped in such

a W Y ihnt i t mlikes sure r x h clriss of application is yaranieed somr sharr ot' rime. Thus

w x t ' s . the u-ork de tï ni te 1 y pro,

\

r

Shared 0 B Memory

Process 0 Time

- b

in Shared Memoty The Csmmi; zspv :ontains rhe ihe Ac:ivc ccpy if rt confarns a valid updat?. most recent commined value. o t h o r ~ s s ir readç the Commit copy.

Process B rolls back

exchange raies - the Aarve copy becomes the new Commit copy. the old Commit becornes the Aaw and 1s invalidated.

><

i1 -

invalidated and will be overwnttefl by future updates. This erases any trace of B's execution. X means the copy ot memory location 1s invalidated.

-- \/

- -

/

Figure 3: Illustration of Commit and Rollback in the Shared Memory architecture.

>< ;

Process 5 C~mmi t s

r

-

Chapter 3

Simulator Mode1

This chiipicr describcs the design and irnplrnirntation ot' the thrsci çlass simulator niodcl.

It alsu diisiisscs ;ihout the nicilioclulogy usrid to riiodcl the system: b~isic principlss bchind

the niodcling. inputs and outpiirs of the system. and workinp of the sysrem in deiail.

A Server provides the data management servrces for a number of client procsses that have need for the information. In this way. general processing 1s distributxi finile thé cornmon data is stored privately to avoid contention.

Figure 4: ClientiServer Model.

E x h procrss type has its own Client and Serwr. The Client3 grnerateh messages tvhereiir

the Ssncr processes them. For esamplr. CALLP client .;ends payload messiipca to [he

C.ULP server. Similiir is the action of mrssaling for thc other two procehses. Multiple

instances of' the srirnr process c m br rxecuted by more than one PE tit a timr. Thus for

rvc ry PE prrsrnt in the system thrre is an instance of c x h Ssrvrr type presrnt. There is a

concept of per-process data. a daia constmct that providcs multiple copies of the daia

structures. one for eüch process instance. Prr-procrss data works bsst for indeprndrnt

procrsscs. though communication and synchron ization anions thtm can bt. achieved wit h

inter-process messages.

3.2 hlodeling with PROPHET

PROPHET i Proprietan, Hiecirchiclil EuI~iution Tool) 1.; a library of C u classez and

funçtions for the drvelopnxnt of simulation progriims. I r is design capture and

performance ~inalysi~ toolkir for multi-tasking/multi-prmssor systcms. It hos hesn

designrd ro 0th- niodrling tlexibiliry. sprrd of eseçurion. and rase of iisc. The current

systcrn'r rnodrling is also dvne usine PROPHET [Prop%]. PROPHET exeutes the

rnodcl and collecrï staristics that c m bc. iisetl to linalyze or prrdict thc pcriorni;incc «f th^

rystcni. To Jwr.lop the siniuliitor with the tool. ~i C++ program is u.ritrt.n thrii d s ~ c r i h c ~

the reril-iirne bchavior of thc s p t r m to modrl. using basic constnicrz frwn the l i h r q . The

modcling is donc in rems of il Client-Server %!*sicni in PROPHET. Client ~ind Scrwr arc

rnodrird as ta&\. the niost important fundiinicntd components in PROPHET. They

usually contain mua[ of the inliirmütion in the niodel. TaAs communic~ire hp exchringin:'

messages. Each message is assioncd ü numerical identifier. which drtrrtnineh its type. In

our modrl. thrre cire threr types of messages sent t'rom three types of processes. .A

sendin; task (Client) sends n message to 3 queue in r r ce i~ ing tüsk (Srrvcr). Message

queues hoid the messages sent to a task until the task uçcrpts them. The Scrver rrceivrs i t

and procrsses it. The Sèrver processing sirnulatrs the rxccution of a pcuîicular class of

application procrss. PROPHET's modeling püradigm is dcpictrd in the form of modrling

layrrs. The threc Iiiyers are namely applications. schcdulin_o. and hrirdwrirr. Fielire 5 b

shows how the modeling is done for our systcm in terrns of these mentioned layen.

Each Client nins on rts own processor Each server simulates a PE and a n s on its ciwn processor.

A 3 d a s s scfieduier ternplate exisrs and processes accrrcring to scneaule recorgea in 19s temolats.

T a s k m CP~C~ient.MAiNT~Clten:.AUDITCP_Ciient,MAiNT_Ciien:.AUDIT_ClientC~~ent C P - ~ ~ ~ ~ ~ ~ . M A I N T - S ~ I V E ~ , A U D I T ~ S ~ ~ ~ ~

FifrSharedOs->tx!ween eacr C:ien:- Server

Figure 5: System Modeling Paradigm in terrns of PROPHET.

The application 1ayr.r reprrsents the software applications in iht: systern. It is the set of

concurrent tasks communicating with messages. A task owns a message queue to hold thc

messages i t rrcrivrs. Each task defines the control losjc. rral-time drlays. and messcipin_o

of a pan of the system being modelrd. A task instance is an independent executable

image of the tüsk. Eüch task instance is ghen an ordrr number to distinyish it frorn other

instances of the sarne iiisk. Instances of the given trisk can be assigncd to different

procrssors. 4 task has a priority and ü time slicr. which is used by the schrdulrr to

arbirrüte contention for the processor. For csaniplr. in our case we have rhrre types of

processes ninning in our systrni. Out of CALLP. NIAINT and .ACDIT. Nonrl Nrtworkz

designers have g ivrn ;i priority of 1 . 3 to each one of them respectively. To esrcurc'.

applicaticms require proçessors rhat compose the hiirdware layer. This laytr is a set of

proccsïors running in parciIlel. A processor clin be sesn a b a passive rrsourcr that 3 rask

nreds to ctsecutr. I t ha3 3 r e h i w s p e d and ri schcduling algorithm. Al1 processors are

"virtud ly cimnrctc. J". and rnes5iigt.s (it the ;ippIiciition Irwl c m tr(i\.cl betwern thein

witho~it restriçtioni. The sclied~iling I ü y is LI set of schd~i l r rs . The purpusc of the

Schsduling Iqe r is tu rcsol\.e çonwntion betur.cn ;ipplicatiuns or !he proceising

rewurces. A scht.du1t.r defincs ho\\ contention tbr proçessors is resolwd. It.; job is io

detcrrnine which t a d i will grt the p i i ~ c c . ~ ~ \\.hm \c.wral ol'thrm are read!. to run on i t .

There is rsacrly one zihcd~11c.r pcr procrshor. Tlie hchrdulcr is d z i , rrprexnted h>* ;i

software p r o p n i iind thus rcqiiirr. thc proccwir 10 r x ~ ~ i t t . . If the .*chrcliilc.r holds ttic

proceszor for 3 non-niill prriod i ~ i tirne. i t is poAhlc for tasks iu reqiictst rhe procewv

whilc the sçlieduler is htis!.. I f thlit happenz. the rqiirsrs are itored in the queue o i

pendinp requesis and are servrd in order of arrivai when the scheduler is ~ i ~ r i i l ~ i b l e tigiiin.

Figure 6 is a representiition of a schedulrr. In the figure. the scheduirtr is s h o w to have

ready queues to store tasks &tins to be euecutçd. If the schrduler holds the processor for

somr period of time. i t is possible for rasks to request the processor during bus) period of

the schedulcr. The schrdulttr th rn hold the rrquests in a queue of pending requests (as

shown in figure 6 ) and serves [hem in the order in which rhry arriced in when the

schedulrr is free again.

Pending Requests

SCHEDULER

Figure 6: Graphical Representation of a Scheduler.

3.3 Scheduler

The scbediller bundles procwr.3 inro clussc.\ for rnmising thc alloc.;ition of CPC.

time Licross procrsses. Class s h m s arc cnginrerd to 3 rnil~imum perccntlige of totiil CPC

timr whrn rhc .;!.stem i \ n innins at nusimiirn ç~iplicity. Scheduler classei, c m hr. uwd ;i\ ii

mode1 for cütsgorizing procrsses and w i p h i n g t hrir contribution to the overall hrhavi«r

of the systrm. This is a thrrr-clriss schedulrr. the classrs bring CALLP. MAINT and

ACDiT. Hrre is a description of hou the schedulrr works. Lo$tically. thrre rire srvcrül

different classes _grouprd togrther in wrious combinations cdled schools. In our case. a.c

have four schools where e x h school is ;i collection of classes. Schools rire built in order

to lower lritency times of process classes. In the systrrn's schedulrr. the ordrr of schools

are CM. ACiM. 1M.A. A where C stands for CALLP. h1 stands for >I.UNT and .A stands

for ,WDiT. The schools are visired in a t k d order and this is criiled ri sc.ht~irrler

rrinp/~ire. Figure 7 shows an example of distribution of SC hcdulrr shares among procrss

classes in our modcl. providrd by Nortel Networks.

Figure 7: CPU shares by Scheduler Classes

Decihion hiis to br nitidc of the rchsdiilrr teinplatr. Jepcnding on Iimv wc çhoohc. rhe input

rtrsîmz of the three clahs data rcspectively. To g iw ;i simple esiirnplc. we tire vihii ing the

schoulz in the follouing ordrr: CM. XIA. A. ACM - CM. hl.-\. .A. ACM. Thar is. the

schedder visits the schools in the order s h o w \'iz. school CM. thsn M A . then ,A and then

XCM cind thrn again the samr order repeated. This is one round of the sçheduler. In the

nsst round i t rv i l l stm the whole process again. The sçhrduler sors from school to xhool

looking for work to do. Within the schools. i t s a n s the class. If there is no work to do in

thrit çlüss. il gors to the nrxt class within the snrnc school. If there is no work to do in the

whole school. it simply goes io the nrst school in the trrnpiate. So i t kecps repeciting this

1 SHARE:

Pâyload (CALLP) 75'0

Non Payload \MAINT) 16'0

Non Payload (AUDITI 9'0

PURPOSE:

Processing a cal1

Critical nehvork and peripheral maintenance

Polling. operational 8 administrative functions.

proces5 ~intil it finds some work to do in somr school. Thus. unused processing timc in

one class is used up by other classes (like an' othrr round-rohin schedulcri. If for somr

reriwn. a school exceeds its tirne dice. the excess amount 1s deductsd from its time stice

the n m t tirne rhe same school is encountmd. There tire coiinters fur each instance of the

c I ; i ~ ninnins at (i rime to krrp track of the tinis sliçe uscd. If one class uses up al1 the its

xi i l~ ih l t . tirne. the counter moves to the ne'it chss in the school.

Sotc rhai in this esaniplr. the MAINT clüss \vas present in î I I the schciols. This to

c n h w t h ~ i t the MAIST prosram runs frsqiienily and providri a rhurt recuver>. iirnr.. In

thc ahwncc o f ii t'ault. the shiirt. for W d S T uill bt. taken by CALLP. in oiir study. the

schools are choscn in order for MAIST dis . \ to have lo\v 1;iienc~ in the \?stem.

3.4 Inputs of the System

Thi' hection d t x r i t w the inptitb to thc hiiil:itor +nsir.iii. onc done throuph PROPHET

coniiyr:irion e t i ip and the other ihroiqh rnciiio1-y tracch.

3.4.1 PKOPHET Configuration

PROPHET rrqiiires an input data tïle. anhich i the niost important for providing input to

the PROPHET-modelcd bimulation systern. It is rchponaihlc for srrting the configuration

of the rxprrimrnt and is rrferred to as PROPHET.çoniïg fils [Prop96]. The configuration

fils conrains plirameters related to the cxperimsnts. such as \varm up Icngth. batch

duration etc. This file is rnandütory. There is no restriction on the nürning conventions of

this file. Figure 8 shows an e s m p l s of PROPHET.config ai th a configurtirion format as

required for our simulation model. A description on erich input parameter is provided

next.

The Nurnber of experirnents indicares the nurnber of experirnents rhr user can make in a

simulation for a prirticular confi_ouriition. the number can bc n. where n is ri positive

intessr. The Clients send messages to the Scrvcr at regular interwls of rime. The inter-

arrivai rate of injrcting messages at Client end is controlled by an input variable I~ltrr-

..ln-ii~il Ti~~re. The tirnc duration between arriva1 of two message3 is _oenernted from an

i nqütivc J esponentiril distribution with ri mean of Inter-Arriva1 rime. The louer the Inrer-

Arri\xl timr. the more work the Client i h injrcting into the qsteni for the w v e r to di).

' CNumber of expertments cWarm-up lenoth> ~Nurnber of batches> <Batch duratrom cCALLP interarrival tirne> <MAINT interarrival lime> <AUDIT mterarrival time> cNumber of P E s ~Rollback Sicherne> <CALLP Service Tirne> <MAINT Service Time> <AUDIT Semice Tirne>

Figure 8: The PROPHET Configuration file.

To adjuhi the overall Utilizrition donc by CPC. the inter a r r iu i rate u n be drcrrnsrd or

increased. Since we have 3 system with thrre different types of processcs. wr have a

sspüratr threc inter-arrivai timr for each of CALLP. .MAMT. and AUDIT. The nurnhrr of

procrssing elements (PEs) is controlled by an input variable rYwuhei- oJPEs. The number

of PEs can vary in the system from 2 to 1 1. The higher the number of processors in the

system. the more processing power is expected to be produced by the system brcause

there is more work being done. The Srriice Tinw is the tims bumt in a rollbiick overhecid

when ri rollbrick occurs. This has a drterministic distribution and Nortsl Networks htis

providsd the exact value for the service times to be uscd in simulation. This merisure

would replicate uphlit is happening on the real switch in terms of whrrc the rime soc.;

when ri rollback occurs. The rollbüçk is charpsd to each clüss whoss proces'; is rollsd

bock. Out of this wholr rime. one-third of the timr is spent in releasing rnçrnon addresses

the rolling bück PE ownrd. The other two-[hird is spcnt in the context switch that occurs

whrn the roll back has 1;1kcn placc. Therr are thrrs inputs Ior the Sen ice tirne as \\:el1

rcich for CALLP. 41AIST and AUDIT. Another input pararnctrr is the ctipability to be

able to choosc the R o l h c k s c h i i r e . The plirametsr ciin be ser t o O. 1 or 2 for .\gr.-B;id.

Bounded-Priority or Rollbnck-P;i?.luücld respectively. The Itirriii-irp of the csperiment can

îlso he set in ternis of niimhcr of cornrniis rhat should wcur by warm-up. For highcr

numbes of PEh. hi2hr.r wrirni-up is requircd for the hystern to produce ;idcurare result~.

Thc .lir~,rh~>r o/h<rrclrcs drciJes iht. Iengih o f rspcrinirnt. The cspcrinirni!. n i d s in rhr

stud) inçiude ti tirne hütch niciins experimenr rngine provided hy PROPHET. This

çonsists of a w;~rrn-i~p (as mcntioned above) and o number of htitche5. .\II thc hacchcs

have a fixed durîrion set ai the beginning of the expsrimrnt. This is storrd in B~irclr

drirlrtion. Statistics collection brzins at the stxt of simulation. Ar the end of w m - u p . al1

stiitistics collectrd so Far are rrset. Sratistics are consolidnred ai the end o lev r r y britch. In

our crise. sach batch ends with rt commit.

3.3.2 Memory Traces

This is a trace-driven simulation and the e'rperiment nins ore hasrd on traces as input ro

the system. Currently. the only availabls data on memoq contention are the traces of

transactions collectrd in labs on the real uniprocrssor-basrd wirchrs. The traces contain

al1 sharrd memory addresses accessed by each ncçrssed procrss çlliss. They Aso show

when in the course of executions are those addresses accessed. Thres trace files k d into

the system. one for rlich type of processes in the systrm. The currrnt set of rracsfilrts that

were avtiilablr at Nonrl Sstworks includcd pa!!ond traces only. Diffcrent rrciçr file5 Kerr.

selected from this set of auilable files to constiturtl the CALLP tract.. .CIAI>T trace. and

ACDIT trace. A trace is a wquential l i s t of al1 rnemory locations that have hccn açcrsied

by a procrssor whilc csecuting ci sclrctrd progrmi segment. Traces of ciills Lire procrsscd

to tind common rncrnor) locations. It is possible. f c v instance. to scpar;itcly tlikr meniory

iraces of tivo criil types and compare thrm. II the' have m y data locations in cornmon.

those location3 are poienticil point3 of data contention and the clil15 could block eiich othcr

if rsrcuisd conciirrciitly. U s i n ~ the knowledgc of which s~ibcalls shiirs mrrnory. hou

long mernon: owncrship would be hrld. the iivernpr work timr of subciills. and thc wight

of the cal1 types in the cal1 modrl. the performcincr of the systern cnn be predictcd.

3.5 Outputs of the System

The aim of writinp ri simulation model for a given systern is to collrcr statistics on ihe

performance indicators of the systern. Tasks. message queues. ready queues and

processors a11 have statistics gathering built in. Users can define thrir ocvn stritisrics and

how and whrn to update them. Statistics providrd in PROPHET are of two types: Sarnplr

Statistics and Time Averaged Statistics. Sample stlitistics collect values independent of

time wherelis in Tirne .Avera_oed Statistics the values rire wrighed hy timr. PROPHET

provides two standard siniulrition engincs to mrasure performance indiclitors of the

systrm. The TimeBntch enginr. which we have usrd in ihis study. consists of ci wam-up

period iollowrd by ii certain number of batches. AI1 the batches have a tïxed duration.

which is sprcilïed. at the beginning of rhe experimcnt. Statistics collection bcgins lit the

start of the hini~illition. .Ar the end of the rwm-up pcriod. ail .;taristics çollrctrd t i l l thuse

points are rrset. Statistics are consolidored at the end of evrtry biitch. The enginc by

dckiuli prinis a britch report at the end of evrry batch and a consolidiiid report lit the end

of the esperiment. Thc user is 31so able to print otit user-dcfinrd stütistics in the

çonsolidrited and hütch report.

3.6 Architecture

As drscrihed in Section 3.1. the motlelin: of the y t c m is done in trrms d a muliiple

Client-Serw- systern. The client srnds o message t» the scrver. Sincc thrw types of

processes can run at the same rime. clittni A sends message type A to the s m w type .A. I f

the scrwr is busy at thtit inst:int. the message is stored in the messcise quriir until the

semer is free nest to process it. When the semer is free. i t allocates a PE to the message

to process it. The message then becomes a unit of rnecution and is referred to as a

process. This is a tmr-driven simulation and the cxperirnent runs are basrd on traces as

input ro the system. Cal1 models cxist thiit are based on the skeleton of real cal1 mixes that

crin run on a switch. Each trace t?le is a mis of call models brokèn down into subsets

callrd subclills. Procrssing a call is performrd in stages called subcalls which correspond

to the diffrrent stages in the iifetime of a call such as going off-hook. srndins dided

digits. call setup. cal1 supervision. and call release. The subcalls access memory

addressss. Each subclill h a a weight wociritrd with it. which is çallcd the work rime.

indicating hou. long the subcall will run for. Thrrr is no t Ï x d ordrr to picking up

subcalls. The process picks up a subcall randornly ( similar to ;i Monte Ciirio simu1;ition i.

Thus a s the procrss st;irts exesution. i t picks iip a trace file bascd on 2 random niimbrtr

2nd srarts reading the addresses in the trace. At the same tims. tinother proccss coiild hr

running holding any class of data and i t coi-rcspondingly could be recidiny another tmce

file. If the two processes running at the scirnr rime do nor rnçountcr ;in addrsss that çould

be comnion briwrrn thcrn. they nin siiçcer~fully to roniplrii»n. There can be ri possibility

thüi procrss 1 is occiipying «rie of the niemory addrc..\seh r har proçess 2 is d s o Iooking Ior

to occiipy during its course of cseciition. At thiit point. ri collision occurs berwrn the

t u a .A decision ha3 to be niüdr as to who ulns ovrr that niemory addrrss. The Jrcision

dcpend~ on the decision making rollbxk scherns. I f process 1 win?;. it continués

rsrcution with the mrrnory address and the procrss 2 rolls back. Proces.; 2 that rolls h x k

is sent to a box implementrd lis a queue cülled per i t r l~ . hos and stays there. The

implcmentation is such that thc process 1 thar caused the rolling back of process 2 is

responsible to free the procsss 2 waitins in the penalty box. Whrn process 1 has

committed. i t makrs sure that the process held in penalty box is frerd. Thus. procrss 1

continues execution rihcad. Once a procrss rolls biick. i r spends some time in cache

flushinp. in releasing the mrrnory addresses i t ouned when it was esecuting. The rollcd

back process olso spends sornr time in contest swirching before the ncw process strirts

sxrcution. Thrse constitute the rollback overhelid that r v e c process that rolls back

experiences. Any process stops exrcution eithrr whrn i t blocks with another proccss or

whrn it commits. A single process cm br involvrd in miiltiple collisions within the srime

simulation csperimeni. ~Mrasurements are obscn-cd and notcd out of the PROPHET

statistics reports. An id rd run is the one whrre CPU is utilizrd to ri cripacity of 9 5 5 . If

the system would tunr to 100%. in reality that would hit the ovsrlorid control. But thrre

are nu mewrès t3kc.n yet to mode1 ovcrload conti.01~. Thui, the simulritions h r i w hesn

rrcorded with CPCr c~ipiicity of opprosimiitsly 95r i .

The siniiilxion nin lenath is user controllccl mi the parameters crin bc ndjusted using the

PROPHET.config file. For al1 simulaiion runs. ive controllrd the run Icngth by drfining

thc nuniber of wnimits in ri batch. Erich run cnds whsn the 4;imulrition has reliched 3000

priylond coinniits. Running the simulation into 30 batchch ivirh 100 cornmitshaich does

rhis. The number of batches is chosen tn hs 30 and the numher of cummits pèr batch is set

to 100. Theht. ligures wrrc ~ised for simulation runs in SA-Core. These nurnbcrs

siig~ested by Nonel Nctworks mhiteçts rire iidcq~irite tor simulation runs to rnert thrir

iicciirac): requircmrnts [S tre97 1.

3.7 Programming Paradigm

The coding of this simulntor mode1 is done using the objrct-oriented progrûmming

language C++. A part of this code was obtainsd from ;i pre-existing single class simulator

writtrn in CH. This section gives a description of using the object-orientrd paradigm in

the mu1 tiprocessor-biised sys tem.

Data contention is the interference betwern two processes that attsmpt to use the same

düta in mernos. The programrning requiremttnt is not only to manries düta contention

usin2 miitual esclusion techniques but also to reducc the amount of data contention thot

crin occur. To meer such purposes. the p r o p m m e r ' s job is ro includr the design of datri

ÿrchitecrure thÿt liniits data contention. Programmin_o for a multiprocessor machins

rrquires rpeciul çarr in the management of progrrirn d m hr.sai~sc. of prrfornimx

pendtics associcitrd ~ v i t h data contention. whrrtt p;irrillr.l processes interfw wirh cüch

orher when attrmptiny to axes.; thé same data strucnms in mrrnory.

Objwt-Orieiited programrning (0-0 is a comprrhrnsiw drrign philowphy mi

methodology thrit offers a formiil data-orirnted ïpproach tu \oft\varc. design. NCK

techniques rire avciilahlr: for detining Jatü domtiins riiid creating progrrini intkiatriicture

around them. A proccss's data domain is the set of al1 Jiit~i thlit the p rocw rr.;itia or u i t e s

during its esccution. In 0-0 terminology. data is encapwlrited memin: thst the d m

structures tire contained aithin objrcts and hidden froni esterniil vis\\.. Encaprulateci data

is only accrsl;iblr via mrthods.

T h u . data miin;igrrnrnt is a compelling recison for considering [lie objrçt-orirnted

programming for rnul tiprocessing applications. 0-0 offers brtttrr program

çomprehrnsion. resilirnce to chanse. and code reusr. A class d i a p m of objrct clrisses

used and construcred in the simulntor code is prrsented in Xppendix i Fisure 16).

3.8 Simulation Software

The simulator software is a program of appro'timately 5000 lines. out of which about

1700 linrs of code are inhrrited from the legacy software. The linrs are s rpmted into

files whrrr each file contriins code rhat implements a distinct part of sirnulator. The

followinp is a description of different files and fiinctions of eoch tile.

Sir~r-i~riti~~.c: This is thc main program body t hat dcfincs th2 dicnts anci h s r w r ï i l a s x ~

thrit torrn the brisic backbone architecture of the simiilritor.

Schet/-rmrNt.c: This is the main body of the schrduler .iiin~il;itiir ihiit cont~iinz colis-

con troll in^ factors likr hrindl in: tirne slicrs. decision-miikin~_ as to whiçh clas.; of process

is schcdulrd to nin etc. It defines the code that controls the pointer to the cliiss n r v t to

run. The rrady-10-run queues are definrd and invokrd h m . These q~ieues queue up a11 the

jobs. which art. rwdy to nin ~ind rire waitin~ fur the CPC' to be fret..

Sc-lir</trniplore. c: This contriins t hr: sched~ilrr templiitr.. The program de fines t hr: order in

which classes of processes are defined and allocatrd for rseciition. I t includes definiticin

of the schools. order of classes in the school and their order in which they would run.

4.2 Age-Based Rollback Scheme

The A@3lised rollback policy is blised primarily on the a_oe of the procrssss ninning ri

piirticulcir cliiss of processes. Herr is an rsample of how the schrme works. For cscimple.

p l and p l arc: two processes mnning in the systcni. Whrn the? start. their priority is set to

zero. If ü collision occurs brtwern the two processes and pl rolls back. thcn priority for

p 1 is increascd by I and pZ continues esrçution. Nou sa); collision occurs cipain betwren

p I and p2 u.ith the same stotiis of priority. p i wins diie to highrr \ duc of priority. The

priority c m go up to an? limit. Thus. x h r n a cdlision occurs brtar.cn w o procesm. the

one with rhe hirher vaiuc of priority winh. The agr o h procrss is used in the follo\ving

w.ay a.hen two processes wiih the same priority cdlidci. When a proccss atarts estuiiing it

is given a fresh time stnrnp indicatins its ngc. Whrn s message arrive5 in the sprrni . its

q e i h zero. N'hm rum processes that have rolled hiick more than once collidti and haw

the samc priority levcil. thsn xçording ro the agc-based cilgorithm. their timr-starnp arc

çornprired and the older one wins. The cilder job 1.; thus assured highcr priority and the

voungest one rolls back. Table I depicts the A p B t i e d priority algorithm shouing

comhinütions of t a o proccsses W h priorities.

Table 1 : Decision Table for the Age-Based algorithm.

I Process 1 [Priority O] l Processl [Priority 11 [ I I i i 1 t

1 [ Process 2 1 Frsi corne firit renie i Process 1 wins i Priority O] i 1

Process 2 i Process 2 wins [Priority 11

Oldest wins

1

4.3 Bounded-priority Rollback Policy

This policy involves 3 priority algorithm with a limit on a priority that a process can

reach. The bound on the priority value is ii positive intrgrr and is fixsd to be 2 in current

simulations. To limit the niimber of tirnes a transaction c m bc rollsd back. this policy hüs

been iipplied to X.4-Core. A rolled back process cannot br rrstaned is long as the

blocking transaction is exrcutine. This prrvents rollhacks from coming büçk too hoon

ai th higher priority and rolling back the hloçking process. When a procm is schedulrd in

for the firsc timc. its priority is set to O. Lirhr.n two proçc.;ssz of pricirity O çollide ovsr a

incmor! access. the first continuez its esecuti«n. The other procrss is rollcd bück and its

priority ir increased hy 1 . Sirniliirly. uhcn [wu proçcsw of priority 1 çollide. the one th:it

owns the addresr continues. and the other ir rolled hück and is 2ivr.n 3 priority of 2 . Let us

con\ider ;i .;!.stem likr ours in which the Lippcr hound on priority is 2. When a colli&n

occiIr5 hetwen thrm and the) have differcnr priori[!. Icvelb. the hishcr pricirity proces

ains. The imponrint tfiiture of thi?; rol1blic.k rchrme is that the machine scrializr's when a

procesb with priority 2 is running in the rystem. U hcn procsws rrüch a priority ifduc. of

2. thry run reririlly. uhich implirs that a prioriry 2 process crinnot collide w~th ;in! othrr

process o f priority 1 running ai the samc tirne. I f ii priority 3 job is ninnins and thert. is

another priority 2 job ready ro run. the rcüdy to run job will w i t in a queue mil the first

job finishrs and thçn gets its chance to run. Thus 311 procrssrs of priority 2 iirr srriciiizrd

so that thry never collidr with rach orher. Thry crin only collide with a procrss of a lotver

priority. therefore they riiwliys continue their cnecutions. This way procrss cannot bc

rollrd back more than twics. The following table describes a combination of various

possibilities of collisions thlit can occur between processes runnins in the system with

differrnt priorities.

I l I I

i Processi ' First corne first serve / Processi wins j Processt wins ! i

; [Priority O] l 1 1

1 Process2 1 ~rocess î wms First corne first serve 1 Processl wins I

[Priority 11 1 1 l

j Process2 Process2 wins ! Process2 wins I Serialization Occurs [Priority 21 I L l

Table 2: Decision Table for the Bounded-Priority Algorithm.

4.4 Roll back-Payload Priority Scheme

Under this policy when ;1 collision oçcurs between a p;i>*loaJ and a non-paylolid procrss.

rhr payload proccss is rollçd ha&. The polit‘ is ripplicahlt. only if nonr o f the contending

process hüs a priority Icvrl of 2. If therc is a Irvel 2-priority procrss. the stsps outlincd in

section 4.3 arc iollourrd. The rationale hehind this r~llback strates!. is provideci. Thrrr is

3 q~lick rrsponse to füults thnt arc procrssed hy non-pqlod procrssss and thus kwèr

c;ills rire losr iindcir hults. in this rollbrick stratcpy. whrn two processes of the samr type

çollide and thrir priority is less than 2 . the one. whiçh has occupird the mcrnory location

first. wins the situation and the latter one rolls back. In the sccnario tvhsre there is a

collision beiwren two processes mnning Paylolid and non-Payload types of procrsscs and

thrir prioritirs is O or 1. the püyloiid (Call-procrssing) one alwiiys rolls back. In the case

where a priority 2 procrss collides with a loiver priority process. the priority 2 procrss

lilways wins irrespective of the fact if i t is caming a paylolid or non-piiyload job. This

1

l [ Processl [PriorityO] I

Processl [Priorityl] 1 Processl [PriorityZ] I

way a procezs cannot be rolled back more than twicr. This is a way to set the limit on

numbrr of tirnes a process clin roll back in XX-Corr brforr i t is providcd full liberty to

execute without ciny interruptions duc ro collisions. Table 3 drscribcs different possible

situations thlit can occur in the systrm.

4.5 Rollbnçlis Charges

Rollbticks that occur in a systrrn cause wastage of CPU powrr. When ti procrbs rollx b x k

thrre is a cost associ;ltr'd with the rollbrick. In this simulation rnodt.1. the rcillback hris an

rissoci;itt.ci rd- t i rne owrhsrid in addition to the lost w r k . The t ~ i l rolltiack awrhcrid

indudes the ow-head incurrd hy the procrss in rrlc;isins thc niemory dJrc.h\es i i o w n d

when i t uxs exccutin~ and the othcr part of the ovsrhead iz coiistituted by ihe tirne thüt i \

s p m in çontcut switching fr«m one process ro ;\nothsr.

4.6 Rollbacks Charged to System

In this .;cherne. the rollbxks cire charged to the wholc syitcni. I f ;i CALLP rollback

occuri. i t ctfkcts not only tlir CALLP c l w but d l the other clnh.rrs as wll and the cffcci

ih scen ivsteni-\vide. Sirnilx schrme is used with othcr classes riinning in the c y e n i .

Chiir~ing rollbacki; to the .;--stem time nieans that al1 d i e d u l e r classes are yxt in tcrd the

predetcrminrd shares of the usrful proccssinp tirne. The uscful proçessing time is the total

spstem tirne minus the tirne spsnt in rollbocks.

Table 3: Decision table for the Rollback-Payload algorithm.

i i

~rocess l 1 ~rocessî ~ r o c e s s ~ ~ r o c e s s ~ / ~rocessî (priorityo) 1 (priorityl ) (priority 2) (priorityo) (priorityl) , C~II-P 1 GAI-P 1 C~II-P i Non Cal14 j Non Call-P ; i 1 1

1 Processî j (priority2) 1 Non Call-P

Processl 1 Processl / Processi 1 Processl 1 P rxess l I

1 wins 1 wins ! wins I l WinS I C~I I -P ! 1 I

1 1 1 1 I , I

1 I I

1 1 l 1 j \ 1 1 ,

i Process2 1 Procrssî ,: First corne Processl , 1 Processl j Process 1 i Processl ,

1 (priorityl) wins 1 first serve w!ns , wins wins wins

1 1

' Process2 1 Process2 , Processl , Serialization Processl ' Processi! ! Serialization (priority?) wins wins wins wins

j C~I I -P I

i l ! --

I I

j I

I Process2 Process2 Process2 Processl First corne Processt , Process 1

(priorityo) wins w~ns w~ns first serve wins wins Non Call-P ,

1 1 - 1 l

l 4

l Process2 ! Process2 Process2 1 Processî ' Process2 First come 1 Process1 (priorityl) , wins , wlns wins , wins first serve w~ns Non Cali-P I I

l I

I

' Process2 l Processa l Process2 1 Serialization : Procéssî Process2 Seriaiizatton : (priority2) 1 wins wins I w~ns wins ,

, Non Call-P I l

4.7 Rollbacks Charged to Class

In this kind of scheme. the rollbacks are charged to individual c lwcs . if a CALLP rolls

biick. rollbück is chargcd to only the CALLP class not rffectins othrr clüssrs thlit are

running in the system. The scheduler share allocated for the procrss class includes with

the useful procçssing time as well as the rollbiick overhead incurred by the class.

4.8 Charge to Class chosen for this study

Charge to class was chosen for this study for the following reason. When a rollbnck is

occurring in ;t class. it dors not impact other clrisses at 311. This stratrgy does not affect a11

the cliisses lit a h i r levrl but charges the class that rollrd back with the rollbiick cost

rathrr than making the other classes victims whu Lire not involveci in the rollback. The

charge to das s s[rategy is moti~rited by the cffcctivc incorporation of thc samc iratcgy iri

charging schrdulcir overhrads. In the Sharrd .Uemory architecture. CPU timr is char2r.d

pcr cl~ish by thc çchedule~~. Schrdiiling ow-hrad is dso charged pcr c h . Ththri charging

nicchiinisms firsaal l t h s processe!, of onc clrtss from mot her.

Thus the CPC rime per payload proccss c m be guaranterd in ordrr to engineer pqluiicl

capacity and prüde of service. Siniilarly. the reiil time for ü non-piiyloctd proce\\ on J i th

grade of service cün br guliriinrerd. By analogy. if the rollbick cost and rolltrcick

ovsrhexis w r t ' c'hargrd to clas?>. thih would flrc wall one cliihs from anorhsr. H o w w r .

this will only he trije if the rollbxk ;iiprirhm Jocs not affect the schcduler shtirc prr

class. H o w w r . wirh the rollback schemes trstcd in this study. ws h ü ~ e cuncludrd thnr if

the rollhackh art: charsecl to the clriss. ihc scheduler shürr.\ cannoi hr: mainriiinrd cind thuh

:ire brokcn. .A detailrd discussion is provideci in çhnpter 5.

1

Chapter 5

Results of Simulation Experiments

Resiilts o t' aimul;iiion s s p r r i iiiciii~ conducted for investi p i i n g the perforniancc of the

rollblick strritcgies rire presrntrld in this chiiptcr. Beforc dchcrihing ihesc. thc csprrirnrnts

dons for vriiidating the simulatioii mode1 ;Ire dessribed.

R i i t d .\\-ailable Poiwr: R n t d ixiilahle powr is LI measurc. of the rcal work donc by

payload processes. The uni t of RAP is in ierrns of PEs. I t is the avttra-c number of

PEs t l i l i t ;ire biisy pcr unit timc in p r o c r s s i n ~ the iisefiil work done hy piiyload

processcs.

Thus RAP = (CP-cornmirs * CP-AWT) / ElapsedTime

Where: CP-Cornmits is the numher of payload (CALLP) cornmits that oçcurred during

the course of simulation experimrnr.

Elapsed-Time is the total elapsed tinie of the simulation esperiment.

CP-AWT is the average work timr of ii püyload (CALLP siib ciil1 mecisurrd in the

e'upsrimsnt.

NumbcrofPEs is the count of processing d c n c n t ~ runnins in thc systcm in ~ h c piiriiciilu

simulation cxprrimcnt.

Csrful Processiny Powrr: The usriful procr.\rinp p o w r is the sum of thr usct'ul uork

donc per unit of timr bp al1 procesa cl;lw.i. Both the h~irdwarr and the soft~varc

architeçtiires impose o\.r.rhr.licl.r tliat can hri atiributrd directly to niiiliiproceiing.

Csef~il Procrssing P o w r is thr componcnt of thc iotd xork d m e prr unit tirne on [tic

witch. that is lcft owr tu Jo the u d i d work a h - niuliiproct.s.ring ovrrheads has

been subtractrd OLIL

The formula for L'PP calculrition ih

UPP = Systrm work timr/Elapsrd Time.

Whrre: System work time is the masurcd tirne the systsm has hpsnt in doing rra1

pciyload and non-payload work sxcluding rvork expendrd dur to rollbücks and its

overhrads. Eiapsed time is the total rlapsed timr for the simulation rxperiment.

RAP and UPP are measures of uiilizrition-s that indicare sysrern capacities. A s cüptured in

Little's Law [ J i n 9 I l . system throughput is directly proportional to systrrn utilizntion.

The contention between different process classes is charaçterized by rhrre contention

parameters thot are definsd next. Qpp is rhe conditional probability of blocking when ri

payload procrss nins concurrently with anothrr payload procrsa. Qnp is the conditiond

probabiliry of blocking whrn a non-piiyload process runs concurrently with anothrr

payload process. Qnn is the conditiond probability of blocking whrn a non-priyload

procrss nins concurrcntly with another non-payloiid process. An operational de finition of

q n p and an ai~orithrn for its esrimation is presentcd in Section 6.1.1.

Blocking content in a scrnario is defined by blocking le\-el. The hiocking probability is an

indicator of the numbrr ot'conimon addresses rhüt exist bctwcien process classr.\ thtit c m

Irad io blocking when thesr processe5 run simultancoualy on the systrtm. Higher the

blocking probability. more cornmon addresses cxist thlit would cciux more colli.*ions in

the systern.

Wr chose thrre input traces of darlt col lrcrrd on ;in un iprucessor-bud switch of vai-ying

blockinr lewls. The different input traces are rekrrccl to 3s "acenarios" in thc chapter.

For this study. ar chose thrrr scrnarios 4s inputs tu our systrrn: scttncirio 1 . scenlirio 2.

and ssrnürio 3. Scrnririo I is the one with loucit hiocking of 3.85. where qnp = qpp =

qnn = 0.038. Scenario 2 is the one with medium blocking probability of 6 .69 wherr qnp

= qpp = qnn = 0.067 and Scenxio 3 is the one with highest blocking problibility of 16.6<i

where qnp = qpp = qnn = 0.167. Thesr input strrams of v q i n g blocking probribilitirs are

chosen in ordrr to study the performance of the system undrr diffrrrnt scrnarios thnt

produce a range of low to high blocking probabilities.

Thrsr scenxios are used to analyzc the differtrnt rollback dgorithrns: the age-based

aI_oorit hm. t hr bounded-priority algorithm and the roll bxk-payload algorithm that are

described in chapter 4.

5.2 Mode1 C'alidation

The goodness o i i simtiiation modd 1s meiisurcd by the c l o s r n w of the model outp~it to

thÿt of thc real systenis. Validation refrrs to snsiirin? thlit the ;issiimpiions usrd in

drwloping the modd are reasonablr in thlit. i f çorrectly implemrntcd. the modrl would

prociiiçe resulis ilo~ to thiit obssrwd in red systrms. In sonic cisr.;. viilid;ition of the

modrl is inkisihlr beciiusr either the rcal hystern docs not rtxist or hrcauxe the

mcasurrments miiy be ioo expensive to cnrry OLII . In such criscs i t mny he possible to

~in~ilyriccilly niodel the systrm to a certain rxtent. The himiliirity of iinalytical prrdiciions

ancf .iimul:ition rc~ulis is L I W ~ {O wlidatt' the sim~lafic)n mode1 [Jaing l 1. Three difkrcnt

techniques were uscd for vcilidating the rnoJc.1 and arc. discussrd hrirHy in the follo~ving

st~bscct ions.

5.2.1 Validation Technique 1

One rvay of validoting the mode1 is to compare the results of the mode1 a i th

merisurements made on the red stvitch under the same worklorid conditions. In Our case,

this method of validation is not actually feilsible because the red telephone suitch \vas

still under drvclopment when the resrarch u s cnrried out. An andytical model does

exist that predicis thlit combination of certain rollback al_oorithms w i t h the policy of

charging rollbricks to cIass couId break the schedulsr shares. Our simulation results dso

demonstrate the same as discussed in Section 5.3.

52.2 Validation Technique 2

Prsvious work existed where a single class simulator usl is used io sim~ilats rhr sxecution

of CiilI-processing processes on ri multiprocrssor-biied switch. Thus to validate our

multi-class sirnullitor mode1 against the cxisting single-clliss simulaior ybrcm. uc. had to

disablc ihc rwo non-payload i 41XLXT and X D I T 1 cliisss in our +.;teni and had juar ihr

Cd/-proccsing proçsshes enablrd. Thus. Ive wrs able to coiripasr. rhs t \ w sin~lc-class

systsms in terma of various parameter3 such as CPC' uriiizcition. queue sizrs. CPU idle

timr ~ n d 3~er;lge waiting rimc for a procesa ro csrcutc. H'r. wcre s u c c r d ~ ~ l l ) able ru

vülid;iit. our modd with results such (i RAP. CPP thnt closely matchrd to those of ~ht '

rsisting modcl. .A srimplc sri of rcsulth cornparing thc tuo siniuliii«rs for boundd-

priority rol1bxic.k algorithm is prcsenied i n Table 4.

Table 4: Results of Validation Technique 2. -- -- -

7 / # of PEh / sinele CG& ~ 6 u l i i t o r , hlulti C1w Sirnikitur I

I 1

1

l 1 1 ; ! t (PEh prr unit timr 1 , ( PEs pes u n i t tiine 1

I 1

I RAP 1 3 1 2.77 ' 2.8 1

! CPP / 8 / 6.85 1 6.S-1

5.2.3 Validation Technique 3

Ssnsitivity .I\niilysis is used to check the model's f a x wlidity. It is ii check to make sure

that the mode1 behwrs in the sxpected way when one or niore input variables is changeci.

Briscd on observations and experiencc on the real singie procrssor-basrd system. thrre is

a notion iit Icctst of the direction and change in mode1 output when an input variable is

increrisrd or drcreasrd. Fur the validiry nf mir .;irn~il;itim rnodçl. wr: prrf~rmèd two

sensitivity ctnalysis espcriments bp changing the worklocid for the non-püyload processrh.

The rt.ult\ for Sensitivity ;\nülysis 1 and 2 q r e e with the espectetl notions bascd on the

niiiltiprocr.szor s y ~ r n . Thry Lire discuhsed in detail3 in Section 52.3.1 and Section

- 7 - 7 3 . - . -y . - .

5.2.-3.1 Srinsiti\.ity .Analysis 1

Sc.nsiti\.ity ancilysis i i u perfomed wiih ditferent blocking prohnbilities for 7 PEh in the

.;>.;tem. For the pi~rposr.~ of mrliauring the srnsitivity to a change in the non-paylmd work

tirne5 in the rnultiproct.~~or spstrm. the s ~ ~ b ~ i i l l ivork rimes of the non-priyload traces w r r

doiiblrd. Sincc the nori-püyloüd arork tirnrs tire doiibled. non-paylocid processe3 will

rrquirc more timr to run. Hencr non-pnylorid will rxperience more collisions. .\.lnreovrr.

ptiyload gets less chance to run and as ri result RAP decrrasrd by nrarly 40% to 455 .

Rrsults are depicted üs in Figure 9 a ) and b). Wr measured the srnsitivity of the sysrcrn

with respect to rwo thingr a) Sensitivity of system performance to change in work timrs

for different rollback schrmes and blocking levels and b) Sensitivity of performance of

rollback schemes ro different blocking levels. The original simulation mrasuremrnts arc

referred to ris generic measuremcnts in the following subsections. The followin~

abbreviations are used in the Figures presented in rhis chaptsr.

1 ) RAP-Rritsd Xvailablè Potver

2 ) UPP-Usrful Processins Power

3) BP-Boundcd-priority alsorithrn

4) RP-Rdlback-paj l o d algurithiii

5 ) AB-;\gr B a s d algorithm

1. Scn\itiviiy of systsm perfomrincr t u change in wurk timrz for different rollback

schsrne~ ;ind blocking 1 ~ 1 ~ 1 s .

CPP \dalucs for scenario I tire higher by O. I to 0.3 PEI and CPP vill~ic\ tbr hccniirio 2 ;ire

1owr.r hy 0.1 tu O. 15 PEs than the generic valur..r ;ih prcisentrtd in Scction 5.3. R&r to

Figure 9 h 1 and Fig~irr 1 O b 1. Thih variation is btccüusr of the differcnçr in iimuuni of

usefiil work beinr done by the systrm. Scrnitrio I is a 1«w blocking sctmiirio. thus in [h i>

cilse whsrc non-pqload w r k times Lire doilbled. rhc effect of Ilirgrr non-priyload u.ork

tirne.; is sii;hily Iess than scsnario 2 u-hich exhibits 11 hizhrr blocking il5 compared to

scrnürio 1. A s sern in Figure 9 a). the RAP vülues for the three rollback algorithrns have

reducsd by 40-45% in cornparison to grneric cases shoivn in Figure 1 1 a ) and Figure 123 1.

Due to liirgcr work tirncs of non-pciyload. the paploiid gets less chance to n i n and iis a

result RAP has decreases.

[I. Sensitivity of performance of roll back schemrs to different blockinp levr 1s.

This section describes the effect of change in blocking lrvels on the performance of the

different rollback algorithms. Figure 10 a ) and Figure 10 b) present the UPP and R;\P

values richieved with the higher non-payload work timrs. Figure I O a) shoivs chat the low

blocking scrnario i.e. Scenario 1 enablrs t h e systrm to richicvr highrr procrsiing power

as coniptirrd CO Sccnario 1. which sxhibi ts a highrr blocking case. As the blocking in rhe

systrm incrrases. the processing porvrr of the bystrm drcrrasrs for a11 rollback

algorithnis. As the blocking incretiw in the systsm. therr are more çollisiunz hcrwrn

non-pctylo~id to non-p;iyio~cl processes and non-pliyload to paylo:itl proccszr.5. The effcct

ih stronger in cornpariam to the p e r i c c a x hecüiiw non-pciyload procewh run t'or

lonsrr periods hcre. This accounih for drcretizti in RXP of the +stem. (Rclrr ro Figure I O

a ) and b).

Age-Based Bounded- Roll- P Payload

Rollbüçk Scheme

Apr-Büsed Bounded-1' Roll-I'ü~loüd

Rollhück Scheme

(b)

Figure 9: Simulation results for sensitivity analysis 1 showing (a) UPP (b) RAP.

scrnürici 1 scenario i

Simulator Inputs

scenario 1 scenürio 2

Sirnulator Inputs

(b)

Figure 10: Simulation results for sensitivity analysis 2 showing (a) UPP (b) RAP.

Collision Stat is an output parameter of the systrrn that counts the number of collisions

thüt took place in the system durin? the course of the single sirnullition exprriment. This

statistic ais0 includrs the collisions that were ignored by choosing to @ore a particulür

type of blocking in the system. Here P smnds for a payload process and N stands for ri

non-payload process. PP Collision Stiit is the numbrr of collisions that took place

bstwcrn two payload processes during the uholr rxperiment. PN Collision Stat is the

numbrrr of collisions thar iook place brtwrcn payloxi and non-pliyload procrssrs during

rhe \vholt. cuperirnent. N?; Collision S t a ~ is the number of collisions that fook plücr

between non-piiyloüd and non-paylotid proceswa during the whole sxperimrnt. In the

Triblc 5 ancl Table 6. coliimn 1 defines the number of PEs ( # of PEsi. column 2 indicritcs

the rullhack nlg«rithm chosrn for ihe 3imulation experiment. colurnn 3 indicatrs which

one of the contention types ipayloüd - pay1oa.I drnoted 3s P-P. non-ptiyload - non-

pii~load denotcd as S-S and non-priyloiid - pliyloüd drnoted iis N-P> i b ignorrd. Thih

çoliimn clin ils su have (i value 'Generic' whiçh statr:, the normal opercition of the

sirnuliition ssprrimcnt with non-zero blocking probobility i none of the blocking leveh

iirr i p m d in the systcmi. Column 4 and 5 indiçate rh r rritrd-avliilnblr powrr i RAP) and

usef~il procrsing pou-rr i UPP) üchirvrd by the systrnt rrspcctively. Colurnns 6. 7 and S

indicate the PP Collision Stüt. PX Collision Stiit. and NN Collision Stiit for the pcirticiilür

simulation experirncnt.

Table 5 shows the rrsults of the srnsitivity analysis trstrd in a multiprocessor systrm with

3 PEs for scenario 2. The RAP and UPP values for al1 the simulation exprriments shown

in Table 5 do not depict a hish variabiiity. This is because the inter-mival tiinrs of

mrssqcs have not c h q e d in between the rnperiment for the samr rollbnck algorithm.

Table 5: Simulation Results for Sensitivity Analysis 2 for 3 PEs 1 #of Rollback 1 Blocking RAP / UPP / PPCollision 1 PN 1 NN Collision 1 ! PEs / Policy / ignored? 1 1 star ! Collision Stat i

! 3 ( Agebased / Generic / 1.165 1 2.54 1 600 l 22 [ 645 I

!

l 1 Age-based 1 P-P / 1.12 12.48 1 785 / 751 l 65 i l

1 / Age-based 1 N-P ; 1.15 1 2.54 / 642 f 119 1 924 1

1 Age-based j N-N 11.12 12.48 1420 i 98 1 1411 I / 3 Bounded- i Generic 1.16 2.56 724 1 12 j 712 4

l

I Priority l I 1 1 I l I

1

' aounaea- i P-P i l 9 1 2.55 1 1202 ' 740 ! / IO4 Priority ! I f ! I 1

Bounded- N-P 1 1 .O8 I 2.42 ' 428 190 1 786 l I I I

I l

I I Bounded- ' N-N , 1 19 I 2.53 , 505 I 45 1 O49 I

I ~rionry 1

3 i Rollback- , Generic 1.14 , 2.53 669 1 6 1 398 1 Payload J

1 Rollback- , P-P , 1 21 2.57 , 1056 128 i 571 1

1 Payload I

, Rollback- N-P 1.24 2.56 t 584 14 A 749 1 ' Payload 1 1 I I

Rollback- I N-N , 1.19 I 2.55 721 163 , 1734 1 Payload I j j l !

Table 6: Simulation results for Sensitivity Analysis 2 for 5 PEs. 1 #of Rollback Blocking RAP UPP , PP Collision . PN t NN Collision

PEs , Policy ignored? I Stat Collision Sta t I 1 Stat

' 5 i Age-based Generic 1 2.45 ' 3.7 1 1038 : 47 i 754

I Age-based : P-P / 2.5 ; 3.8 2025 i 123 1 391 I 1

I Age-based N-P 2.6 3.6 : 1230 ) 313 ô54 1

' Age-based N-N ' 2.55 ; 3.6 ! 1074 1 76 332

5 1 Bounded- Generic , 2.27 a 3.5 f 134 I 1 Priority 1 i l 1

' Bounded- i f - P 1 2.37 j 3.5 / 1974 1 648

l 1 403

j I ~ r i o r i t y I I I Bounded- ! N-P 1 2.2 3.6 989 1 385 1 787

IPriority I l 1 1 l 1 FnIaPed- U N / 2.2 1 3.5 1 1080 1 273 1 1269 Y ! I l I I

; 5 1 Rollback- 1 Generic / 2.2 1 3.6 1 1236 1 179 1 699 1 I i

2.19 1 3.5 11 1 366 1 Payload i

I Rollback- ) P-P ! / 2 1 5 4

1

Payload I

Rollback- / N-P i 2.2 3.6 / 989 1 385 l 787

j I Payload l 1 Rollback- ; 2.2 3.5 735 1 568 1 2100

i I

I

i 1 Payload 1 I

As w r ignore the blocking between payload and pliyload processes. n growth is observed

in the nurnber of overall collisions in the system between piiyload and non-paylolid

processes for age-based. bounded-priority and rollback-payload algorithms. Wi th no

blocking betwren payloüd -payload processes in the systrm. more paylorid processes =et a

chance to exrcute quickly Irnding to a rise in nurnbrr of payload - payload collisions (PP

Collision Statl. Note that the payioad-payload blockin_o was ignored in ordrr to sirnulütr

the activities on the switch with qpp=O. Fast processin_o of paylocid processes giws ri

chüncr to non-paylod prosesses to run throiigh the systern. thus a highcr count of non-

priyload - paylotid and non-püyload - non-payioüd collisions (PX Collision Stnt and NN

Collision Strit ).

By ignoring the blockinp bztwcrn püyload and non-paylotid processes. in a~e-based

rcillback algorithm. this bumps up the coiini of pciyloüd - non-payload collisions üs rhe

processes rire runnins through the shared addresses at ri hst ptice. Irtidiny to hi$ numbcr

of collisions. Thus the collihions would groa evrn brtwren non-payloüd - non-payloücl

procrssrs. But in case of bounded-priority and roll bück- payload algori1hm.r. t hrre i b

diffrrent rffect obsrrved wi t h the stiitiis of col1 isions hrtwren püyload - pqloüd

procrssrs. Both the algorithms involve priority 7 serialization in the machine. Dur to

highest share of pnyload processes in the schrdulrr. there are high numbrrs of priority 2

payload processes in the system. With the serialization of machine. there are no collision^

between priority 2 processes and thus lower overall system collision counts.

When the blocking between non-payload and non-payload is ignored. the count of non-

payload - non-payload collisions grow up because of fast processing through the s h u e d

addresses brtwern two non-payload processes. rven though no rollbacks actually rake

place in the system. Measured rrsults also show n jump in the u l u r of NN Collision Star

for al1 the rollback algorithms. Thus payload processrs. due to thrir l a r s share in the

schrdulcr trmplate get to run in parallel more with non-payload processes and thus

col1 ide with each othrr mors often. This increases the count of payload - non-pqload

collisions as well in the system ( P X Collision Stnti from 6 to 167. Since the rollbxks are

charprd to indiviciual classes in the systrrn. ignoring non-ptiyloiid and non-püyload

blocking should not have riny effect on the collisions betwren payload - payload

processes. Thoush the çount of PP Collision Stiit herms to have bcsn drcrccised. i t is not

an cfkci of qnn. i t is simply an cffrct of less ptiyloiid work done in the systcrn.

Table 6 illustrates the results of the srnsitivitv cinrilyIi\ for 3 PEI in scenario 2 . The

nititisured values for RAP and CPP for al1 the ziniuliition rxpcrirnents. as in Tnhlr 6 do

not show a high degrce of variance. This is becaux the demancl on the systcrn ha\ noi

çhmped in betwcn the expcrimenth.

As blocking incrrasrs in the systrm due tu highrr nurnber of P b in the systrm. rht. effrct

of isnoring any type of blocking dors not cause any vcirilincr on the collisions betwen

two non-paylood processes in the crise of age-based rollback dgorithm. As more

procrsscs block. the count of collisions increasrs in the system. Due to high shcire of

payload processrs in the scheduler. there are more payload rollbacks under al1 situations.

Also in age-based algorithm the older process wins any collision thüt occur in the system.

The probability of a payload process winning a collision is higher than a non-payload

process. Since the rollbacks are charged to class. non-paylorid processes do not get the

allocated CPU share and thus do noi get 3 chance to run.

By ignoring blocking between payload - paylorid procrssrs in ihe jystem. more payloiid

processes set a chance to cxecute quickly Iroding to a rise in number of payload - payload

collisions (PP Collision Stat ). iVeasured results in Table 6 indicate this to be true for riIl

rollbrick cilgorithm. Refer to value of PP Collision Stat for grneric case and case with

where poyload - payload blocking is ignored. the count of collisions risrs by an average of

44%. By ignoring the blocking brtween pnyload - payload processes. a differencr in

results is noticrd for 5 PEs iis compxed to 3 PEs in the systrm. With 5 PEs in the systrm.

ihe effrcr of charging rollbacks to clah.\ shows a strong impxt idiit: to highrir blockins of

proceses in rhs sgstem with more numhrr of PEsi and as a resulr paylocid uhich owns ;i

large 3chedulr.r shars gets to run for l o n p prriods. thiis more püyload work is dont: in

the sysrsm. Duc ro this. the non-priylond processes thÿt own a srnall CPC shnre in !hr

schrduler anywiiy do not get thrir çomplctr sharr. Bo~inded - priority algorithm and

roll haçk-priyload algorithm show himillirities in mecisurcd results for the srnsitivity

analyis for a highrr blockine systtni ( 5 PESL Ssr NS Collisiori Stat and PS Collision

Su t in Table 6 h r hounded-priority and rollbrick-pliyload dgorithrns. The similarity in

the ruu rollbaçk d p - i t h r n s is dur to srriiilizing priority 2 processes in the systsm.

5.3 Results of Esperirnents

Rrs~ilts of the simulation experimrnts are prescnted. A discussion of the resulth obtainrd

with cach scrnririo is presentrd in sepuate suh sections as follows.

5.3.1 Scenario 1

This is a low blocking scenario. Figure I 1 a ) and 1 1 b ) show the graphs displayhg the

relütionship of ratrà ctvaillible power and usrful processing power with the number of

PEs. In Figure I 1 a ) and 1 1 b). it is observed from the sirnulritor results that the schedulsr

sharrs as detined in the schedulrr templatr are not maintüined. Refer to Table 7 and

Figure 7 (Section 3 . 3 for details. Table 7 describes the rnraïured dota during ~imul iu inn

wherecis Figure 7 presrnts the actuül schedulrr shrires. The main reason behind this is thlit

rollbacks i re char@ to clriss. The class. which is ç h q e d . grts punishrd and spends relil

time in tertns of rollbrick overhead. Thus the schedulsr s h m s arc broken and the

metisured CPC shürcs depend on the counr of ro1lb;icks ecich çliiss ssprrienccs.

.Althou;h the systcrn atteinpts to run the difkrent proçcss clasïrs in accordance with the

pre- allocoted schcdriler shares. the differrnt number of rolibücks incurred by the di fkrent

classes Icad to a drviritiori trorn these pre allocütcd shares.

Table 7: Measured CPU shares for different rollback policies for scenario 1

; Rollback 1 # of PEs CPU Util O/O 1 CP Ütil ?'O MAINT Util 0o i AUDIT Util % i Scheme l 1 I

In Figure 1 I a). it is obsrrvsd that for the case of 3 PEs. the RAP value for rollback-

payload schrme and age büsed schrn~ss are almost the same. As u e increase the number

of PEs that is for the case of 5.7 and 9 PEs. the age-hascd scheme shows a lower RAP as

compnred to the rollbtick-pa!4oad policy. In the agr-blised rollbuck schemr. due to Iarpx

scheduler sharc. for payioad processe.;. therr is a high probability of non-payloacl ivinning

a contention (probabiliry of heing the older job) between a ptiylocid and non-payloaci

procsss. As the niimher of processors increase. the systtrn sxprrirnces more blocking ;ind

thus more piiylorid rollbiicks according to the algorithm.

In Figure I 1 bi. CPP is quite stable for a11 rollbxk ülgorithnih. Simulation mctisiircnients

show thnt t h r bounded-priori ry rilsori t hm üc hieves the brsr SC heduler s harcs fur pa) lotiil

and non-pnyloüd (clusszc to the cictiinl CPC' sharrs in the scheduler> as sho\vn in Table 7.

I t dmionstrateh the best perfw-mance in terms of CPP and RAP ïcilues out o f the thrcr.

rollback dgorithn~s for $cenario 1 .

Sccnxio 1 is a low blocking sscnx-io. Since thrre are frwrr collisionr. more proceisei

art' iiblr ro commit ~ucçcssfully and thiis more work sets donc out of rhr sysreni. Scrniirio

1 is able to r i ch i rv r the highest ratcd ~i\*üilublr powrr out of the thrrr scrnario trstrd fur

rhis study. It achicves n ratcd üvnilable power of total 4.5 PEs wirh the rollbiick ztrritcgy

of boiinded-priority algorithm for a systrrn with 9 PEs.

6.2.1.1 Discussion

PI-r-sr~xrtioir oj' Shcii-es: The CPC shtirei, ric hicved to di fferent cllisses do not agres with

the intrndrd CPC shares. Refrr Table 7 for CPU ?;hues for scrnario 1 and Figure 7 for

actuül CPU shores riIloclitrd by the scheduler.

S c t r l d d i ~ ~ : As rsprcted both the RXP and CPP incréase with an increasr in the number

producc iiny bene f i t .

The CPP ;itt;iincd wirh a11 the dporirhnis arc conipnrcihlr. to one iiniither. Thus the policieh

thar produce louw c3pacity !Or p;iyloiid giw risr to ;i hipher non-piiylcud clipacity. This is

Iarger than non-paylotid. ii payload process i.r likrly to bs less old thün a non-püyiocid

back more oftrn leadin_o to a lowrr RAP and a higher non-pliplotid capacit!. for the tige-

boscd üI_oorithm in çomptirison to the othrr rwo.

5.3.2 Scenario 2

This is a medium blocking ~ r n a r i o . Figiire 12 a) and 12 b) show i.hc rffect of the number

of PEs on the rated avaiiriblc power and usefiil procrssing power resprctivrly. In this

scenario. the blocking level is highrr thm sccnario i . Thus the R.4P and UPP values for

sceniirio 2 are lowcr than scrnxio 1 for ÿ. given rollback str;its_oy or giwn numbrr of PEs.

For rxcirnpir. as shown in Figure 1 1 a) . the maximum RAP xh ieved u-ith 9 PEs in

scrnitrio 1 is 4.5 PEs. whrreas as shown in Figure 13 a) . the RXP for 9 PEs in scenario 7

is 3.9 PEs. In sccnlirio 1. i t is ügain ohscrved that the CPC shares of the classes arc not

maintüinrd. Sée Table S for detoils. The rcrson hchind this i.: thiit rdlbaçks are ch;irssJ to

ciriss. The class. which i.; chargçd. spends rra1 time in terms of rollback ovcrhecid. Thus

the '~chticl~iler 5 h~irt.4; Lire broken.

In wennrio 2. as the numbrr of proçrs.rors in the ysteni increüsr. (rom 2 IO 9. the

roll hacl;-p;iylod d_oorithm performs the heht out of the three rdlbock algorithni3 \i-ith

increlising niimher of PEs in the systeni. This is bccausr thi3 rilporithni atternpts to red~icr

the nunihcr of pii!Ioüd [ci non-pnyloitd contentions. Sirice ~i non-püyload Frocess ; t l \wv~

winz ~ i t priori[! O and 1 Irvcls. the proportion of time both piiylod and non-püyluicl run

on thr +stem is reducrd and rhere by the numher of püyload-non-pnylod contention?. 15

rrdiiccd. Thiis. rollbrick-püyloiid policy show the highcst RAP \.ciIiic.s oui of the thrre

rollbitck slgorithrns whrrras bounded-priorit!. rollbaçk nlsorithm \ho\\.\ the lowst v;ilurs

of RAP for 5.7 and 9 number of PEs in scenririo 2 .

3 5 7 9

Numbcr of PEs

(a )

3 5 7 9 Number of PEs

(b)

Figure I l : Simulation results for Scenario 1 showing (a) RAP, (b) UPP.

Table S: Mriisured CPU shares for different rollback policies for scenario 2.

Age-based hris perlrmcd w l l for scrnürio 2 as u*cll. it produces RAP and CPP value3 in

beruwn hoiinded-priority cherne and rollback-payload scheme. For 5 and 7 PEs in thc

systtlrn. agr-based algori t hm show sirnilx perlormancr as rollback-paylcmi scheme. This

is because. age-bascd scheme dso k~vors non-ptiylocid processes tsimilrir to the brhivior

in rollback-payload scheme) thus accountin_o for close RAP values for 5 and 7 PEs.

As shown in Fiprr 12 b). the values ofuseful proccssing powrr for a11 the thrrr. rollbtick

lilgorithms are reasonably close to one another. F o r an average lcvcl of blockin_~ case like

scrnario 2. mrasuremrnts show thiit the number of non-payload comrnits is high for oll

threr rollback schemes. compared to scrnario 1. As the blockins increlises. RAP goes

down. but the increasc in the number of non-payload commits compensates for the

decrerise.

1 Rollback / # of PEs 1 Scheme 1 / BP 1

/ 5

CPU Util X ' CP Util % / MAINT Util % ' AUDIT Util % i

91.5 1 41.7

I 1 1 92.5

l

25.9 / 23.8 1 l

d 91.5 1 63.2 1 12.9 1 14.3

80.6 5.9 1 6.4

I 9 1 81.9 1 3.8 [ 2.8 I 1

AB / ' 98.4 / 60.: 1 20.8 I

i 77.4 l

1 I

11.2 1 11.8 1 I

5

j ' 96.5 f 85.4 [ 4.8 j 6.7 1

1 l ' 9 1 98.1 86.2 : 5.5 6.3 I

97.2 74.1

PI-cstv-rrrriori tfShrrrs: The CPU shrires richis\.t.d hy difkrent classes do not rii!rse C urith

the intrnded CPU shares. Rrfer to Tnble S for CPLT sharrs for sccntirio 1 and Figuré 7 for

acrual s harrs allocnrsd by the schedulcr.

Scdtrbilin.: The RAP tichiwsd witii the boundrd priority and the rollback pqload

policies increasc with an incrrase in the number of procrssors. .Although iniiiallp. the

RAP produçed hy the a-e bascd iilipriihni incrraws wirh an incruse in the numhcr of

prowswrs. the R.AP x h i c i ~ d \i.itli 9 PE's ( Rr\P = 2.26 PEh I is not rignitic;inrl~ 1argc.r

than that x h i e i u i with 7 PE's ( RAP = 3.24PEs 1 . The rrttionrile for stich ii t?t.hai.ic.ir is

i inilar to that ohrerwd i n Scenririo 3 iind i h ilisc~rsstxi in detail in the tiillowinp Section

5.3.3. Boundcd-priority zchcmc and rollhiick-poylonci 3chernr.s m a k rhe sysrcni qiiite

iicalrible ris the RAP iincl 1:PP incrctise wi th incretlse in nuniber ot' PEr.

P~~~jimii<iirce coi ri pi^-isotr: Rollb;icl; papload dernonstrates the bcst R X P éspecirill!. sr

higher number of priwssors. It is initxesting ii? obsc'rvt. th;![ ahhoigh rollhac-k p;iyl«;id

discriminütrs agriinst payloail procrssrs i t achic\.rs the bcst RAP. This is brc;iusr this

algorithm attempts ro reducr the niimber of payload to non-püyloiid contentions. Sincc ii

non-paylocid procrss a l t i q s wins at priority O and 1 le\-cis. the proponion of timc huth

püp load and non-püylocid riin on the sy tem is rrducrd and thrre hy the numhrr of

paylotid-non-püylod contentions is rrciuçed. As discussed in the contrst of Scen;irio 1.

the age-basrd algorithm also favors non-püylotid procrssrs and produces a performance

that is close to that of rollback payload.

All the three stratrgirs main a comparable UPP for any given number of proccssors. This

indicrites that although the rollback algoriihms have an impact on RXP the ovrrall systeni

performance ( CIPP) remains unaltered when the rcillback algorithm is changed.

5.3.3 Scenario 3

This is n high blocking scenario. Figures I ? a ) and 13 b i show the rffeçt of the numbrr of

PEs on the rnted available powrr and the ussful processing powr . resprctivcly. In this

ssrnnrio. in rhr simulation rxprriments witli the iipe-b3sr.d rollbaçk schenir. r he svsrcrn

failcd to adjuit to CPLr utiliziition of 95ri. In a fsw cases. the systcm runs out of

mernoru and cannot âI10~ate an). more memory. .Au obwrved in scenario 1 and scenririo 2.

in cilses of bounded-priority rollbück polisy and rollbiick-pciyload poliçy. the schrduler

shlirss art' broken as well t See Table 9 hslo\v ;ind Figure 7 in Section 3.3 i.

.-\nothcr important observation O L I ~ of sceniirio 3 is that the system is sçcilühle iipto 7 PEs

for boiindrd-priority schrmr. With incressc in numher of PEs to 9 in the sysrcm. the CPP

and RAP drop dou-n. Thc ysteni is noi sciilablc with rollback-pajVlo;id scheme brçaiiw

C'PP and RXP of the systrm Jecrrcisr with increrise in nurnbcr of procrswrs.

- 3 7 9

Number of PEs

( a )

5 7

Nurnber of PEs

(b)

Figure 12: Simulation Results for scenario 2 showing (a) RAP (b) UPP.

Table 9: Measured CPU shares for different rollback policies for scenario 3.

pi.~!~loüd gr.111 more chance to run. Açcording to the ao_c-hüsed rollblick cherne. the oldrr

Roltback 1 Scheme BP

job wins and thrre is a hisher probahility that an oldcr job in the situation of a colli.\ion

# of PEs CPU Util 90 1 CP Util % 1 MAINT Util \ / AUDIT Util 1 l t t I

3 / 97.5 1 87.3 1 0.2 1 10.04 l l

with high contention levcl. rhc: count of overoll collisi«ns in the .;>*stem is quitc high.

Thrre is so much work in the w r k queues that the simulator ninb out of \.irt~icil meniun.

Figure 13 b ) shows the relationship brtwren ~iseful procrssing p o w r and the numbrr of

PEs. .An observation from Figure 13 a ) and b ) is that therc is more non-payload work

donc in case of rollback-pciyload algorithm. This is brcüuse püylocld rolls hack more often

dur to the design of the respective rollback strüte_oy. This aspect of rollbask payload is

more pronounced in this scenario which corresponds to the highest blocking level. thus

non-payload gets to nin more and more non-priylorid work sets dons oui of the system.

The RAP and UPP in the crise of rollback-paylorid algorithm _oo down with an increrisc in

number of the processors in the system. Bounded-priority policy demonstrates bettrr Rd4P

for 5.7.9 PEs than the rollbrick-payload policy for scrniirio 3. The nieasured results

indicate thüt for high numtxr of PEs. the bounded-priority ai_oorithrn can cichieve hishrr

ratrd wailable power than rollbac k-payload algorithm in a high blockiry scrnario ( rekr

to Figure 13 a). At highrr number of PES. as more piiyloiid prncrsscs lire run

simultaneoiisly on the systern. payload-payload contention brcomcs more dominant in

cornparison to non-payload-piiylolid contention. In rollbrick-pciyload stritrzy. the pciylo3d

is made to roll back in a collision brtwren pciyload and non-paylocid processes. This

fiictor is niore prominent in a h i s h blocking case likr rccn;irio -3 and thus accounrs for

dccre~ihc in RAP and LTPP with increasing numbrr of PEs. This docs not happen in

boundd-priorit!. schemc.. Hrre a hiehcr priority prowss alwap wins a collision hctivrcn

pnylorid and non-paglorid process. This improws t h s RAP ;ichiéwd ait h the houndrd

priorit). ;ilprithin in cornpririzon to rollbnck pci!.loüd cit highcr nuniher of PE\.

Ficure li li) shows thüt the boundrd-priority rollbücli schcrnt. results in a bel1 hhüprd

çurw tSr the ratcd ~i\.ailahlr p o w r values for 3.5.7.9 nuriiber of PEs. The ratcd a\.riilcihlc.

p o w r increasss with an increasr in numher of PEs uniil the numher of PEb in the systcni

is 7. and thcn it drops fur 9 PEs. Scrnario 3 is a high blocking swnxio. Due to hish

blocking. the count of priority 1 processes in the system is more. Sincs priority 1

processes in the system nin srrially. the machine is serialized fur longer periods thiis

siving lcss chance tu prinrity O and 1 procrsses to mn ro complriion (priority 2 procesc

always wins ovrr lowrr priority procrsses in a collision betwrrn the ~ 0 ) . Thus the RAP

values for 3.5. 7 and 9 PEs are low. With the increasr in number of processors from 3 to

7. as the blocking Irvel increasrs. the bounded-priority scheme irnproves in performance

for a high blocking case. In this rollbück policy. a higher priority procrss wins the

collision over 3 low priority process. As the number of procsssor increasr in the system

from 3 to 7 PEs. dur to the priority scheme (whrre a higher priority process wins ovsr a

Iowrr priority processi. the RXP of the system increliscs from 0.22 to 2.79 PEs. Thar is.

with increase in number of processors i n the system. the numbrr of payload prioricy 2

processes also incrsliseh. rrsuliinp in more payloüd wins ovcr non-payloüd processes as

thc numbcr of proc<fjsi)rs p up in tlie h>rtc.ni. thus sccuunting hr ri'ie in RAP values.

The observation from Figure I ? ai 2nd h) shows thüt the systrrn reaçhes a thrrshold ar 7

PE\ and the RAP drap\ down again rrhrtn thc systcni encouniers estrenie case of blocking

;it 9 PEs. Mrcisiircd resulis show a high nurnhrr of pliyloud rollbxks for 9 PEh. A siniilar

pattern is obscrwd in Fig~irr 13 b I iiherr CPP s h o w 3 groivtli iill 7 PEh and thrn drops

down for 9 PEs. This indicates thai t'or ii s y s i m eshibithg rht. blocking charüclcristic\ of

scc.n;iri» 3. ~ising linblocking or othcr more cl'fectivc rollback policies are ncceiscirv.

7 * 5 .~ .1 .1 Discussion

P w s t w ~ r r i o r i ( fS i r~ i i -us : The CPC shares achkvrid by clifkrent classes do not agrec ivith

the intrnded CPC shnres. R&r to Table 9 for CPC shnres for scmcirio 2 and Figure 7 for

lictual shares allocated by the scheduler.

Sccilohiliry Both the RAP and CPP rittaincd with the rollbacli payioad aigorithm drcrelise

with an increcisr in the number of processors. The RAP and UPP for the agr büsed

aigorithm increnses initially with an increasr in the number of PES upto 7 PEs but

dccrcases when the number of PES is changed from 7 to 9. A brief explmation for this

anomalous behxior is provided. As the number of processors increases there is more

livailable processing power to do useful work. As more numbcr of processes run on the

systrm howevrr. the number of collisions increascs for a givrn set of contention Isvels.

The system capacity attained is a tradeoff betwern these Fxtors. For rollbrick payloüd the

second factor dominates and performance dcgrades with an increüse in the numbrr of

PES. In case of the boundsd-priority cilgorithm. although the first fx to r dominates

initiliily. the second t:.ictor brçomes more prominent with 9 PE's and both RAP and C'PP

are observed tu decrcast..

Prrfi)n>rcrr>ce Cot~~prrisorr: The UPP iittnined with rollbaçk-payiond is higher than that

airaincd with houndcd priority cipecially when the n~iniber of processors is srndl. Bv

fworing non-püylotid processc.~. the diirütion of tirne during which non-pü~lond proceswi

and payloiid processes nin simu1t;tneously on the \!.stem is rrducrd: as ti result the

nurnber of non-payload-püyioctd contentions is rrduccd leading to hisher CPP for rhc

rollback pliylond algoriihrn. .At highcr nuniber of P E i more payload processes run un r h r

system and pciyload-payload contention dcminüteh. A s ii rcs~ilt the suprriorit!. of rol1h;ick

püyload is rrduced and hoth algorithms Jenionstrate LI cnrnpcircibie CPP.

For rrasons describsd in rhr p r r ~ i o u s prirligrüph: rollback payloxi drmonstrritc.; a

substanti;illy higher RAP in comparison to boundrd priority h r 3 PE's. At hipher numher

of PE's. as more püylotid processes arc run simultaneously on the system. pt~ylorid-

priyload contention bscomes more dominant in comparison to non-paylolid-pqlo;id

contention. By usin: a priority levrl 7 queue the bounded priority algorithm makes $Lire

that the priority levrl 2 processes complete without blockin_g. T h improvçs ihc RAP

achieved with the bounded priority algorithm in cornparison to rollback payloüd at hieher

number of PE's.

Number of PEs

3 5 7 9

Number of PEs

(b)

Figure 13: Simulation Results for scenwio 3 showing (a) RAP ( b 1 UPP.

Chapter 6

Payload and

This chliptcr h c r i he?

Non-Payload contention

component breaks down [Ben97]. MAINT is responsible for network and peripheral

maintenance. collection of operational mrasurements. TCPm communication etc.

AUDIT is a background process that scrins the integrity of the application or resource

data. detects data inconsistrncy. dçtects invalid states of the machine etc. With shrired

msmory architecture. contentions cannot be avoidrd. Thus tcchniqw?; are devslopcd thai

c m rnablr the system to evperiencr fewer contentions. In this chapter. auch (i technique i i

presentrd wherr the hishly contentious groups of shared-mrmory variables are identifid

for rlirninütion h m the input data trace3 in order to brin: down the contention Ièvel

brtwcen payload and non-paylorid.

6.2 The qnp fiictor

In ordrr t« control thc püyloüd - non-piiyload contention%. ;in estimrite of the probability

of blocking bctwt.cn the two proceh3 c l a w \ is required. The blocliin~ Irwl is n ~ u w r c d

in iermh of a hctor rekrrrd to as qnp. which is ihe conditioncil probability of blocking

when ;I pliyload procrss runs concurrently with a non-payload procrss. This probübiliry

factor is conditional upon the hct that püyload is running on one procrwx and non-

pqload is ninnine on another. Mcli~ures have been taken in order to reducr the value of

qnp. which results in hisher capacity of the systrrn. A tool thlit computrs qnp tiorn a trace

collected on a multiprocessor-büsed rrlephone switch is presentrd in this Section 6.2.2.

6.3.1 Algorithm for qnp script

The followin_j technique is ustd to calculate blocking factor bçtwssn pnyload and non-

piiyload procrssrs.

Payioad process sn PEI Non-payloau process on PE2 Sei of Sli:ired addresses i

~i = l lu ni

Figure 14: Overlap of data domain of a payload & non-payload process running in parallel.

In the Fip-cl 14. PE 1 and PEI arc running piiylocid 2nd non-payload processes

respectivrly. Figure 14 illustnitrs an exampic thiit shows that paylolid cind non-payio;id

lue ninninp in prirallel to rach othrr. A subset oTdaia exists that is shürrd bctwen thcm

and causes bloçkinp between thern. The motivations for the too1 discussrd in this çhiiptrr

is to idenrify the set of shared data and reduce the levrl of blockin_o bctwren pliyload and

non-payload processes [Kols97].

An oprrationül definition of qnp is provided by Dnvrigri [Dnv97] and the following

expression is used for the estimation of qnp:

For each location i shared by pa~~lolid and non-p;iyIoad.

Bi = (total lock timr of i by paylolid 1 total priyload tirne ).

Ci = i total lock time of i by non-p;tyload / total non-priyload timc 1. 2nd denote

pn)drrctA = n ( 1 - Ai)

The sri i i.; collection of mrmory addrrwb .;hard hy pqlond and non-pa>.load

proceam. A s shoun in Figiire 14. the rnrmory loçiition hhared hy pqload and non-

paylorid procrssea can be bundled in a set ohtaincd wi th thc ovcrlap of their data domciin\.

LOCI; timc ia drfined as the tirnr period in ~inits for which a process i piiyload or non-

pa!.lulitl) l o c h ;i particular memur> iidc1rr.s~ Juring i t h w n e r h i p of ihr: paniçular ~idrlrchh.

Total payload or non-pliylood tirnos arc delincd 35 the execution tirne. i.e. total iime in

tirne ~iniis that the rcspeçtiw piiyload and non-p;iyloiid p roc rws cmwrne during rhtiir

ext'cution.

Ai i b the prob;ihilit>. that (i rnrmory location i is hsld by a procw. The value of { I - A i }

is de fined as the probability thüt the memory riddress is not hcld hp riny process. and

prntktct.4 gives the probability of dl the memory addresses shrired by pliyload and non-

payload processes not being hrld by any procrss (no blocking).

Thus the vdue of { I - productA) is the probability that blocking would occur between

payload and non-payload process we r the shared memory addresses. where i denoies

rach shared address. Note that a uniform probability distribution is assumed: a rnemory

addrsss is accrssed uniformly over the tims payload and non-paylciod processrs nin in

parallcl [Dnv97].

6.22 The "qnp" Measuring Tool

.\ script was developed in the Ianguüge Perl for irnplcmcntinp the algorithm pressnted in

the previous sub-section. Düto is t'ed inio the script in the form of trace5 conttiining

rnemor) iiddrrsscs. thlit is ri mix of cirita for payloüd and non-payluad proçrcses. collccted

on ii reiil telephone suitch. [Dr\t97] .Vonel Nctworks ilse the script in thc following u a j r .

Dump fils3 cire collectrid in r h r form o f memur). ;iddresses that w r e açcessed on the rra1

teleplione witch. The prosram script i3 able to identif! cantcst su.itchris that ocçiirred in

the \ysirrn during the riin. the nieniory addrewh ricwswd bc twrn one contrst sivitch ro

anothcr. and the ichrduler class c x h p r » ç w belongrd 10. Also i t ha3 11 tkaturr of

idcntifying the rype of process. ahrther i t is ;i piiyloiid or non-payload procesh. M'ith the

help of X.4-Core ream ot'dr\xIopers. ünother procedurr. hiis been deidopcd thiit cniihlcs

the user to r d ci~ntentioiis data by how oftrn rhe Jatü bloclis. Sor~e l Xriivorkh designers

are drtdoping ti prosrss called unblocking by whiçh it is possible to re-snpinrcr the

esisting code and rliminatr blockins at a spccitic set of nirrnory tiddrezses. It is not

nrcrssciry ro unblock a11 conicniious data. rrither only critical oncs those hlock ovcr 2nd

ovrr again. Conirntious data itrms are sonèd by thrir holding tirnrs form a hotlist and the

most contrntious items near the top of the list are known as hotspots. The rool providrs

an automatrd means of generating a hotiist. Xftrr converting the location address to

program symbols. designers identified the contrntious data structures and could evciluüte

how critical the- were to the blocking removal program. The list of the rnozt contentious

addresses refrrred to as the '-hottest addresses" is crrated. In order to investigare the

impact of code re-engineering for removins contention on this address set. the hottest

addresses wrre removed from the inp~it traces. As a part of this stiidy. the tool was

moditird such thnt when the program runs through the unblocked traces ayin. i t rxcludes

w x y timr i t çonies across an address out of the hot riddrru list. The pertorniancr

calctilations ufrrc redonr: cvrry rime certain locations were redesmed to be unblocked and

removcd (rom the list of contentious location.;. With ihiz itentivr procrss. i t wiis possible

to arrive ~ i t an optimizcition program thlit could mrrt caplicity ohjrctiws by bringing

down thc value of qnp. In the rsprrinirnts dont: for a paniçuliir set of input data tracts.

the q n p u lue w s reduced from 0.57 to 0.17 with the h d p of this tool.

6.2.3 Recalculated qnp value

Wr cspcrimcnird with two hets of input data one with qnp = 0. 172 and ihc othrr u ith

qnp = 0.036. With the rrduction in the value of ynp froni 0.1727 to 0.036 <u.ith the abow

technique 1. the riited nvailablr powrr of the systrni increases from 3.1 PEs to 4.5 PEh for

a 9 PE systcm. RAP of the systeni is cdculatrd with the formula discussrd in Section 5.2

of chaptrr 5. Figure 15 presents the RAP values. Capaçity of rhr systrm shows an

improvemrnt brcwse of rrduction in blocking lrvcls berwcrn payload and non-pqload

processes. The Iowcr the levels of blocking. the iessrr the numbrr of collisions. the

oreater the amount of useful work done and the higher the cripacity of the systern. C

No. of PEs

Figure 15: The effect of decrease in value of qnp on RAP.

from its allocated time in the scheduler sharr and sprnds i t in cost associated with the

rollback. This cosr incurred by each class is not necessarily proportional to the

corresponding c1ass shxe allocatrd by the scheduler. Thus the cliiss shlires as iillocatrd in

the scheduler tempiatr canzot be accomplished and are broken. Thers are three rollback

stratqies investiptsd in t his thesis. These are the agr-based roll back schrmc. the

boundrd-priori tv rollbiiçk scheme and the roll back-piiyload sc hr nie. Dur to rhr

combination of charsing rollbacks to cliiss and the design rilgorithms of rollback schemes.

i t \vas notiwd thrit the schedukr shares were broken. This vio1iitt.s one of the rriost

important requircments of the telephone switch.

The factor p-value is defined ns a ratio of ril i ip~d tirne whrn bloçking occurr. mciisured

Hom the s~ibcdl starting rime and the rxecution tirne. If a proces starts and çollidcs right

a\\.civ or blocks in close vicinity of ils stiirting point. its p - ~ Y A I ~ is small. In cahe of agc-

brizcd rollblick policy. rhri older job \vins the collision. P:iyload being the clash havinp the

Iarprst schrduler shnre gets the rnasimum chance to sxcc~its. In the case of age-brisrd

rollback idgorithm. when a frresh paylolid procers stans exscuiion and hlcicks with a non-

püylood process. therr is ii high probribility ol' paylorid h n i n ~ a srniiller p-vüliie as

cornparrd to non-paylocid procrss. The probiibility of non-payload process winning the

collision brtween payload and non-payload process is more. as non-priyload process

would normally be the older job (dur to lowrr share allocrited io non-paylorid processrsi.

Thus payload is chargeci more rollbacks the cost of which is nor proponional to its

schedulcr shxes. As a result the rollback policy is unable to achieve the designated

scheduler shxes.

in case of bounded-priority rollback algorithm and rollbrick-payload algorithm. the

combination of rollbück algorithms and c h q i n g roll biicks to class breaks the scheduler

shürrs as urll. Both the rollback schcmes involvr a concept of serialization of priority 2

procrsses. Whilr a priority 2 process is mnning. another priority 2 process cnnnot run.

Othrr lowrr priority processes cnn run but i f they collids with a priority 2 procrss thrn the

highcr priority procrss wins. This factor cliusing srrialization of processes ow-powers the

allocation of scheduler sharrs and is a reÿson for brealiinp thsm. This implics that due to

the m%.dizcition of a priority 2 procesh. miin? other priority 2 processr3 could be witing

in the queue to be nin. Since d l thesr procrsses need to hc esecuted seriülly. the

processors in the sytcm can be forced to br idlc for longer periods. This rxpénds CPC

tirne of the qsiem and makrs i t hard for the schrdiilsr to providr the nllocatrd CPU s h x r

for s x h c'lass.

7.1.2 Scala b i l i t ~ of the system

The system is bc;tlablr: for qc-biissd rollbacl policy. houndrd-priority policy and

rollbnck-püyload poliçy for low r scenariu 1 and medium i scrnlirid) blocking sçenarior.

For high blocking scenario (scrnnrio3 ). an obaerv;ition is made that cise-based rollbacli

scheme does not work due to extreme blocking. The rerison behind this is the hct that

a p b a s r d policy favors non-payload processes. Dstails arc prrssnted in Section 5.3.3.

Though. bounded priority policy results in low RXP of the system. the systsm is still

scalable upto 7 PEs and the RAP drops down for hizhsr number of PEs. Rrsults show

that in case of very high blocking (scenario3). rollback priyioad is not scalable at al1 and

RAP decreases with an increase in numbsr of PEs. This is due to the fact thrit the rollback

pay lorid favors t hr non-payload processes.

Thus bounded-priority algorithm i s the only one that assures scaliiblr systems up to 9 PEs

in çass of low and medium blocking scenarios and u p to 7 PEs in high blocking scsnxio.

Due to propr ie tq issues. a stlitemrnt cannoi br made on numbrr of calls that eo through

the mua l X.4-Core telephone switch. Thus no turther comment c m he rncidr: on

sclilribility of the switch.

7.1.3 Cornparison of Rollback Policies

.Ab said iihow. none of the rollback policies help in mainrainin: the xhcdiilcr ahare3 of

~Ia'iscs. The CPP cittained with 311 the algorithrns Lire coniparable io one cinoihrr. Thus the

policies rhx produce lowr capcicitu for piiyloiid g i w r i h r to ri higher n o n - p q h d

capaciry. This is intuitive in case of ro1lb;ick pciylod: by f;lwrins 3 nnn-p+-loatl proccsh

thc ;ilgorithm zi'_ive\ riss to a higher non-paylocici ciipncitv. u-hich is acccimpnied hy a

proportional drcrecisr in payload ciipacity. Aiso. age hiised fai\.ors non-priyloüd proce5sc.s.

Thus. knowlrdgc oT ags of the procrss or iis clriss type does not seem to üdd riiuch bcnefit

to the rwmll systrm performance as ciiptured by UPP. Boundcd-priority policg performb

well in low and medium blocking scrnario at hish number of PEs and contributes to

scalnbility of the system. [n the worsr crise of blocking (scenario 3 i. boundrd priority

policy shows a good performance. It produces Iow RAP in the sy t sm but dors sclile in

systems with upto 7 PEs. Thus boundrd priority policy crin br stated to perform the best

in case of high blocking in the system. Rollback payload policy 1s the ncxt brst policy for

Iow and medium-blocking cases of input out of the three policies experimsnted in the

system.

7.2 Scope of Future Research

Explainrd in rhr Section 7.1 are the conditions that prevent the simulation systrni lrom

reaching the capacity _ooals. A valuable a x a of rcsearch would btr the dcsign uf new

rollh;ick ;iio_orithnih ihrit \vould ensure none of the îbovr: iruütions thar have a tendcncy

of breakhg the achciluler sharcs. Another arca thü t can he ssplored coiilci be charring

rollbacks to rystem insterid of clliss. Anorhrr rrsenrchcr is estending the prtiaent

miilriprocessor ~imulütion. to br able to charge rollhacks io thc system. Thih woiild ht: a

way to compare the n r w rollhack-chargins schenic with existing rol lback schernes and

cinülyze cüpxity of the systern. It d s o givea a chiincc to csperiment with the nrw

chxgins 2ptr.m uith new rollback policies ( k i n g designcd ai Sortel Set\wrlts i ai ri hier

point.

The des iy of rollbock policica is ü vrry senxititpe arra to euplore. The thrsis results have

shown thlit the combination of charginp rollbricks to ciriss and pünicular rollbnçk designs

could break the schrduler shares. This has rnçouragrd Nonel Netuorks to explore ncw

d r s i p for rollback policies. New rollback algorithms rire undçr study by somc

developrrs ai Nonel Nrtworks. Due to non-disclosure matters. the details of such

algorithm cannot be disclosed.

7.3 M y ideas to future research

7.3.1 Algorithm 1

An m a of futurs rssearch that I would likr to swgest is towÿrds design of ri nrw roilback

algorithm callrd "S hm-check before rol1b;ick" whcre the following mlrs apply for al1

types of processes ninning in the y t c m with priority lrwl O and 1 . The wheduling Iriycr

would requirr most of the re-designing for this algorithm. Ar al1 timrs. the shnre

accomplished hy a giwn class of proccsac~ is notrd and Ç ~ I C L I I ; L I L I L ~ ils to ~ O W t'dr iz i t from

oççupying its tutai s h x r in the scheduler templatr. Thc following es~rnplr. is provideci to

es plain the concept of the nrw 1-01 1 bück :ilgorithm. Consider that hloçkinp occiirs h e t ~ i w n

procesa A and procesa B. u.herc procrss A has urilized say 2 2 5 of itr shrire oiit of total

allociitrd bharr. u f sa> 6bC2 in the ~clirdiilei ternpl;itt.. On the other hiind. procrss B bas

utilizd oiit of 365 ol'itz * h m . Th~iz t i l l the point o f hl«ckin_o. proccss .A hlis

utilized 33.Y;- of i t h i r c lvhcrc a* psocrhs B h35 taken up 36.65 o f i t b s h ~ r e . Proce:;?. .-\

iz Ia~_;in-_ hehind procehz B in t e r m ci' utilizing its s h m rind thcrehre processes A

shoiild bc a l l o u ~ d t« irin the collision. This tu!. tve can assure procrzr.5 k i n g cible to

maintain their ~chrduler shüres. Xote that in rhis algorithni. a priority 2 proccas riI\iqs

wins ri collision with a priority 0 or priority I process. Also a priority 2 job mliy serializr

the machine as i t does in boundrd-prioritu algorithm.

7.3.2 Algorithm 2

This is anorher algorithm called "Predict cornpletion time" that is based rnostly on

prediction. Let us drîïnr another facior: c-value whosr definition is as follows. It is the

ratio of time lrft to the prrdicted complction tirne to the predicted total complrtion time.

If thrre is a collision between two processes of differrnt priority levsls. ihrir c-values are

comp~ited. The proces.; with lowrr c-value should br alloued to win thc coflision. L0a.c.r

c-value implies that the process is closrr to çornplerion [han the other proces5 involveci in

collision. In O L I ~ case. complrtion point of n procrss is not predictiible. In iimulationh. wc

can do thüt by checkhg out the wight associated ivith cach suhc;iII in the rnemory traces

which basically indiclites how long the subccill would run for. Simulüt ion measursnir.ni\

can alho çalciilate average work timrs ( AWTi of a wbccill. Prediçtion on the haais of

mrü5iired AWT of paylond and non-payload procesres c m br. donc to judge ho\\ Lu the

subciill is from completion. For csrimplc. in mir simulation measurements. mcasurrd

XWT for a payload procrss is ribout 10 timc tiniis and wrieh from +Y> IO -Y+ hti tuwn

esperimcnts. Similnrly AWT idr non-püyloxi d s o lies within ü range of values. C ~ i n g

this knowlrdgc of values. the subcall lengthh crin be predicted.

References

[Ban971 Brningrr B. "XA-Core: Guide to hlultiprocrhing". Nortrl Nrtworks. 1997.

[CofflS] Coffman E.G.. Carey M.R.. Johnson D.S.. "An application of Bin Packing to

Muitiproccssor scheduling". SIAM Journal of Cornpuring. vo1.7. F e b n i a ~ 1978. pp. 1 - 17.

[Dand9>I Dandamudi D.. Eagcr D.L.. "Hot-Spot contention in Binary Hyperciibe

Networks". E E E Trans. On Cornputers. vol. 4 1 . no.,. Frbruciry 1991.

[Drw96.1] Dnve iga T. "Ca!culating ccipücity degrridarion bu nxmory contention betwcen

non-pqlond and pn~.locid". Nortel N r t w r k ~ . 1996.

[Drw')6.7 1 Drwei2;i T. "S h m d Slrmory Contention cind i t h Impact on hIultiprocr~a«r

Cal! Control Throughput". 1 jlh International Tclctraffic Congrchb. June 1997. pp.22-27.

[ D r ~ 9 7 ] Drwcigri T. "Forrnulab for Cnlculat ing qnp". Sortel Netu-orks. 1997.

[Jiiin') 1 J Jüin R.. "The .Art of compurcr S p t c m Perhrmtincc Xnalysis". Techniques tQr

Espcrimentd Design. klrasurcmenr. Simulation. and Modttling Wikv Prok.r\ional

Compiitin_o Puhlicntions. 199 1 .

[JalS9] Jalote P. "Fault Tolerant Processes. Distriburd Computing. Sprinser-Verlas.

New York. voi.3. Xlarch 1989. pp. IS7- 195.

(Juan9 11 J u n g T. and Vrnkiitrsan S.. "Crash R e c o v e ~ , with littlr Overhetid".

Procrrdings of the 1 lIh International conference on Distributrd Computing Systerns. >ln-

199 1. pp. 454-46 1.

[Koo87] Koo R.. and Toueg S.. "Cheçkpointing and Roll bxk-recovery for Distri hutttd

Systems". IEEE Transactions on Software Engineering. vol. 13. no. 1. January 1957.

pp13-3 1 .

[KimSSj Kim. K.H.. "Programmer-Transparrnt Coordination of Recovery Concurrent

Processes: Philosophy and Rules for Efficient Implementat ion." IEEE Transactions on

Software Engineering. vol. 14. no.6. ppS IO-S2 1. June 198s.

[Kolsc17] Kolski S. "Determinins Data Domain Overlrip

Trrift'ic Data". Yortel Networks. 1997.

[hlajLic)7] Ma-iumdar S.. "Effectivcnrss of Sçhrdulins Poli

Contention in Multiproce~sor Systems". Decmbctr 1999. 2 1 p;igrs [submittrd for

publication 1

[Maj99.1 ] Xliijurndar S.. "Scheduling Sti:itr.gir.s t'or Controllin2 Resource Contention of

hlultiprocrssor SysternC. Procctedings of the 6"' Interniitii,nlil Cmkrencr. on High

Performance Cornputing ( HiPC '99 1. Calciitta. Indiii. pp. 15 l - 157. Drçemhrr 1999.

[IliijW.2] Najumdor S.. Strcibcl D.. Bcninrer B. Ciirroll B.. Vrirrna Y.. Liu M..

"Coniroll ing ineiiiory contention on ti scülshlt: mulriproçrs~«r-bo3cd rclephonr suitch".

Es [en Jed .-\hsirrict: Industrial Esptxienw. Proccciiings of .-\CM S IGMETRICS'99

Con fer ence. .Atlanta. Mn! 1 999. pp2lS-229.

(Pf is05] Pfistrr C.. Norton V A . 'Hot-Spot Contemion and Ci>nihining in .\lultiStlige

Interconnrction Networks". IEEE Trms. on Compiitrrs. vol. C-74. No. IO. 0çtobt.r 1995.

[Prop96] "PROPHET User Guide and Refcrencc bllinuül". Re lsasct 2.0.3. Document

Issue 1.3. Nortel Networks. 1996.

[Wb941 Silbrrschütz .A.. Calvin PB.. "Opernting Sytems Concepts (Founh Edition 1".

Addison-Wesley. 1994.

[Sin@] Sinphd M.. Shivaratri 'I.G.. "Advlincrd Concepts in Operüting Systems".

McGraw-Hill Publiczitions 1 994.

[S tre971 Streibel Dale. "Simulation Study of a single class multiprocessor system". Norte1

Nstworks, 1 997.

[Wood931 Woodside C.M. and C.G. Monforton. "Fast alIocririon of Processes in

Distributrd iind Parallrl Spsiems". IEEE Triins. On Pariil le1 iind Distributrd S ysteins.

voi.4. no.?. Feb. 1993. pp. 164-174.

[Wu901 Wii K.L. and Fuchs W.K.. "Errer Rrcnwry in S l i m d 4lrmoq Multiprocrssor

Machines ~is ing Priwtr CacheC E E E Transactions on PartiIlel and Distributrd Systrnis.

vol. 1 . no.:. Apri l 1 990. pp. 22 1-240.

Appendix

Cirtss Inheritzince in the 3-clriss simuhtor code in our s t u d ~

The following diagram shows a pictorial view of inheritlincr. and relationship between

objrct classes as stmctured in the sirnuliitor code. The base clriss in a Prophct mode1 that

dstines a task is the c1;tss Task. SOS Tzisk is the clriss that inherirs charricteristics trom the

SOS Task s

Figure 16: Class Diagram of object classes used in the simuiator.

Base cI;1ss in Prophet and posxsses SOS btihwior. The CALLP. MAINT and ACDIT

servers are object ciasses constructeci in the code that inhrrits ihr. attrihutes and

ch;irricteristics of a SOS Task. Description of Client5 and Ssrvrrs rire provideci in Section

6.3.