Software Processes in BaBar Gabriele Cosmo CERN - IT/ASD [email protected] 3 rd LHC Computing...

Software Processes in BaBar

Gabriele CosmoCERN - IT/ASD

[email protected]

3rd LHC Computing Workshop

Marseille, September 28th- October 1st, 1999

September 29th, 1999 Software Processes in BaBar - G.Cosmo

2

LHC Computing Workshop - Marseille (France)

Outline

The BaBar environmentTraining and courses Tools for Software developmentDocumentation and HyperNewsThe Release mechanismThe User EnvironmentConclusions


3


The BaBar environment

More than 70 collaborating institutes distributed environment and computing

resourcesCurrently supported architectures/OS: UNIX

SUN, OSF, (in a near future also Linux)Programming Languages:

C++, C, Fortran wide spectrum of expertise

need for proper training to OO Programming


4


Training and courses

OO Methodology: Object Mentor Course Principles of UML and OOD Advanced OOAD and Design Patterns See also: www.oma.com/Brochure/training.html

Course on C++ programming languageCourses on specific BaBar software

applicationsOrganized regularly according to BaBar

collaboration meetings schedule


5


Tools for Software Development

CASE tools: Rational-Rose identified a team of experts for helping in

designingSoftware QA tools: Code WizardRun-time checkers

Insure++ , Great Circle, PurifySpecial builds regularly performed over

“stable” or “production” releasesresults automatically posted to the package

coordinatorsummary published on Web


6


WWW and HyperNewsDistributed environment

WWW plays a fundamental rule !Software standards, requirements & policesMeetings, minutes, slides, tutorials, ...Calendars, release schedule, computing resources info

HyperNews: discussion forum organized by topic uses BFMAIL users are free to subscribe or unsubscribe possibility to add new topics the Release mechanism is integrated a special topic: authoritative software recipes


7


Documentation

FAQ: to focus on the most common problems

Workbook: collects docs for users & developers

LIGHT: code browsing & navigation Code documentation

LaTeX, FrameMaker, HTML Cross Reference

KDOC, Javadoc


8


Production QA

Dedicated QA team & QA tools (sim & reco) at every new release: reference data from old

release are compared and the new reference set is updated

A set of references is needed for each different run type which the QA Production Tools use in their comparison testshistograms & data

automated mechanism in production main results published on web


9


SRT - Software Release Tools

Originally developed in BaBar by Bob Jacobsen, first release in November

1994Based upon CVS: the Version Control

System the actual code repository

Uses GNU MakeIt’s a Package !

set of GNUmake scripts (Tools)


10


SRT: What is all about ?

Set of tools (GNU make scripts) useful to: define the release environment build software releases on different

architectures and compilers in a homogeneous manner

define common rules used in the GNUmakefilesDB setup, schemas, libraries, binaries, docs, tests

provide utilities to control and make easier the release process in a distributed environmentimportrel, importver, updrel, rmrel, auditrel, statusrel,

listtag, ...


11


Packages

Package: a logical collection of classes/functions

providing a set of services and performing a well defined task (Booch Category)

Maintained as a separate CVS module Contains: code, docs, a GNUmakefile, scripts… A package coordinator (or more) is responsible

for:the package development and testingtagging and announcing the package for the release


12


Versioning and Tagging

CVS is used for versioningFixed tag format, i.e. V01-25-08[-02]Tags are also used as names of subdirectoriesA tag can survive to more than one release

until a new tag is announcedRequest for creation of a new package must

take place some time before the scheduled release, specifying: tag, coordinator, README file


13


The BaBar Software Directory tree

Access to CVS modules can be controlled via AFS ACLs if required. Package coordinators are notified automatically when new code is checked in.


14


A Release in BaBarA snapshot of a consistent set of package versions

(online, offline, utilities, scripts, makefiles, …) a general rule: never use the CVS HEAD !

It includes libraries and binaries for each BaBar supported platform more than 300 libraries and 80 executables

Built every two weeks. Indexed by a release number i.e. - release 8.2.7

Test Release - a successful buildCurrent or Production Release - a stable tested

releaseNewest Release - the last available release


15


Release: the Working Model - 1

Package coordinators release their package by testing, committing, tagging and announcing their code for a release and/or nightly build Automated system based on Web, HyperNews and BFMAIL

Nightly builds start automatically each night on all supported platforms, collecting all announced tags. problems are automatically posted to the PC the day after

Every two weeks, at the scheduled day & time, all tags are collected and the release process starts Problems are automatically posted to the PC the day after A summary of the build is posted together with notes by

the Release Coordinator


16


Release: the Working Model - 2

Automated testing based on specific tests provided by the package

coordinators can be run automatically at release stage output compared to supplied reference

Error from all release stages are automatically posted to package coordinators problems not fixed are “gently” reminded every 7 days

by the BaBar Problem Tracking System

Outside release builds, bug reporting and tracking is controlled through Remedy ARWeb based on ORACLE


17


The User Environment - 1

SRT requires few environment variables and entries in $PATH and $MANPATH variables

Automatically set via centralized HEPiX scripts

Test Release - the user copy of the original release structure for a specific version newrel, addpkg bin, lib, tmp, shlib, shtmp, doc, test,

GNUmakefile


18


The User Environment -2

Only packages added in the Test Release are built

The final executable is linked against libraries of Test Release and the correspondent SRT release

The user can add any version of a package example: addpkg L3Trigger V00-03-11

SRT’s rules can be extended for a given package custom rules/targets in the package GNUmakefile keep the SRT’s standard GNUmakefile body


19


Tools: Workdir

It’s a (simple) package Includes a “special” GNUmakefile integrated in

the SRT environmentAllows to the user to run applications from

every area (even outside the user Test Release)

Ability to choose in a flexible way the release (or Test Release) to which refer to

Creates automatically links to major scripts


20


Tools: PackageList It’s a package !Contains information pertaining to every

package which is part of the BaBar software release system example: PACKAGELIST += L1Sim ifneq ($(LINK_L1Sim),) override LOADLIBES += -lL1Sim -include L1Sim/link_L1Sim.mk endif

Requires a file link_$PACKAGE.mk in each package specifying its dependencies over other packages creation can be automated

Objective: define a standard link orderRelies on good consciousness of the developer


21


The BaBar Application Framework - 1

It’s a package ! developed by D.Quarrie

Implements a Modular approach a module is a fragment of executable code which has well-

defined interfaces and perform a well-defined service modules are totally independent each other, have their own

configuration and data-taking run specific information each module provides an interface to the Framework which

includes a unique name and functions to be called at the beginning/end of the job, data-taking run, event modifier


22


The BaBar Application Framework - 2

Modules are associated into Sequences a sequence of modules is a list of one or more modules

executed in the order they appear the order of execution and the messaging of input run-

time parameters for each module are controlled by Tcl scripts

the event is the central data-structure and is designed to be an “object dictionary”

other concepts: path, output stream, processor

Automatic regression-testing of a single framework package integrated in the system


23


Conclusions -1

OO Methodology (UML) Training and tutorials for developers

CASE tools for analysis and design of software units

QA tools integrated in the software environment available for daily software development

Use of WWW and BFMAIL documentation & communication

(user/developer) automated release mechanism


24


Conclusions - 2

SRT+CVS+HyperNews a working combination of products for Software

Processes in a distributed environment of a modern HEP experiment

SRT: originally developed in BaBar, now adopted and constantly customized by more and more experiments world-wide: FNAL, ATLAS, CMS, CDF, D0, BTeV requires an effort and a certain degree of responsibility

by the user (Package Coordinator) to achieve the best use offers a flexible and efficient user environment can be easily extended with new tools


25

LHC Computing Workshop - Marseille (France)Sometimes, no matter how hard you work, you

can’t meet management’s objectives

Software Processes in BaBar Gabriele Cosmo CERN - IT/ASD [email protected] 3 rd LHC Computing...

Documents

Transcript of Software Processes in BaBar Gabriele Cosmo CERN - IT/ASD [email protected] 3 rd LHC Computing...