Code Development on NTCode Development on NTSome notes on experience gained in developing cross-platform Some notes on experience gained in developing cross-platform

(NT/Unix) HEP software for the Linear Collider Detector(NT/Unix) HEP software for the Linear Collider Detector

More of a Random Walk than a Formal Overview of Development ToolsMore of a Random Walk than a Formal Overview of Development Tools

HEP-NT workshopSLAC - October 1999

Tony JohnsonTony Waite*

(tonyj,apw@slac.stanford.edu)

* In absentia

NT Code DevelopmentNT Code Development

Topics– NT Code development tools for C++/Fortran/Java– Code development issues– Code management

• SourceSafe• CVS• DEC -

– Using NT as a data server– Conclusions

Windows NT Development ToolsWindows NT Development Tools

For windows only development, tools are excellent:– Visual C++/J++ (MS Java)/Compaq Fortran

• Excellent IDE allows development in all three languages

– Integrated editor, cross-language debugger– Integrated build tool (make clone)– Incremental linker (that works)– Integrated code management (visual

sourcesafe)– Object browser

– SLAC has site license for MS developer tools

C++ for cross platform developmentC++ for cross platform development Cross Platform C++ development has many problems

– Makefile for each platform• Need to build/debug on each platform• Versionitis

– External dependencies– Compiler flags

• Even egcs compiler has many system specific flags– Different methods for building DLL’s/C++ name mangling– Compiler bugs– Differing Levels of ANSI/STL Support – Architecture specific bugs

• Found bugs in both Visual C++ (5.0) and egcs compiler for intel.

C++ for cross-platform developmentC++ for cross-platform development

Certainly possible - Simulation package does work on both NT and Unix.

Large amount of time goes into testing/debugging on

different platforms.

Java for Cross Platform DevelopmentJava for Cross Platform Development

Have succeeded in writing entire reconstruction/analysis package in Java

Currently using Visual J++/Sourcesafe– All development currently being done on NT– JAR files built on NT and put on web server for

download by others.

Java code can be run– Standalone from command prompt/batch job– Inside Java Analysis Studio (JAS)

Java Analysis StudioJava Analysis Studio

Cross Platform Java GUI for data analysis

Java for cross-platform Graphics/GUIJava for cross-platform Graphics/GUI

Java ExperienceJava Experience

Write Once Run Anywhere– Really works (for applications, if not applets).– Develop on NT, runs without changes elsewhere

Developers have been able to learn language and tools and get up-to-speed fast

Performance - Adequate, getting better all the time

Need to abandon Visual J++, Source SafeCluster Finding

0

0.2

0.4

0.6

0.8

1

1.2

Virtual Machine

Sec

on

ds/

Eve

nt

JDK1.1.8 -nojit

JDK1.1.8

MS 5.00.3177

IBM1.1.7

IBM1.1.8

JDK 1.2.1 Classic

JDK 1.2.1 HotSpot

Track Finding + Fitting

0

5

10

15

20

25

30

35

40

Virtual Machine

Sec

onds

/Eve

nt

JDK1.1.8 -nojit

JDK1.1.8

MS 5.00.3177

IBM1.1.7

IBM1.1.8

JDK 1.2.1 Classic

JDK 1.2.1 HotSpot

CVS for cross platform developmentCVS for cross platform development

CVS is available for NT as well as Unix

NT tools include:– line mode tools– WinCVS

• Rather confusing GUI– jCVS

• Cross platform CVS GUI written in Java• Doesn’t support ssh access to remote repository?

Several tools available for web based access to code– CVSWeb - perl script– jCVS servlet - works well on NT

jCVS ClientjCVS Client

jCVS ServletjCVS Servlet

CVS security issuesCVS security issues

Supports distributed access to code repository– bewildering array of access methods/security

issues• rsh - no longer allowed at SLAC• pserver -

– needs custom server,– passes weakly encrypted passwords over

network• ssh

– doesn’t work with all CVS clients– complex to set up client– no AFS token support

CVS CVS vs.vs. Sourcesafe Sourcesafe

SourceSafe CVSGetting started Install + Play Read the manual

Integration intoVisual Studio

Yes No

Cross platform No YesRemote Access Not really Yes

Laptop Friendly Not really Yes

Is CVS sufficient for a physics collaboration?Is CVS sufficient for a physics collaboration? If using Java - probably

If using C++, not really, – Need to be able to switch between configurations

easily• Logical names where are you?

– Location of external dependencies– Would like to use pre-built dll’s where possible.– Want to isolate users from system dependencies

as far as possible.– Want to avoid “nightly builds”

Need to layer tool on top of CVS, make– LCD Solution:

• Development Environment Controller (DEC)

Development Environment ControllerDevelopment Environment Controller

Written for LCD by Tony Waite, could be used by others– Hides details of file organization from user

• User deals with subsystems, Gismo, Recon, etc.

• Easy to switch between PROD,DEV,Test version of each subsystem

• User Profiles allow saving/restoring environment.

– Hides machine dependencies of BUILD operation• Automatically archives CVS version numbers

for production BUILD’s

Running a (Java) Data Server on NTRunning a (Java) Data Server on NT

Use NT Service Manager to Run Data Server– Microsoft Java has built-in support for running as

NT server• Application can be distributed as a single .exe

which installs itself as an NT service– Other Java implementations can be run as an NT

service with 3rd party add-ons (or roll your own).

Error logging using NT event log/viewer

Desktop Client

Network Data Server DIM

Remote Data

NT Service - Pros and ConsNT Service - Pros and Cons Pros

– Auto start on reboot– Runs in background– Use service manager to start/stop service

Real life problems– Does not auto restart

• Application can unexpectedly pop up dialog box!– Remote control difficult

• Remote Server Manager only distributed with NT server (or with NT resource kit).

• Requires port 137/138 access (blocked at SLAC firewall) or VPN/PPTP

Net Meeting for Remote ControlNet Meeting for Remote Control

Netmeeting 3.0 has “Remote Desktop Feature”– If run on server allows remote access to server

desktop from any NT/95/98 machine.– Free!

Security– Claims to use encrypted communication channel– Requires NT login to access remote desktop– Security dubious

Potentially very useful, perhaps worth considering

NT Disk Cache ProblemNT Disk Cache Problem

Problem with reading large datasets– NT will grow disk cache until it takes 99.5% of physical memory,

regardless of who/what else requires memory.– Reading large datasets can result in NT totally grinding to a halt.– No system parameter to limit disk cache size

Workarounds– If you have access to call to C open statement you can turn off

disk caching (big if)– Can run program which constantly flushes disk cache

Conclusion– NT not ready for primetime data server applications– Windows 2000 better?

Things I didn’t talk aboutThings I didn’t talk about

Because I don’t know anything about them– Sniff+ - Cross platform IDE

• http://www.takefive.com/– Cross platform GUI development with C++

• QT Widgets– http://www.troll.no/

– Libtool/configure• NT ports exist but not for non-experts

– cygwin• Unix environment under NT

ConclusionsConclusions

Tools for code development under NT only are great– Development tools probably better than Unix

Cross platform development– In C++ is a lot of extra work– Java can make things much easier, especially for:

• GUI applications• Client/Server applications

NT for HEP batch/server side data processing– No obvious advantage over Unix/Linux– Tools and OS not as well suited to task as Unix

NT still maintains lead as desktop machine– Unix seems to be catching up fast (gnome, * office)