GRAPPA

• Part of Active Notebook Science Portal project

• A “notebook” like GRAPPA consists of – Set of ordinary web pages, viewable from any browser– Editable execution scripts in Python/Jpython

• Provides direct access to COG, GDK, VDT tools• Flexible, powerful language with bindings to other languages• Notebook can be run without browser• Satisfies O-O fundamentalists, but usable by real scientists

– Input forms that feed parameters to scripts

• Notebooks use MyProxy server for credentials, Tomcat• ANSP notebooks can be run in “personal” mode

GRAPPA

• Notebook “sessions” are live XML documents– Can be revisited, rerun later– Can be copied/modified for new run– Archived and shared with a collaborator (including live object

references, allowing another user remote method invocation)

• Notebook goals are to provide– Scripting, launching of jobs via GRAM, Condor, Condor-G, ssh, exec– Interface to information subsystems– Ability for end user to attach arbitrary data to a session: graphs, notes,

links, ....

GRAPPA

• Application components run by ANSP are standalone executables– Can read/write files– Send event messages (event in CS-talk, not HEP-talk)– Provide DoE Common Component Architecture “port interfaces”– Peers that can keep running when other parts of distributed comp fail ...

Component App Manager Model

• Application may be untouchable (no source code, etc)

• Idea is to make it appear as a fully-enabled component• Create a proxy that manages app and framework comm.

ComponentApp Manager

Fortran Application(a.out)

GRAPPAOutFile1InputFilesInputFilesInputFiles

exec

Start running

Component App Manager Model

• App manager is responsible for – making sure all input files are ready before running– event notifications to GRAPPA and other interested parties– publishing file locations, and moving files via a file manager

component

• When output file of one component is needed as input file of another, receiver is responsible for file move.

• App manager can use user's CA to use VDT tools for file transfers, app execution.

• App manager can (via resource manager) actually run app on a different machine

• Communication about progress, status is primarily via publish/subscribe event messages

File Management

• File Manager: component that provides low-level, simple metadata about files– Uses JDBC interface to relational DB– Starts running with a handle to a DB server (may be a “personal mode”

DB)– Multiple file managers, with handles to different DB's, can be running

in a single application.– File managers use RMI for interchange of data– Likely to be replaced by VDT tools as they come available

File Management

• Application developer provides a description of each file an application component reads or writes:

<filename>matstruct.gif</filename> Filename the app “opens”

<direction>output</direction> Input, output, both

<termination>total</termination> Can be streamed or not

<format>binary</format> ASCII, binary, or other

<mimetype>image/gif</mimetype> Optional; provide if known

<description>This is an image of the sparsity structure of the matrix

being analyzed; it is part of the overall matstruct.html file

</description>

File Management

• Notebook developer provides additional information for each file, things which are outside the scope of individual application– Whether file is to be locally archived, remotely archived, or is volatile– Whether file should be cleaned up after/between runs– What kind of compression should be used (if any)– Naming convention for archived files

• basename.machine.timestamp.suffix– Location to archive files (machine, directory)

• Notebook must provide user with easy, coherent picture of the files

• Notebook must also provide for additional files: user notes, etc.

GRAPPA: Overall Picture

Users interact viaWeb browser orscripting

Authorization ishidden in serverin this picture

GRAPPA: Overall Picture

Server accesses anotebook DB foruser's sessions

GRAPPA: Overall Picture

User's script is partof stored session

Components are started through ajob launch tool thatconsults with resource manager

Request might be accompanied by requirements

GRAPPA: Overall Picture

Executables rununder applicationmanager Pythonscripts

GRAPPA: Overall Picture

Application managers publishevent messagesto file managerabout file events

GRAPPA: Overall Picture

Application mgr,Grid resource toolspublish events toevent channels

Filtering eventchannels allow notebook to selectevents to record,present

GRAPPA/GriPhyN Generalities

• Not intended as competitor with VDT, but a user-level interface to VDT tools

• Standalone mode is critical– Federated model of information/data management is highly desirable– Handles some security/privacy issues

• Event message channel model provides some robustness– Components run more as peers– Components can continue when framework crashes