Distributed Computing with DAFFIE Glenn Bresnahan Boston University Scientific Computing and...

Distributed Computing with Distributed Computing with DAFFIEDAFFIE

Glenn Bresnahan

Boston University

Scientific Computing and Visualization

(http://scv.bu.edu/DAFFIE/Earlab-Oct02)

Scientific Computing and Visualization 2

What is DAFFIE?What is DAFFIE?

Distributed Applications Framework For Immersive Environments

System for easily creating distributed and shared virtual environments

Authored by Glenn Bresnahan, Erik Brisson, Robert Putnam


DAFFIE Messaging SystemDAFFIE Messaging System

Application protocol

Client servicesClient application library (API)

Message router (“event server”)


DAFFIE Message routingDAFFIE Message routing

client

event event serverserver

client

clientproxyproxy

clientproxyproxy

client

client

client

client

client


Application ArchitectureApplication Architecture

App threadApp thread App threadApp thread …

Services(timing, streaming, file sharing)

Message Transport

Event Messaging


API - ControlAPI - Control

id = event_join(server, &nc); rc = event_leave();

id = event_id(); nc = event_nclient(); millisec = event_clock(); event_sleep(millisec);

event_register(class, species, instance);


Event headers informationEvent headers information

EventType(event);EventSize(event);EventFrom(event);EventTo(event);EventSeq(event);


System Events – Join & LeaveSystem Events – Join & Leave

typedef struct { EVENT_HEAD head; int id; int nmember;} EVENT_JOIN;

typedef struct { EVENT_HEAD head; int id; short detail; short exitcode;} EVENT_LEAVE


System Events – IdentificationSystem Events – Identification

typedef struct { EVENT_HEAD head; int tag;} EVENT_IDENTITY;

typedef struct { EVENT_HEAD head;

char host[MAXIDSTR];char class[MAXIDSTR];char species[MAXIDSTR];char instance[MAXIDSTR];int tag;

} EVENT_IDENTIFICATION;


API – SendAPI – Send

rc = event_send(to, type, &event, size);– “to” is a client id or one of the following:

• EVENT_BCAST_ALL• EVENT_BCAST_NOTME• EVENT_BCAST_SERVER• EVENT_BCAST_LOCAL

– Macros• SEND(to, type, event);• BCAST(type, event);

event_drain();


API - ReceiveAPI - Receive

rc = event_receive(&event);

event_receive_type(n, type1, …); event_select(on_off, et_low, et_high);

event_wait(); event_wait_timeout(millisec);


API – Receive queueAPI – Receive queue

event_scan_type(n, type1, …); rc = event_scan(&event); n = event_queued();

event_flush(flag); event_flush_sync(); event_receive_enable(on_off);


API – Active messagesAPI – Active messages

event_callback(type, func);

event_add_callback(type, func);event_delete_calllback(type, func);event_protect_type(on_off, et_low,

et_high);


Client ServicesClient Services

Streaming– Data chunking – Bandwidth management– Dynamic tuning

File sharing– File servers– Conditional upload/download


Job Initiation & MonitoringJob Initiation & Monitoring

Rely on external authentication and job initiation (e.g. ssh, Globus)– Launch scripts (ad hoc)

• cgl-video; visible.bu.edu:vgrab -compress -process; -:vswirl -steps 9999999 -np 4; -:vbroadcast; .:vdirect -object poolanim_grp ;

– Agent initiation daemon– Logging daemon


Message TransportsMessage Transports

TCP/IP– Single channel or striped

Shared memoryMyrinetIP Multicast


Defining Upper Layer ProtocolDefining Upper Layer Protocol

Components– Event type ID (#define in a .h)– Data layout (compile-time) (C struct in a .h) – Data layout (run-time) (event_define)• Needed for canonical byte ordering

Types statically (compile time) defined

Protocol compiler


Message RoutingMessage Routing

Message router (“event server”)– Multiple delivery service classes

• Globally sequenced, reliable delivery

– Client management– Message subscription– Multiprocessor, high bandwidth

Hierarchical servers Proxy routers

– Fan-in/fan-out Tunneling

– Direct routing for P2P


Messaging APIMessaging API

Multithreaded, non-blocking Dynamic join and leave Point-to-point and broadcast messages Polled or callback (active messages) Subscription based receive Low level services– Registration, timing, synchronization, query


Service DiscoveryService Discovery

Client registration/identification– Class (Viewer, Audio, Video, Agent)– Species (video: source, broadcaster, sink,

processor)– Instance (video: stream-name)– ID– Hostname

Client location services Service request/acknowledgement


Client MonitoringClient Monitoring

Dynamic clients– JOIN, EXIT, IDENTIFICATION events

Event server monitoring– Heart beat– Latency monitor– Congestion control– Statistic generation

Client monitoring– Process level monitoring– Congestion management– Remote process termination


Leaky Example: HeadersLeaky Example: Headers

typedef struct {EVENT_HEAD head;char filename[200];

} EVENT_GETPARAM;

typedef struct {EVENT_HEAD head;int logger, frame_size, sample_rate);

} EVENT_START;

typedef struct {EVENT_HEAD head;char msg[1000];

} EVENT_LOGINFO;



typedef struct {

EVENT_HEAD head;

int client;

int size;

float pressure[1];

} EVENT_ADVANCE;



#define ET_START 1001#define ET_GETPARAM 1002#define ET_LOGINFO 1003#define ET_ADVANCE 1004#define ET_STOP 1005

event_define(ET_START, “start”, 1, 3, EVENT_DATA_INT);event_define(ET_GETPARAM, “getparam”, 1, 200, EVENT_DATA_CHAR);event_define(ET_ADVANCE, “advance”, 3,

1, EVENT_DATA_INT,1, EVENT_DATA_INT,0, EVENT_DATA_FLOAT);


Leaky Example: InitializationLeaky Example: Initialization

id = event_join(“earlab.bu.edu”, &nc);printf(“Joined as client id %d of %d\n”, id, nc);

event_register(“integrator”, “leaky”, “1.0”);

event_receive_type(ET_GETPARAM, ET_START, ET_ADVANCE, ET_STOP);

event_callback(ET_START, do_start);event_callback(ET_GETPARAM, do_getparam);…


Leaky Example: File ServiceLeaky Example: File Service

GotParamFile = 0;

efs_init();

efs_chdir_receive(“/tmp/earlab”);


Leaky Example: GetParametersLeaky Example: GetParameters

do_getparams(EVENT_GETPARM *egp) {

efs_download_cond(egp->filename,

egp->filename, xfer_done, 0);

return(1);

}

xfer_done(int reqid, int fsid, inr rc, char *file, int tag) {

char path[1000];

strcpy(path, “/tmp/earlab/”); strcat(path, file);

GotParamFile = 1;

Leaky_GetParameters(path);

);


Leaky Example: StartLeaky Example: Start

do_start(EVENT_START *est) {

Logger = est->logger;

while (!GotParamFile) event_sleep(100);

Leaky_Start(egp->frame_size, egp->sample_rate);

return(1);

}


Leaky Example: LogInfoLeaky Example: LogInfo

void LogInfo(char *msg) {

EVENT_LOGINFO einfo;

strcpy(einfo->msg, msg);

event_send(Logger, ET_LOGINFO, &einfo, sizeof(einfo));

}


Leaky Example: AdvanceLeaky Example: Advance

do_advance(EVENT_ADVANCE *ein) {int bytes;EVENT_ADVANCE *eout;

bytes = sizeof(EVENT_ADVANCE) + (ein->size-1)*sizeof(float);

eout = (EVENT_ADVANCE *) malloc(bytes);

Leaky_Advance(&ein->pressure, &eout->pressure);eout->client = event_clientid();eout->size = ein->size;event_send(ein->client, ET_ADVANCE, eout, bytes);

free(ein);return(1);

}


DAFFIE Message routingDAFFIE Message routing

client

event event serverserver

client

clientproxyproxy

clientproxyproxy

client

client

client

client

client

Distributed Computing with DAFFIE Glenn Bresnahan Boston University Scientific Computing and...

Documents

Transcript of Distributed Computing with DAFFIE Glenn Bresnahan Boston University Scientific Computing and...