(ATS3-DEV06) Web Development with Accelrys Enterprise Platform
(ATS3-GS03) Accelrys Enterprise Platform Deeper Dive
-
Upload
accelrys -
Category
Technology
-
view
1.191 -
download
1
description
Transcript of (ATS3-GS03) Accelrys Enterprise Platform Deeper Dive
(ATS3-GS03) Accelrys Enterprise Platform Architecture Deeper Dive
Jason BenedictSr. Architect, Platform R&[email protected]
The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.
• Core services and security• Job launching• Process management• Latency and scalability data• Clustering methods
Agenda
LIMS
IsentrisNotebookVaultDocsORACLE
…
ExperimentWorkflow
InstrumentInterfaces
ReportsWorkRequest
SchedulingExperiment Design
RegistrationBiologyModeling & Simulation
Virtual Chemistry
Data Mining & Analytics
Imaging
3rd Party Applications
Accelrys Enterprise PlatformWeb Applications Thick Client Applications
SharePoint & Office Applications
Web Application Framework Client Integration APIs MS Office Integration
OtherLEA
SOA Integration
Scientific and Generic Services Data Management Services
Accelrys Enterprise Platform
Accelrys Enterprise Platform Integration
DBs
Pipeline Pilot Enterprise Server
Protocol Runtime Environment (scisvr)VB Script
(On Client)
Run Program Java Perl Python
VB Script (On
Server)
Admin Portal Help PortalWeb Apps Web Services API
.NET SOAP & HTTP
Telnet / FTP SSH / SCP ODBC /
JDBC
VB Script Cmd Line Java Classes
Perl Scripts
.NET Classes
RESTService
SOAP Service Cmd Line
Server IntegrationExtend pipelines with new components that integrate your code, data and services.
Professional ClientRun Protocol
Command Line Client
Web Browser
Web Port
JavaScript Client SDK
JavaScript Client
.NET Client SDK
.NET Client
Java Client SDK
Java Client
Web Services API
SOAP Client
Client IntegrationBuild clients that connect to Pipeline Pilot and run protocol services.
REST API
Grid System Integration (optional)
File System
Pipeline Server Architecture
Apache HTTP Server
Authentication and Authorization Security Module
XMLDB Service
Locator Service
RunnerService
LoggingService
File AccessService
Protocol Web
Services
Admin Portal
Help Portal
Data Flow Services
XMLDB User Data Job Data
Corporate Directory
CMS
SOA
DB’s
runjob CGI
WSDL CGI
Mod_balancer
Mod_proxy
Apache Tomcat
Query Service
Scheduler
1 .. N1 .. 1
Launching Asynchronous (polling) Jobs
Apache HTTP ServerAuthentication and Authorization Security Module
Runner Service
Need to fork scisvr?
scisvr(.exe) JVMCLR
Job Folder
2.1.
Create job directory with compressed protocol.xml and uploaded input files
3. 4.
Poll job status via sts file
Monitor Job existence via lck file and process status
5.
Read result file from disk and return to client through Apache
Write sts file and results files to job directory
Write lck file to lck directory
Apache HTTP Server
XMLDB
Launching Synchronous (blocking) Jobs
Authentication and Authorization Security Module
Runner Service
Need to fork scisvr?
scisvr(.exe) JVMCLR
2.1.
Get protocol XML from Protocol DB
3.
Connect to scisvr pipe. Send protocol XML and request parameters
4.
Send notification to apache via pipe when done
5.
Stream results back to Apache on pipe
XMLDB XMLDB
Write lck file to lck directory
Job Settings
• Set Max Running Jobs to 2x available cores
• Set Blocking Job timeout between 10-30 seconds, not more due to client starvation
• Maximum Number of Parallel Processing is a guideline, not a strict maximum. Set to 2x cores
• Set Maximum Job Daemons Per Pool to 2x available cores
• Job Readiness Refresh Rate assists with multipurpose servers which can become “cold”
• Read application specific recommendations for more details
– Identified by __poolid=<name> parameter on request.– Needs to be sent from the
client, not from the saved protocol
– Latency of 20-200 ms– Creates a pool of scisvr.exe
processes dedicated to that pool
– Enables caching of expensive resources:• JVM• CLR• Database connections• Protocol DB Shortcuts
and References
Process Management - Pools
– Impersonation create a small pool for each user for each pool
– Lower the pool sizes to accommodate this behaviors
Process Management – Pools w/ Impersonation
Scisvr Pool Settings – Config filesSetting Default Description
Start Servers 0 Number of initial processes in this pool, created when apache starts
Min Spare Servers 1 Min number of idle processes to keep alive
Max Spare Servers 1 Max number of “available”processes to keep alive
Max Spare Servers Trim Time 0 Time to wait (seconds) before pruning “Available” servers exceeding Max Spare Servers value
Max Servers 16 The total number of servers to allow for this pool
Max Queue Depth 32 maximum number of jobs to queue before rejected. Can be 0 or -1 for infinite
Max Requests Per Server -1 Maximum number of requests to handle in a single server before exiting, -1 is infinite
Time to Live 300 Idle timeout (seconds) for pooled server to live
Warm-up Protocol Path to initial protocol to run
Memory Threshold 80 Max % phys mem use by all proc’s before pruning
Individual Usage Threshold 15 Max % phys mem use by one proc before pruning
Web Job Launch Scalability Improvements
Framework overhead on blocking, pooled jobs on 8 core Windows 2008 R2 (64 bit)
Web Job Launch Scalability Improvements LinuxFor simple chemistry fetch of 10 records to JSON on 8 core RedHat Linux ES5 (64 bit)
Identical tests on Windows 2008 RS on identical hardware
Performance Tuning Document
• Guide available on Accelrys Forums– http://doc.accelrys.com/library/PipelinePilot/doc/performance
_tuning.pdf
Public Cluster
16
Clients
Primary Pipeline Pilot Server
SecondaryPipelinePilot Servers
Login
Execute
NFS
Users
Users
Private Cluster
17Clients
Primary Pipeline Pilot Server
SecondaryPipelinePilot Servers
Login
NFS
Execute
Users
Users
Users
Grid (SGE, PBS, LSF, other)
18Clients
Primary Pipeline Pilot serverandgrid submission server
Grid Nodes: do not require Apache HTTPD
Login
NFS
Execute
Users
Users
Users
Grid software and SOAP
IP-based Load Balancing 1
19
Clients
Reverse Proxyor
IP-based Load Balancer
Login
Execute
Users
Users
Symmetrical Pipeline Pilot Server Nodes Shared Storage
XMLDB
Job FoldersUser Folders
File share
• What we learned– Apache service and launching system– Job launching and settings– Process management for pooling– How pooling has improved latency (snappiness)– Clustering and grids
Summary
The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.
For more information on the Accelrys Tech Summits and other IT & Developer information, please visit:https://community.accelrys.com/groups/it-dev