Post on 03-Jan-2016
description
1
IDGFIDGF
IInternationalnternational DDesktopesktop GGridrid FFederationederation
PORTING APPLICATION TO THE PORTING APPLICATION TO THE BOINC DESKTOP GRID: ISA RAS BOINC DESKTOP GRID: ISA RAS
EXPERIENCEEXPERIENCE
Mikhail PosypkinMikhail Posypkin
Oleg ZaikinOleg Zaikin
Nikolay KhrapovNikolay Khrapov
DEGISCO is supported by the FP7 Capacities Programme under contract nr RI-261561.DEGISCO is supported by the FP7 Capacities Programme under contract nr RI-261561.
2DEGISCO WP4 20/04/23 2
TALK OUTLINETALK OUTLINE
Introduction to the Russian Chapter Our BOINC and combined infrastructures OPTIMA@home project: using GenWrapper SAT@home project: using DC-API Performance Assessment Lessons learnt
3DEGISCO WP4 20/04/23 3
Who we areWho we are
Institute for Systems Analysis of RAS (ISA RAS) www.isa.ru (founded in 1976) Centre for Grid Technologies and Distributed Computing:
Desktop and Service Grids Service-oriented Science Parallel algorithms Numerical methods in optimization
Partner in the DEGISCO project Founding member of IDGF
4DEGISCO WP4 20/04/23 4
The Goals of Russian ChapterThe Goals of Russian Chapter
Help Russian research and industry teams to develop, deploy and maintain their desktop-grid distributed applications
Maintain DCI infrastructures for our projects and for other organizations
Provide courses and tutorials on desktop grids and volunteer computing for Russian academic and industrial community
Promote Russian BOINC projects Explain the mission of IDGF and benefits of the
membership
5DEGISCO WP4 20/04/23 5
Our Desktop Grid InfrastructureOur Desktop Grid Infrastructure
BOINC Educational Infrastructureboinc-edu.isa.ru
BOINC Test Infrastructureboinc-test.isa.ru
BOINC Production Infrastructureboinc.isa.ru,sat.isa.ru
Based on SZTAKI Desktop Grid Package
6DEGISCO WP4 20/04/23 6
The Typical PassThe Typical Pass
Netmax OPTIMA@homeSAT@home
Lots of (student) projects
7DEGISCO WP4 20/04/23 7
Production BOINC InfrastructureProduction BOINC Infrastructure
Currently we have two projects running in production: OPTIMA@home (0.5 Tflops real performance)and SAT@home (1.3 Tflops real performance)
Two ways of application porting: – GenWrapper (OPTIMA@home)– DC-API (SAT@home)
= 3 x 48-core servers
8DEGISCO WP4 20/04/23 8
Service Grid InfrastructureService Grid Infrastructure
4 nodes: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 4 GB RAM,640 GB Disk space
RU-ISA-CGTDC : grid.isa.ru
9DEGISCO WP4 20/04/23 9
Combined InfrastructureCombined Infrastructure
EGI Desktop Grid VO
OPTIMA@HOME
11
OPTIMA@homeOPTIMA@home
•OPTIMA@HOME is a research project that uses Internet-connected computers to solve challenging large-scale optimization problems. The goal of optimization is to find a minimum (or maximum) for a given function.
•Currently we use Monotonic Basin Hopping Method implemented in the BNB-Solver framework
12
Monotonic Basin-Hopping MethodMonotonic Basin-Hopping Method
Maximal No Improve (MNI)
MNI controls the run-time
“seed”- starting point
13
Distributed Version of Basin-HoppingDistributed Version of Basin-Hopping
Generate Seeds
ComputeResults
Collect and ProcessResults (find minimum)
14
ImplementationImplementation
BOINC CoreClient
GenWrapper
BNB-Solver
BOINC Server
Server Workstation
Client Workstation
15DEGISCO WP4 20/04/23 15
GenWrapper- A generic BOINC GenWrapper- A generic BOINC wrapper for legacy applicationswrapper for legacy applications
Free tool developed by SZTAKIhttp://genwrapper.sourceforge.net/
GenWrapper is a generic BOINC wrapper for legacy applications utilizing GitBox (a variant of BusyBox): Use POSIX like shell scripting and built-in commands like tar, awk, sed, zip, etc. to control and execute your legacy Desktop Grid application.
16
GenWrapper script for GenWrapper script for OPTIMA@home (on the client side)OPTIMA@home (on the client side)
IN=`boinc resolve_filename in`
OUT=`boinc resolve_filename out`
BNBAPP=`boinc resolve_filename bnbss`
./"${BNBAPP}" "${IN}" "${OUT}"
17DEGISCO WP4 20/04/23 17
On the server sideOn the server side
Generate and store input files Use create_work to submit workunits Use sample_assimilator (shipped with BOINC)
to collect results in a folder Use scripts to process results
18
More complex scenarioMore complex scenario
Generate Starting Points
ComputeResults
CollectResults and generate new workunits
19
More elaborate versionMore elaborate version
We use a perl script that • runs as a periodic task• processes results , generates and submits new workunits• uses found local minima as starting points for new workunits
Adding periodic task (in config.xml):<task> <cmd> ma.pl </cmd> <period> 30 minutes </period> <output> ma.out </output> </task>
20DEGISCO WP4 20/04/23 20
Resume: GenWrapperResume: GenWrapper
GenWrapper is perfect for porting “legacy” applications
You can organize complex workflows with GenWrapper using BOINC periodic tasks
If you need more control over your application use DC-API
22
SAT@home projectSAT@home project
SAT@home is a research project that uses volunteer computing to solve hard and practically important problems that can be effectively reduced to Boolean satisfiability problem.
Applicability• discrete functions inversion problems• cryptography• discrete optimization • bioinformatics
Problems that being solved currently:Problems of inversion of some cryptographic functions used in stream ciphers are being solved.
23
The SAT@home approachThe SAT@home approach
Desktop grid decisionSAT
problem
CNF in DIMACS format
decompose
solve Satisfying assignment or UNSAT
CNF encoding of the original problem is decomposed into a family of CNFs which are "simpler" for SAT solvers. For finding "good" parameters of decomposition we use special technique of predictive functions which take into account peculiarities of the original problems.
Originalproblem
24
SAT@home ImplementationSAT@home Implementation
Project was implemented using BOINC platform and DC-API library (http://www.desktopgrid.hu).Server part of the application terminates the experiment as soon as the 1st satisfying assignment foundClient part of the application is based on publically available SAT solvers minisat 1.14.1 and minisat 2.0 (http://minisat.se) modified to take into account peculiarities of an original problem and its CNF encodingControl points
25DEGISCO WP4 20/04/23 25
DC-APIDC-API
DC-API applications consist of two major components: a master application and one or more client applications.
The master application usually runs as a daemon, but it is also possible to write a master that runs periodically. Master generates workunits, collects and processes results.
Client applications are simple sequential programs that take their input from the master, perform some computation on it and produce some output.
Free: http://www.desktopgrid.hu
26DEGISCO WP4 20/04/23 26
Resume: DC-APIResume: DC-API
Flexible C-based Gives comprehensive control over your
application Much easier than BOINC API
27
GenWrapper vs DC-API: when to use GenWrapper vs DC-API: when to use what?what?
Gen Wrapper• You have a big complex
application • You have a non-C (C ++)
application• You don’t need a fine
control over your distributed application
DC-API• You write application from
scratch• Your application is written in
C/C++• You need complex
distributed scenario and fine control over your application
• You target several DG platforms: OurGrid, Condor, etc.
28
Performance evaluation: SZTAKI toolsPerformance evaluation: SZTAKI tools
29
Performance evaluation: ISA RAS toolsPerformance evaluation: ISA RAS tools
30
Performance evaluation: ISA RAS toolsPerformance evaluation: ISA RAS tools
Tuesday, November 8, 2011Session S8 16:15 Assessing the Performance of Desktop Grid Applications, A. Afanasiev, N. Khrapov, and M. Posypkin
31DEGISCO WP4 20/04/23 31
Lessons LearntLessons Learnt
Choose proper technology (GenWrapper or DC-API)
Invest to advertisement (forums, statistics etc.)
Evaluate and analyze your performance (use tools)
Be careful with your donors – always respond their questions and don’t ignore their complaints
32
http://desktopgridfederation.org/http://desktopgridfederation.org/
Globe adapted from http://upload.wikimedia.org/wikipedia/commons/f/fa/Globe.svg
Globe adapted from http://upload.wikimedia.org/wikipedia/commons/f/fa/Globe.svg
IDGFIDGF
IInternationalnternational DDesktopesktop GGridrid FFederationederation