LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and...
-
date post
18-Dec-2015 -
Category
Documents
-
view
220 -
download
0
Transcript of LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and...
![Page 1: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/1.jpg)
LUNARC, Lund University LSCS 2002
Transparent access to finite element applications using grid and web technology
J. Lindemann P.A. Wernberg and G. SandbergLUNARC, Lund University
![Page 2: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/2.jpg)
LUNARC, Lund University LSCS 2002
Some earlier work
— R. Larsson, Encapsulation of a finite element program using a distributed object model, Masters dissertation TVSM-5095, Division of Structural Mechanics, Lund University, 1999
— J. Lindemann, O. Dahlblom, G. Sandberg, An Approach For Distribution Of Resources In Structural Analysis Software, ECCM ’99, Munich, Germany, 1999
— Programming and Visualisation Techniques in Finite element Software, Division of Structural Mechanics, Licentiate Dissertation, TVSM-3050, 2001
— Using CORBA middleware in finite element software, ICCS 2002
— Using CORBA middleware in finite element software, Future Generation Computer Systems, Elsevier, 2002, Accepted for publication
![Page 3: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/3.jpg)
LUNARC, Lund University LSCS 2002
Transparent access
• Creating a easier interface to present clusters
• Extend user interface with grid services• Distributed applications integrating
seamlessly with clusters and grids• CORBA interfaces to clusters and grids
![Page 4: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/4.jpg)
LUNARC, Lund University LSCS 2002
Using clusters
• Log in– Instructions on how to use SSH– Getting SSH client software if client is MS Windows– Instructions on unix tools
• mkdir proj• sftp/scp user@...
– Instructions on how to transfer files
• vi/joe submit script– Submit script documentation
• Queue management – qsub script– qstat ...
• Transfer result files back– sftp/scp
For many, this is a straightforward process, but why do we get so many questions??
![Page 5: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/5.jpg)
LUNARC, Lund University LSCS 2002
Cluster Web Portal
• Web interface well established interface– Most users know what to do
• Multiple entrypoints– Novice users (web interface)– Intermediate users (web/cli modification)– Experienced users (web monitoring/cli)
• Project based– One directory per project– Project includes input files and submission scripts
• CWP creates submit scripts from job definition stored in database– templates for different application types
![Page 6: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/6.jpg)
LUNARC, Lund University LSCS 2002
• CWP installed on retired production cluster– Easier to experiment without any users...
• 8 nodes (legs...)– AMD 1100 Mhz– 1 Gb memory– 2x15 Gb striped disks– RedHat Linux 7.2
• Frontend– AMD 1000 Mhz– 256 Mb memory– 80 Gb disk– RedHat Linux 7.2
• Network– 100Mbit Fast Ethernet
The Sleipner Test Cluster
Lunarc clusters
Whenim64 65 nodesHusmodern 32 nodesSleipner 8 nodes
![Page 7: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/7.jpg)
LUNARC, Lund University LSCS 2002
CWP Example
![Page 8: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/8.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 9: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/9.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 10: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/10.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 11: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/11.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 12: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/12.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 13: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/13.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 14: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/14.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 15: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/15.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 16: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/16.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 17: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/17.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 18: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/18.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 19: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/19.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 20: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/20.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 21: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/21.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 22: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/22.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 23: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/23.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 24: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/24.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 25: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/25.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 26: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/26.jpg)
LUNARC, Lund University LSCS 2002
CWP
![Page 27: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/27.jpg)
LUNARC, Lund University LSCS 2002
CWP Implementation
• Apache Web Server SSL• PHP 4.x• OpenPBS• OpenSSH
– Execution of commands
• MySQL database– Job descriptions– User accounts (web accounts)– Announcements/news– Too be extended...
![Page 28: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/28.jpg)
LUNARC, Lund University LSCS 2002
CWP Future
• Jobsubmission through Nordugrid– PHP security functions must be implemented
• Submission of jobs+input files to other clusters without shared filesystems or Nordugrid connection– Packing job and description into a job-
package
– Users must have account on other cluster
• More application templates
• Debugging, documentation and installation scripts
![Page 29: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/29.jpg)
LUNARC, Lund University LSCS 2002
Components in software
• Hardware often consist of components• Software components = hardware
components.• Components in software can be
exchanged.– If the interface is the same– Component model / Object model needed
• CORBA, DCOM, RMI use an intermediate language for describing interfaces– Enabling component design
![Page 30: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/30.jpg)
LUNARC, Lund University LSCS 2002
CORBA
• Object Management Group’s specification for interoperability and interaction between objects and applications.
• CORBA 1.1 released in 1991.
• Language neutral.
• Object oriented.
• Object interfaces described using the Interface Definition Language, IDL.
![Page 31: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/31.jpg)
LUNARC, Lund University LSCS 2002
CORBA concepts and terminology
• client - entity that invokes a request on a CORBA object.
• CORBA object - ”virtual” entity capable of being located by an ORB and having client requests invoked on it.
• server - application in which one or more CORBA objects exist.
• object reference - handle used to identify, locate and address a CORBA object.
• servant - programming language entity that implements one or more CORBA objects.
![Page 32: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/32.jpg)
LUNARC, Lund University LSCS 2002
Interface Definition language
interface Echo {string Shout(in string word);
}
class Echo : virtual public CORBA::Object{
Echo(const Echo&);...
public:...char* Shout(const char* word);
};
IDL-compiler (C++)
![Page 33: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/33.jpg)
LUNARC, Lund University LSCS 2002
Name service
• CORBA objects acessed by object references– Unique identifier, also contains ip/port
information– Must be retrieved somehow– CORBA 2.3 introduced a Name service
• Provides location transparency– bm/servers/factories/forcepad -> object
reference– Name service location provided at startup
• DNS for CORBA objects
![Page 34: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/34.jpg)
LUNARC, Lund University LSCS 2002
Finite element CORBA implementation
![Page 35: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/35.jpg)
LUNARC, Lund University LSCS 2002
Finite element CORBA implementation
User interface
ForcePadSolverserver
FE Solver
Name server
Presentation layer
Middle layer
Implementation layer
User interface
ForcePadSolverserver
FE Solver
Name server
Presentation layer
Middle layer
Implementation layer
User interface
ForcePadSolverserver
FE Solver
Name server
Presentation layer
Middle layer
Implementation layer
![Page 36: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/36.jpg)
LUNARC, Lund University LSCS 2002
ForcePAD server interface
• Designed to reduce the number of requests– Object request has a cost determined by
latency and marshaling rate
• Latency the most critical factor– A CORBA request 50-5000 times slower than
a function call in C++
• Marshaling rate determined by network bandwith
![Page 37: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/37.jpg)
LUNARC, Lund University LSCS 2002
ForcePAD interfaces
![Page 38: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/38.jpg)
LUNARC, Lund University LSCS 2002
Client/server configuration
User interface
ForcePadSolverserver
FE Solver
Name server
Local computer
Presentation layer
Middle layer
Implementation layer
![Page 39: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/39.jpg)
LUNARC, Lund University LSCS 2002
Client/server configuration
ForcePadSolverserver
FE Solver
Name server
Middle layer
Implementation layer
Remote computer
User interface
Local computer
Presentation layer
![Page 40: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/40.jpg)
LUNARC, Lund University LSCS 2002
Client/server configuration
User interface
ForcePadSolverserver
FE Solver
Name server
Local computer
Remote computer 1
Remote computer 2
Presentation layer
Middle layer
Implementation layer
![Page 41: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/41.jpg)
LUNARC, Lund University LSCS 2002
CORBA and FE applications
• For more information – Programming and Visualisation techniques in
Finite Element Software, TVSM-3050– Using CORBA middleware in finite element
software, Future Generation Computer Systems, Elsevier, 2002
![Page 42: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/42.jpg)
LUNARC, Lund University LSCS 2002
CORBA in clusters and grids
• CORBA enabled applications requires a CORBA aware cluster
• Globus toolkit has CORBA bindings for job submission and resource management – Bindings not very well supported/developed– How do we run CORBA enabled finite
element applications using Globus?
• Initial development of a CORBA enabled cluster has been done– Sleipner will testbed for these efforts
![Page 43: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/43.jpg)
LUNARC, Lund University LSCS 2002
CORBA enabled cluster
• Interface definitions needed (IDL) for– Cluster information system– Resource management– File transfer– Queue management– Job submission– Security
• Each cluster becomes a set of CORBA objects
• Simple grid solutions can be created by connecting CORBA enabled clusters
• Communication over SSL
![Page 44: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/44.jpg)
LUNARC, Lund University LSCS 2002
CORBA cluster
Client
ControllerSecurityManager GridNode
FileTransfer NodeInformation CommandInterface
1 n11
ServantController
ClusterNode1 n
Queue system
![Page 45: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/45.jpg)
LUNARC, Lund University LSCS 2002
Web based engineering tools
![Page 46: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/46.jpg)
LUNARC, Lund University LSCS 2002
Web based engineering tools
![Page 47: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/47.jpg)
LUNARC, Lund University LSCS 2002
Web based engineering tools
![Page 48: LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.](https://reader035.fdocuments.net/reader035/viewer/2022062313/56649d225503460f949f8914/html5/thumbnails/48.jpg)
LUNARC, Lund University LSCS 2002
Conclusions
• Broader user base requires– Simple user interface– Smooth learning curve
• Experienced users benefit a better interface– Monitoring– Simple control
• Web portal can be central point for grid based resources
• CORBA based distributed finite element applications can be supported on clusters– CORBA based cluster abstraction– Globus CORBA interface
• CORBA based applications enable powerful web based engineering tools