BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND –...
Transcript of BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND –...
![Page 1: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/1.jpg)
www.beegfs.comFebruary 2016 | Sven Breuner, CEO, ThinkParQ
BeeGFSSolid, fast andmade in Europe
David Ramírez Alvarez
HPC INTEGRATOR MANAGER
WWW.SIE.ES
ADMINTECH 2016
Thanks to Sven for info!!!
![Page 2: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/2.jpg)
February 2016 | Sven Breuner
What is BeeGFS?
Storage Server #1 Storage Server #2 Storage Server #3 Storage Server #4 Storage Server #5 Metadata Server #1
File #1
1 1 2
File #2File #3
2 3 31 2 3 M M M
➔ Simply grow capacity and performance to the level that you need
BeeGFS is…
…designed for performance-critical environments
…a hardware-independent parallel file system
/beegfs/dir1
![Page 3: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/3.jpg)
February 2016 | Sven Breuner
BeeGFS Architecture• Client
• Native Linux module to mount the file system
• Storage Service• Store the (distributed) file contents
• Metadata Service• Maintain striping information for files
• Not involved in data access between file open/close
• Management Service• Glue everything together and watch services
• Graphical Administration and Monitoring System• GUI to perform administrative tasks and monitor system
information• Can be used for “Windows-style installation“
![Page 4: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/4.jpg)
February 2016 | Sven Breuner
Key Aspects
•Performance & Scalability• Initially optimized for HPC• Completely multi-threaded – lightweight design• Supports RDMA/RoCE and TCP (Infiniband, 40/10/1GbE, …)• Distributed file contents:
aggregated throughput of multiple servers• Distributed metadata across multiple servers • High single stream performance (multiple GB/s)
![Page 5: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/5.jpg)
February 2016 | Sven Breuner
Throughput Scalability
![Page 6: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/6.jpg)
February 2016 | Sven Breuner
Metadata Scalability
![Page 7: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/7.jpg)
February 2016 | Sven Breuner
Key Aspects
•Performance & scalability
•Flexibility• Multiple services (any combination) can run
together on the same machine
• Flexible striping per-file / per-directory
• Add servers at runtime
• On demand filesystem „per job“ possible (BeeOND)
• Runs on ancient and modern Linux distros/kernels
• Runs on different Architectures, e.g.• ARM, Xeon Phi, Power, Tilera, …
• NFS & SMB/CIFS re-export possible
![Page 8: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/8.jpg)
February 2016 | Sven Breuner
Key Aspects
•Performance & scalability•Flexibility•Robust & easy to use
• Applications access BeeGFS as a “normal“ (very fast)file system mountpoint
• Applications do not need to implement a special API
• Servers daemons run in user space on top of standard local filesystems (ext4, xfs, zfs, …)
• No client kernel patches, kernel updates are trivially simple • Packages for Redhat, SuSE, Debian and derivatives• Hardware independent (runs on shared-nothing HW)• Graphical monitoring tool
![Page 9: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/9.jpg)
February 2016 | Sven Breuner
Live Throughput Overview
![Page 10: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/10.jpg)
February 2016 | Sven Breuner
Live per-Client and per-User Statistics
![Page 11: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/11.jpg)
February 2016 | Sven Breuner
GUI for Windows-style Installation
![Page 12: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/12.jpg)
February 2016 | Sven Breuner
The easiest way to setup a parallel FS…
# EXAMPLE…$ beeond start –n $NODEFILE –d /local_disk/beeond –c /my_scratch
Starting BeeOND Services…Mounting BeeOND at /my_scratch…Done.
-------------------------------------------------
# GENERAL USAGE…$ beeond start –n <nodefile> -d <storagedir> -c <clientmount>
![Page 13: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/13.jpg)
February 2016 | Sven Breuner
BeeOND – BeeGFS On Demand
•Create a parallel file system instance on-the-fly
•Start/stop with one simple command
•Use cases: cloud computing, test systems, cluster compute nodes, …..
•Can be integrated in cluster batch system (e.g. PBS)
•Common use case: "per-job parallel file system"• Aggregate the performance and capacity of local
SSDs/disks in compute nodes of a job
• Take load from global storage
• Speed up "dirty" I/O patterns
![Page 14: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/14.jpg)
February 2016 | Sven Breuner
Built-in Data Mirroring
• Based on „mirror buddy groups“ of storage targets• Primary/secondary target in a buddy group
internally replicate chunks• But: Targets can still also store non-mirrored chunks• Write operations are forwarded for high throughput• Read possible from both targets
• Internal failover mechanisms• In case primary is unreachable or
fails, an automatic switch is performed• Self-healing (differential rebuild)
when buddy comes back
• Flexible: Can be enabled globally oron a per-directory basis
…
… … … …Tgt 1
MBG 1Tgt 2
MBG 2Tgt 3
MBG 1Tgt 4
MBG 3Tgt 5
MBG 2Tgt 6 Tgt 7
MBG 3Tgt 8
Network
![Page 15: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/15.jpg)
February 2016 | Sven Breuner
Buddy Mirroring per Directory
1
File #2
1
3
5
Target 1
BuddyGroup 1
5 7 n 2 4
Target 2
BuddyGroup 2
6 8 m1
Target 3
BuddyGroup 1
2 3
Target 4
BuddyGroup 2
2 6 3 74 8
1 3 5 7 n 2 4 6 8 m1 2 3
/beegfs/mirrored /beegfs/unmirrored
4 4
File #1File #3
![Page 16: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/16.jpg)
February 2016 | Sven Breuner
Business Model
■ BeeGFS is free to use for end users: www.beegfs.com/download
■ Ready-to-install binaries, compelete source code also available
■ System integrators/partners for turn-key solutions
■ System setup and tuning
■ First point of contact (1st- and 2nd-level support)
■ Partners make back2back contract with ThinkParQ for 3rd-level support
■ Professional 3rd-level support
■ Pricing based on number of servers and timeframe (e.g. 3 or 5 years)
■ Access to enterprise features (Buddy Mirroring, ACLs, quota enforcement)
■ Special customer website area: www.beegfs.com/customerlogin
BeeGFS allows us to easily deliver peta-scale turn-key storage solutions
- transtec
![Page 17: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/17.jpg)
February 2016 | Sven Breuner
What's new in the 2015 Release Series?
•BeeOND (BeeGFS On Demand)
•Trinity:
•Quota Enforcement
•Access Control Lists (ACLs)
•Built-in data mirroring
•Per-User Statistics in Admon GUI
•New manual setup tools (/opt/beegfs/sbin/beegfs-setup…)
•BeeGFS C API
•…
![Page 18: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/18.jpg)
February 2016 | Sven Breuner
Topics for 2016
• Buddy mirroring for metadata• Work in progress, expected Q2/2016
• BeeGFS as a service on Amazon Cloud• Received funding from Amazon• Also in touch with Microsoft for Azure cloud
• Target pools for different hardware (e.g. fast vs big)• Striping with parity across servers
• Tolerate server failures with less capacity overhad compared to mirroring• Configurable on a per-directory basis
• Object interface for HTTP put/get style access• To support applications that were written for such interfaces
![Page 19: BeeGFS made in Europe - HPC ADMINTECH · 2018-10-12 · February 2016 | Sven Breuner BeeOND – BeeGFS On Demand •Create a parallel file system instance on-the-fly •Start/stop](https://reader035.fdocuments.net/reader035/viewer/2022070920/5fb8e9529c05bd0f2a152651/html5/thumbnails/19.jpg)
February 2016 | Sven Breuner
•LIVE TRAINING / DEMO!!!!