#OSSPARIS17 - CHOReVOLUTION An IDRE for IoT-enabled applications, by SEBASTIEN KELLER, Thales
#OSSPARIS17 - PrestoCloud Project: Big Data Orchestration for Batch+Stream in Clouds&Fogs with...
-
Upload
paris-open-source-summit -
Category
Technology
-
view
72 -
download
1
Transcript of #OSSPARIS17 - PrestoCloud Project: Big Data Orchestration for Batch+Stream in Clouds&Fogs with...
Paris, Sophia Antipolis, London, San Jose USA
PrestoCloud: Big Data Orchestration for
Batch+Stream in Clouds & Fogs with ActiveEon
Denis Caromel, Vincent Kherbache, Sophie Song, Pascal Urso
Global Locations
Partnerships
Key information
Management
Denis
Caromel, CEO
François
Tournesac, CSO
Brian Amedro,
CTO
Founded in 2007 by Denis Caromel in Sophia-Antipolis, Spin-off of INRIA
Addressing $80 Billion Hybrid Cloud Market with 27% CAGR
Disruptive Patented Technology w/ Exceptional Business Outcomes
60% of the revenue from international
Sophia-Antipolis (France)
Paris (France)
London (United Kingdom)
San-Jose (United States)
Montreal (Canada)
Fribourg (Switzerland)
Dakar (Senegal)
ProActive Solution
Job Scheduling, Workload Automation
Orchestration & Meta-Scheduling
On-premises and on all clouds
Open Source
Core Product
Processing & Automation Workflows
Scheduling & Orchestration
Resource
Management
& Monitoring
Workflow
Studio
Job
Console
Resource
Manager
Big Compute for Everyone
</>
Remote
Visualization
Integrated Web
PortalsAPI
Scheduler & Meta-Scheduler
Resource Manager
Clusters
LSF
Application-specific
Interfaces
ProActive Workflows
Fault Tolerance
Cloud bursting
Resource agnostic
Micro-service
PrEstoCloud
Proactive Cloud Resources Management At The Edge For Efficient Real-time Big Data Processing
PrEstoCloud Factsheet
Partners
• 36 months • Starts on January 2017
• Ends on December 2019
• ActiveEon: • Core tech Provider
• 60 PM
• 501K€
• Provides Cloud Automation, Workflows and Scheduling for Real-Time Big Data Processing
PrEstoCloudPrEstoCloud Platform
Cloud-Edge Communication Layer
Cloud Application
Communication & Message Broker
Devices Layer
ApplicationDeveloper
Control Layer
Code-level Annotation
s
Application Requirements
Meta-Management Layer
Semantic Model
DevOps placement requirements
scalability requirements
Events & Messages
Mesh Network-aware application
Spatio- Temporal Processing
Library
Resources Adaptation
Recommender
Application Fragmentation &
Deployment Recommender
Situation Detection
Mechanism
Workload predictor
Mobile Context Analyzer
predictions
Mesh Network-aware application
Spatio- Temporal Processing
Library
Security Enforcement Mechanism
Cloud Infrastructure Layer
Events & Messages
Application Placement & Scheduling
Controller
Autonomic ResourceManager
Autonomic Data Intensive Application
Manager
DevOps
Cloud Deployment and Management
Edge Deployment and Management
Monitoring Probes
On/Offloading Agents
Monitoring Probes
On/Offloading Agents
Monitoring Probes
Inter-site Network Virtualisation
Mobile On/Offload Processing
Cloud and Edge
Resources Repository
Events & Messages
Real Time Data Streams
Edge application
Monitoring Probes
On/Offloading Agents
Events & Messages
Mesh Network-aware application
Spatio- Temporal Processing
Library
Monitoring Probes
On/Offloading Agents
Real Time Data Streams
Monitoring Probes Installer
8
Resources Management
Extended TOSCA specification
• On public & private clouds: Virtual machines, Containers
• On edge devices: Containers, Mobile app
Inter-site, multi-layer network virtualization
• Route-based VPN
• Security enforcement mechanism (RBAC, NFV)
Real-time big data processing
Resource provisioning
• Scheduling and orchestration of (de)commissioning workflows
• Real-time events for the monitoring of Cloud/Edge resources and applications
Message broker (AMQP, MQTT, etc.)
• Decision-making in the resources placement
Recommendation system (Constraints solver)
Data-intensive applications management
• Predictive Workload management
• Scheduling and orchestration of deployment workflows
(Re)configuration
Kafka Storm Visdom Streaming
Real-Time Data Streaming using ActiveEonWorkflows:
• Apache Kafka (Data streaming)
• Apache Storm (Real-time stream processing)
• Visdom (Stream Visualization from Facebook)
https://www.youtube.com/watch?v=i6ZOyoXSq-A&t=5s
Azure PoC in the Box
Azure Node Source InfrastructureAzure Node Source Infrastructure
Scale automaticallyLeverage Azure services
Existing Resources
Local & Network Resources Private Cloud, HPC & Others
Resource
Manager
Workflow
Scheduler
</>
Azure Node Source Infrastructure
LSF
Using Azure Scale Sets
Deploy over
150 k nodes
Paris, Sophia Antipolis, London, San Jose USA @activeeon
+33 988 777 660
Automate Accelerate & Scale
10K Nodes, 20K Tasks, 1M Jobs