BDE Platform ArchitectureSC 4 Webinar27 June 2016

AgendaPlatform architecture

User interfaces

Platform architecture

Platform architecture

DEPRECATED

Platform architecture

Platform architecture

Platform installationManual installation guide

Using Docker MachineOn local machine (VirtualBox)In the cloud (AWS, DigitalOcean, Azure)Bare metal

Platform componentsBase Docker images:

Serves as a template for a (Big Data) technology

Easily extendable with custom algorithm/data

Automating as much as possible

May be opinionated

Platform componentsPublished componentsImage repositories on GitHubAutomated builds on Docker HubDocumentation on BDE Wikidocker-compose snippet in README

Developing a Pipeline

Big Data pipelinesRun pipeline: $ docker-compose up

Orchestrator required for initialization processComponents may depend on each otherComponents may require manual intervention

Platform architecture

User Interfaces

User interfacesWorkflow UIsWorkflow BuilderBuild a workflow to configure the init daemon serviceWorkflow MonitorMonitor a pipeline's initialization process

Swarm UIDeploy a pipeline on the platform

Integration of existing UIsBDE CSS based on Materialize CSSIntegrator UI: present GUIs in a unified interface

UI to follow-up status of the pipelineStatus of each componentManually finish a step in the pipeline

Integrated as component inthe application's docker-compose.yml

Pipeline monitor application

Thank you