BBC's GraphDB (formerly Owlim) AWS Cloud Migration
33
Moving OWLIM to Amazon Web Services
-
Upload
logomachy -
Category
Technology
-
view
159 -
download
0
description
BBC's presentation for the GraphDB user group
Transcript of BBC's GraphDB (formerly Owlim) AWS Cloud Migration
Moving OWLIMto Amazon Web Services
Moving OWLIMto Amazon Web Services
GraphDB
Introduction to LDP
● Stands for Linked Data Platform● Building APIs and Tools for semantic
publishing at the BBC● Team of 8-12 people (5-8 developers) ● Based in London (moving to Salford)
Introduction to LDP
● Tagging and surfacing content
Why move to AWS?
● Started with taking ownership of Owlim● BBC shift to move to AWS● Loss of key people● Freedom with responsibility
Current Architecture
Cloud Architecture
Keeping in sync
OpsWorks
● Part of AWS● Easily deploy things to EC2
OpsWorks
● Define infrastructure in a stack● The stack is comprised of layers● Each layer has a role (eg. master, slave)● Layers contain a number of instances
OpsWorks – Advantages
● Conceptually, fairly simple.● Entire stack defined as JSON.
○ Can be (and is) version controlled.
● Very easy to clone an entire stack.○ We use this to spin up ‘experimental’ environments.○ Next step: blue/green deployments?
OpsWorks – Disadvantages
● It can be a little slow.● Chef support can be tricky to debug.
● “Infrastructure as code”● Cookbook, made up of recipes.
○ Each recipe installs/configures a specific thing.
● Each layer has a set of recipes that define it.
Chef
include_recipe 'tomcat::default'
cookbook_file 'owlim-production.license' do path File.join(node[:owlim][:basedir], 'owlim.license') mode 0644 owner 'tomcat' group 'tomcat'end
execute 'create_default_cluster_repository' do command SystemRepository.create_repo 'default-cluster' not_if SystemRepository.repo_exists? 'default-cluster' action :nothingend
Lifecycleevents
Recipes for each event type
● A common and well-documented tool.○ And it’s Ruby, a skill we possess.
● Can re-use existing cookbooks.○ eg. Tomcat, Yum.
● Should be platform-agnostic.○ Can theoretically use the same Chef recipes against
any OS, if written well.
Chef – Advantages
● OpsWorks support is a little lacking.○ Using Berkshelf for dependencies is tricky.○ Many of the error messages are a bit cryptic.
Chef – Disadvantages
What next?
What next?
● Autoscaling○ Not supported by OpsWorks.○ For now: predict spikes, use time-based instances.
What next?
● Autoscaling○ Not supported by OpsWorks.○ For now: predict spikes, use time-based instances.
● Backups○ We still do these from our DCs.
What next?
● Autoscaling○ Not supported by OpsWorks.○ For now: predict spikes, use time-based instances.
● Backups○ We still do these from our DCs.
● Chef improvements○ The recipes do too much at the moment.
What next?
● Improve Chef recipes○ The recipes do too much at the moment.○ Example: cluster management.
Questions?
