Fosec2011 keynote address

From imagination to impact

Engineering Cloud Computing SolutionsThe Enterprise Consumer Perspective

Dr. Anna LiuResearch Group LeaderSoftware SystemsNational ICT Australia

The Land Down Under

Sydney

5

About NICTA

National ICT Australia

• Federal and state funded research company established in 2002

• Largest ICT research resource in Australia

• National impact is an important success metric

• ~700 staff/students working in 5 labs across major capital cities

• 7 university partners• Providing R&D services,

knowledge transfer to Australian (and global) ICT industry

NICTA technology is in over 1 billion mobile phones

6

Australia’s National Centre of Excellence in ICT Research

Industry engagement

MEDICAL

AutoMapRedLizards.com

NICTA’s mission: to be an enduring world-class ICT research institute that generates national benefit.

Industry outcomesEnduring solutions‘Spinout’ companies

Publicly funded, not for profit

Research focused on areas of importance to Australia

Engagement models include… Contract R&D Consulting services Strategic Partnerships Licensing

Best of breed research teams (400 staff + 300 students)

7

Research Areas at NICTA

Networks

Aruna Seneviratne

Anna LiuGernot Heiser

Software Systems

Machine Learning

Bob Williamson

Computer Vision

Nick Barnes,\ Richard Hartley Peter Corke

Rob Evans

Control & Signal Processing

Mark Wallace, Sylvie Thiebaux, Toby Walsh

Optimisation

8

Our Research Capabilityspans cloud computing, web, SOA, distributed systems, data management, analytics, performance monitoring, DR, automated reasoning, ontologies, AI…

Intelligent management

Business continuity

Dynamic

Cost optimised

High availability

High performance

Disaster recovery

Systems resilience

Real-time monitoring

Actionable analytics

Hybrid cloudOnsite/offsite

ElasticReal time

Our team’s mission: help enterprises take full advantage as software extends into cloud!

Agenda

• The Enterprise Perspective• Evaluating cloud computing• Business opportunities• challenges

• Proof on Concept Experience• Workload appropriate for cloud• Technical architecture• Migration issues• Business and commercial considerations

• Future of Software Engineering FOR and IN Cloud• What’s so different about cloud?• NICTA current research in cloud• What’s to come

Enterprise Cloud ComputingThe Business Values

• High Elasticity/Scalability leads to agility– Virtually infinite amount of resources is available on

demand

• Reduce cost and complexity– Pay per usage, economies of scale

• Generally speaking, non-7x24x365 systems with higher resource usage bring large cost savings

– No in-house IT maintenance– No up-front cost , geographically distributed disaster

recovery

• Innovation Possibilities– Ease of Use, speed to market with minimum capex– Processing Big Data

• Cost of 1 machine for 100 hours = Cost of 100 machines for 1 hour

Enterprise Cloud Computing - The Challenges

12

•Top risks/adoption issues:–Security & privacy - Migration challenges

–Ownership of data – Service levels–Lock-in / interoperability – Performance

–Availability / reliability – Cost and ROI–Monitoring & control – Governance–Operational challenges - Competencies–Compliance and regulation –Software licensing in cloud–Contracts and commercials–new roles and responsibilities–Payment model, metering/charge backs

•Risks vary with service model and provider

Australian Cloud Adoption Snapshot

• Software as a service• Enterprise and SME• Productivity suites, CRM• Telco and SaaS vendor partnership• emerging tier 2 System integrator

• Platform and Infrastructure as a Service• SME, startups well on their way• Enterprise doing evaluation

• Government Cloud, Community Cloud• Data centre consolidation• SOA, shared services• Financial industry leadership

13

Some Australian Enterprise Proof of Concepts

• Internet scale web applications• User base from around the world• Integration with existing web APIs • Transient campaigns

• Many Mobile devices connecting to cloud• Good adoption in utilities industries

• Development/Test environment• Dynamic provisioning of dev/test resources• Pay for usage

• Bursty workload• Web apps

• Large scale data analysis• eScience, Financial risk calculations, Government statistical

data14

Agenda




16

Proof of Concept Overview

• Objective• reduce IT cost• evaluate cloud opportunity and risks

• Test and Dev environment, as opposed to production

• Maximise re-applicability of learning experience across other apps

• Evaluation dimensions• Performance, security, feasibility• cost and license, flexibility and elasticity• integration with existing environment, migration effort• disaster recovery and backup, new roles and

responsibilities• …

Solution Design Rationale

• POC Solution Design Rationale• Standard 3 tier web application, with backend and

authentication server integration • Location of data tier• Maintain as much as dev/test configuration as common as

possible• PaaS or IaaS• Selection of cloud platform for POC

• Project Management• Governance: CIO/Director level sponsorship• Project participants: enterprise architect, solution developer,

security specialist, commercial specialist• NICTA: cloud computing experience and evaluation

framework• 2 wks POC selection; 6 wks POC; 2 wks consolidate findings17

18

Architecture of a Hybrid Dev Environment

Private Cloud (Isolated Network)Only accessible from NICTA

NICTA Corporate Network

Isolated Network in Amazon

Amazon Cloud (US-East Datacenter)

IPSec VPNapprox 230ms

RTT

Remote-desktop to XX.XX.0.*(No direct access to Amazon VPC)

Internet

On-Premise Servers

Enterprise Data storeAuthentication server

Virtual Machines

Business Web application

19

Security

• There is ‘Secure integration to cloud’ solutions emerging– Amazon VPC, Google Secure Data Connector, Azure App

Fabric, etc

• Standard IPSec-VPN brings peace of mind to enterprise users– One of the strong key enablers for enterprise use– Fit in an existing security policy

• Data masking could increase the cost/effort– An automated method is necessary for further cost/effort

reduction

• Secure Software Development Lifecycle– Process change required

20

Performance

• The performance of each component (network, VMs, …) in cloud is comparable to or better than current on-premise components– For dev/test environments, suitable for production

systems?

• Do not underestimate the latency in hybrid environments– Many of traditional applications and protocols are not

optimized for a high-latency/WAN environment• E.g., a protocol is too “chatty” and we observed that the

network usage never exceeds 0.1% in some cases

– There are performance improvement opportunities• Alternative solution design, Configuration and tuning

21

Cost

• Many companies use ‘private cloud’; however, current offering is seen to be more expensive and less flexible– increasingly Pay-as-you-go options are available– unit price is typically more costly for storage – SLA & management services usually included– Cost of keeping data/VMs is larger

0.00

500.00

1000.00

1500.00

2000.00

2500.00

USD

Min Max

Annual Operating Cost

Monitoring

Storage

Data Transfer

VPN

VM/License

• Current Cost would vary depending on the SLA tiers of service

Infrastructure Configuration(VPN, VMs, Disk, …)

Infrastructure Configuration(VPN, VMs, Disk, …)

OS

/Ap

plic

ati

on

Secu

rity

(e.g

., A

ctiv

e D

irect

ory

)O

S/A

pplic

ati

on

Secu

rity

(e.g

., A

ctiv

e D

irect

ory

)

OS/Middleware Installation/ConfigurationOS/Middleware Installation/Configuration

OSPatching

OSPatching

Application Installation/ConfigurationApplication Installation/Configuration

ApplicationPatching

ApplicationPatching

Bill

ing

(Cost

Cen

ter

Ch

arg

ing)

Bill

ing

(Cost

Cen

ter

Ch

arg

ing)

AntivirusAntivirus OSBackup

OSBackup

OSMonitoring

OSMonitoring

App DataBackup

App DataBackup

ApplicationMonitoringApplicationMonitoring

Amazon EC2(IaaS providers)

InfrastructureMonitoring

(CPU, Disk, Net, …)

InfrastructureMonitoring

(CPU, Disk, Net, …)

Usage Reportand

Basic Billing

Usage Reportand

Basic Billing

Access Controlto IaaS

Access Controlto IaaS

Customers’ Responsibility in IaaS Cloud

Customers’Responsibility

Commercial Implications

• Software Licensing in the cloud?• Reuse enterprise license• Pay for usage software license model

• Payment model?• enterprise governance model• Metering and chargeback

• Service level agreement?• Monitoring and management• Contracts• Backup, disaster recovery

• New roles and responsibility?• Existing IT outsourcing arrangements

23

POC Experience Summary

• Cloud Computing has the potential to reduce existing enterprise IT cost

• There are technical solutions for managing performance, security risks

• Need some fresh approach to manage:• Enterprise architecture and governance• Commercial implications such as SLA, new roles and

responsibility

24

Agenda




What’s so Different About the Cloud?

• Key Architectural Differences• Data structure (key value store, NOSQL vs relational)• Transactional guarantee (BASE vs ACID)• Elastic compute capability• Unpredictable Unavailability • Geographic distribution (latency across WAN)• Tight integration between development and deployment...

• These differences directly Impact Software Engineering and Software Architecture best practice!

• New data architecture, abstractions, programming models• New architecture trade off concerns, architecture patterns• Replicate everything architecture, new disaster recovery

mechanisms• Emergence of ‘DevOps’ influences future software

engineering process

27

Elastic Compute Capability

• Elasticity is the defining characteristic of cloud computing

• The aim is to allocate sufficient resource to do the job, but not too much such that it wastes resources

• There are broadly 2 architectures that achieves elastic compute capability– Push architecture– Pull architecture

28

Elastic Compute Capability Reference Architecture –Push Architecture

• The Push architecture is typically used for web applications– Web browser (client) send a request to the web

application side– Load balancer receives the request and “push” to one of

the web servers running on a compute node• Requests are forwarded immediately (or at a certain rate)• Load balancer is aware of the intensity of the workload

29Fig 1. Push Architecture Pattern

Forward to nodes

Load Balancer/Queue

Clients

Send request/connect to server

e.g., web browser, DB client

e.g., Amazon Elastic LB, GAE

Task Queue

Computing Nodes (e.g., VMs, processes, …)

Resource Pool

Monitor

provision

deprovision

invoke

e.g., Amazon CloudWatch, Azure

Diagnostic API

monitor

Controller

Rules

use

e.g., Amazon Auto Scaling

Elastic Compute Capability Reference Architecture

30

Fig 2. Pull Architecture Pattern

dequeue (pull) jobsand process

Computing Nodes (e.g., VMs, processes, …)

Queue

Monitor

Clients

enqueue

e.g., web layer, client apps

monitor

e.g., Amazon SQS, Azure Queue

Controller

Rules

use

provision

deprovision

invoke

Resource Pool

Elastic Compute Capability Reference Architecture –Pull Architecture

• The Pull architecture is often seen as an application-level architecture– Also known as the Producer-Consumer design pattern– Requests are sent to a queue

• In contrast to the Push architecture, it does not forward the request (hence less suitable for web applications)

– Compute nodes polls the queue periodically for jobs• Requests are processed one at a time• Polling frequently can induce overhead

– Easier to implement fail-safe mechanism• Compute nodes need NOT inform the queue in case of

failure• Typical fail-safe mechanism involves a queue (e.g., AWS SQS

or Azure Queue) that employs a lock attached with a timer. A message is locked when polled by a node. In case of a node failure, the message lock expires and return the message back to the queue.

31

Using Cloud for Business Continuity

• Two main usages of cloud for Business Continuity:– Provides highly available systems for day-to-day business– Serves as a technology platform to implement disaster

recovery• Some definitions:

– Business Continuity: “Activity performed by an organisation to ensure that critical business functions will be available to customers, suppliers, regulators and other entities…”

– Disaster Recovery: “A small subset of business continuity. The process, policies and procedures related to preparing for recovery or continuation of technology infrastructure critical to an organisation after a natural or human-induced disaster”

– Fault Tolerance: “The property that enables a system to continue operating properly, possibly at a reduced quality level…”

32

Building Highly Reliable Systems with Cloud

• Must address potential failures at two levels:– Hardware/Infrastructure

• To prevent Single-Point-of-Failure (SPOF) by adding redundancy in all hardware components (i.e., redundant disks, redundant network devices, redundant power supply, etc.)

• NOT all cloud providers provide enterprise grade availability. Check your SLA!!

– Application• Prepare fail-over system to take over in case of a failure• Database replicates to minimise downtime and loss of data• Replicate to geographically different location (e.g., to avoid

natural disasters such as floods)

33

Case Study: Building Reliable System using EC2

• Highly replicated architecture of cloud makes them great as foundations for business continuity solutions

• Globally distributed nature further enhances the disaster recovery capability of cloud

• Availability limitations means need to be realistic about Hot vs Warm vs Cold standby options

34

Availability Zone A Availability Zone B

Minimum Size= 1Availability Zones = A, B, CMinimum Size= 1Availability Zones = A, B, C

Auto Scaling Rule

Availability Zone C

EC2 Instance

Create

Elastic IP addressxxx.xxx.xxx.xxx

Allocate

Availability Zone A Availability Zone B

Minimum Size= 2Availability Zones = A, B, CMinimum Size= 2Availability Zones = A, B, C

Auto Scaling Rule

Availability Zone C

EC2 Instance

Forward Request

EC2 Instance

Elastic Load Balancer

Request from Clients Availability Zones= A, B, C

35

The Reality of Eventual Consistency in Amazon SimpleDB

• The probability to read updated data in SimpleDB in US West– An application reads data X (ms) after it has written data

• SimpleDB has two read operations– Eventual Consistent

Read– Consistent Read

• This pattern is consistent regardless of the time of day

Eventual ConsistentConsistent Read

36

Other Commercial NoSQL Databases

• Google App Engine– Offers eventual consistent read and consistent

read– Behavior of eventual consistent read is

completely different from Amazon’s– In GAE, both types of reads behave exactly

same unless data centers have a failure(s)• Windows Azure

– Offers no options for read– Always consistent

Reference: H Wada, A Fekete, L Zhao, K Lee, A Liu, “Data Consistency PropertiesAnd the Trade-offs in Commercial Cloud Storage: The Consumers’ Perspective”,CiDR 2011. http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper15.pdf

http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper15.pdf

What’s so Different About the Cloud?

• Key Architectural Differences• Data structure (key value store, NOSQL vs relational)• Transactional guarantee (BASE vs ACID)• Elastic compute capability• Unpredictable Unavailability • Geographic distribution (latency across WAN)• Tight integration between development and deployment...

• These differences directly Impact Software Engineering and Software Architecture best practice!

• New data architecture, abstractions, programming models• New architecture trade off concerns, architecture patterns• Replicate everything architecture, new disaster recovery

mechanisms• Emergence of ‘DevOps’ influences future software

engineering process

37

Research Agenda

• Enterprise Architecture Framework• Evaluation, acquisition, effort estimation, project and risk

management

• Software Development Lifecycle• Requirement solicitation for cloud, design for interoperable

services, MDA/MDD/DSL, testing at massively parallel scale, cloud design patterns

• Interoperability and Integration• Hybrid cloud, integration challenges across clouds

• Performance Engineering• Monitoring and measurement, performance modelling,

prediction and analysis, quality of service, SLA and assurance

• Many more…

38

Cost Effort Estimation for Cloud Migration

Cost implication/estimation for cloud migration is especially challenging because:– Applications and migration projects vary in terms of:

size/complexity, functionality, quality requirements, target deployment platforms...

– Cloud computing is new and different from traditional software engineering paradigm: different development and deployment models, non-functional characteristics, pricing models...

– Migration effort/cost estimation is not trivial– Little Empirical Data in cloud

• V Tran, K Lee, A Fekete, A Liu, J Keung, “Size Estimation of Cloud Migration Projects with Cloud Migration Point (CMP)”, 5th Intl Symposium on Empirical Software Engineering and Measurement

• V Tran, J Keung, A Liu, A Fekete, “Application Migration to Cloud: A Taxonomy of Critical Factors”, ICSE Software Engineering For Cloud Computing Workshop 2011. 39

Adaptive Cloud Middleware Research

• Evaluating Cloud Performance – Measuring Elasticity• Achieving Cloudburst – Integrated monitoring and

management• Cloud Data Management – Elastic Data Store

– S Sakr, L Zhao, H Wada, A Liu, “CloudDB AutoAdmin: Towards a Truly Elastic Cloud-Based Data Store”, 9th IEEE Intl Conf on Web Service ICWS 2011.

– S Islam, J Keung, K Lee, A Liu, “An Empirical Study into Adaptive Resource Provisioning in the Cloud”, IEEE Intl Conf on Utility and Cloud Computing UCC2010.

– L Zhao, A Liu, J Keung, “Evaluating Cloud Platform Architecture with the CARE Framework”, APSEC 2010.

– P Brebner, A Liu, “Modeling Cloud Cost and Performance”, Cloud Computing and Virtualisation (CCV 2010)

40

What Is Cloudburst?

Application A

Application B

Application C

Private Cloud

Spikes in demand for App.C but your private cloud has no resources! Application C

Application A

Application BIf App. C has huge amount of data or has sensitive data to transfer

Cloudburst

reconfiguration Application A

Application B

Application C

Application C

Public Cloud

Rent computing resources in public cloud(s) and replicated App. C to meet the (short-time) demand

• Dynamic reconfiguration of applications to use a public cloud when a private cloud cannot provide enough computing resources49

Conclusion

• Cloud Computing adoption is happening rapidly at the long-tail

• Challenges remain for Enterprise to adopt cloud computing

• The cloud computing model embodies many architectural differences that requires different software engineering approaches

• There are many tough Software Engineering research challenges to be solved in the new cloud context

53

Standing on the shoulder of giants

• The teamHiroshi Wada, Kevin

Lee, Adnene Guabtni, Sherif Sakr, Alan Fekete, Quanqing Xu, Sean Xiong, Bruce McCabe, Jacky Keung, Paul Bannerman, Liang Zhao, Sadeka Islam, Van Tran, Xiaomin Wu…

Getting Involved

• Linkage with National ICT Australia

• Research Collaboration• Researcher exchanges• Expert Advisory Services, Architecture

Reviews• Public and In-house Training Courses • Market Surveys, Case Studies• Professional in Research Residence

[email protected], @annaliuhttp://blogs.unsw.edu.au/annaliu/

mailto:[email protected]

http://blogs.unsw.edu.au/annaliu/

From imagination to impact

57

Alternative Architecture of a Hybrid Dev Environment (Non-VPN based)

Private Cloud (Isolated Network)Only accessible from NICTA

NICTA Corporate Network

Isolated Network in Amazon

Amazon Cloud (US-East Datacenter)

Secure connection (e.g.,

SSL)

Remote-desktop to XX.XX.0.*(Possible direct access to Amazon VPC)

Internet

On-Premise Servers

Enterprise Data storeAuthentication server

Virtual Machines

Business Web application

58

Alternative Architecture of a Hybrid Dev Environment (contd)

• Characteristics of a non-VPN based architecture:– Simpler to setup and more light-weight

• No special hardware required• Preserves isolated network in Amazon (i.e., cloud hosts with

private IPs)

– VPC host can directly access the internet• Assign elastic IP (i.e., public IP) to VPC host if internet access

is required• Arguably less secure (because two firewalls to take care of)• Yields better throughput to internet hosts (because no

rerouting through in-house network)

– Suitable for applications with fewer connection points between in-house and cloud

2. Hybrid Cloud Control Centre

• Extensible architectures supporting various plug-ins

• Diagnose and suggest optimal system configurations

• Auto generation of reconfiguration workflows

04/10/202359

Understandat a Glance

Public Cloud

In-House Data Center

Hybrid Cloud Environment

Diagnose and Plan

Your Future

Automate AdaptationsMonitor

Everything You Have

Monitoring Engine Decision Making Support

• Integrated monitoring across local and remote public clouds

• Works with existing enterprise monitoring and mgmt tools

3. Cloud Computing Cost Estimator

System Monitoring(ACT Monitor)

IT Administrator

• Resource consumption per business transaction

• Daily, weekly, monthly, yearly usage patterns

• Possible deployment locations - US, EU, Asia or Australia

Application Profile

Live Usage Patternor

“What-If” Scenarios

Knowledge base oncost model, SLA, …

• Total operating cost on each vendor

• Monthly cost and break-downEstimated Operating Cost

Cloud Cost Estimator• Calculate operating cost of

applications

Cloud Computing Providers

Bondi Beach

Fosec2011 keynote address

Technology

Transcript of Fosec2011 keynote address