Parallel session: cloud services

Parallel session D: Cloud services

Chair: Ian Shepherd

Please switch your mobile phones to silent

19:30

No fire alarms scheduled. In the event of an alarm, please follow directions of NCC staff

Dinner (now full)Entrance via Goldsmith Street

16:30 - 17:30

Birds of a feather sessions

15:20 - 16:00

Lightning talks

Office365 in a smaller institution

Kevin Hill and Matthew Collins Leeds Trinity University

03/05/2023

Introduction

Office 365 in a Smaller Institution

Kevin Hill› IT Infrastructure Specialist – Core Software Systems

›Responsible for Office 365 deployment & management

Matthew Collins› IT Infrastructure Specialist – Networks›Responsible for projects integrating Office 365

03/05/2023

Leeds Trinity University

›Celebrated 50th Anniversary in 2016›One of the UK's top universities for employability

›Pioneered the inclusion of professional work placements with every degree

›Small University with 3700 Students and 400 Staff

› IT Department is only 15 people

Introduction


03/05/2023

Agenda

»Where were we?»What did we do?»Where are we now?»Where are we going?»What have we learned?»Summary and Questions


03/05/2023

Where were we?

Challenges»Coming out of a managed service with a 3rd Party»New Learning and Teaching Strategy› Emphasises active enquiry and collaboration› Flexible provision and choice in managing learning› Make full use of technology in teaching and learning

»Increasing demands on storage space»Improve Staff and Student communication and

engagementOffice 365 in a Smaller Institution

03/05/2023

Where were we?


Limitations

»Crumbly on premise Exchange 2010 environment

»Small ‘home folder’ quota file storage provision »SharePoint 2010 based intranet»Legacy telephone system

03/05/2023

What did we do?


03/05/2023

Where are we now?


03/05/2023

Where are we going?


03/05/2023

The Good Stuff…›Free!›Level the playing field with larger institutions›Platform For Possibilities›Builds on existing familiar experiences›Microsoft’s feature releases are now ‘cloud first’

What have we learned?


03/05/2023


The Good Stuff…

›New services & features launched regularly›Less on premise infrastructure›Better business continuity›Microsoft & Community Support›ADFS authentication platform


03/05/2023


The Bad Stuff…

›New services & features launched regularly›No traditional backup›OneDrive access methods / clients›Still requires onsite servers


03/05/2023


The Ugly Stuff…

›New services & features launched regularly–No change control–Licencing controls–Adverts for paid services


03/05/2023

Summary

»Move to Office 365 can be swift»Office 365 and Skype for Business presents lots of opportunities, especially if it fits strategically

»There are challenges – Do the research!»Deployment and user adoption strategy is criticalOffice 365 in a Smaller Institution

03/05/2023

Questions

Any Questions?


Kevin Hill: [email protected] Collins: [email protected]

All aboard the Cloud Express

Microsoft Azure ExpressRoute via the Janet network

03/05/2023

The Importance of Networking in the Cloud Federico GuerriniMicrosoft EMEA Technical Lead, Azure Networking & Security

All Aboard the Cloud Express

03/05/2023

What’s the challenge?


“Resulting from poor network latency, or how fast data is transferred from one location to another, these delays are frustrating to as end users, but are far more costly to businesses that depend on lightning-quick web experiences for their customers.” (3)

“Performance and uptime have a direct effect on the bottom line, and corporates are still worried about this aspect of cloud computing.” (2)

“The number one challenge to cloud adoption is networking issues, identified by 37% of the IT professionals surveyed.” (1)

“Your application performance is only as good as your network” (4)

03/05/2023

What’s the challenge?


“You cannot have a first class cloud without a first class network”Yousef Khalidi, Microsoft Corporate Vice President, Ignite 20160%

5%10%15%20%25%30%35%40%45%

Workloads in the cloud if network issues resolved

25%

42%

Workloads in the cloud if network issues resolved

Workloads in the cloud

today

+17pts+68%

03/05/2023

What’s the solution?


• 38 Azure Regions • 100+ Data Centres• Global and sovereign offerings• Ongoing commitment to local and industry compliance• Top 3 Networks Worldwide • 37 ExpressRoute locations – More than any cloud

03/05/2023

What’s the solution?


Cloud Customer Segment and workloads

Secure site-to-site VPN connectivity

• SMB, Enterprises• Connect to Azure

compute

ExpressRoute private

connectivity

• SMB & Enterprises• Mission critical workloads• Backup/DR, media, HPC• Connect to Azure & CRM

services

Internet Connectivity

• Consumers• Access over public IP• DNS resolution• Connect from anywhere

Secure point-to-site connectivity

• Developers• POC Efforts• Small scale deployments• Connect from anywhere

03/05/2023

ExpressRoute: Dedicated Connectivity


Customer Network

Customer Datacentre

Internet

Microsoft Network

Azure DatacentreAzure

Datacentre

ExpressRoute Location

Microsoft-owned

Customer buys these

links

Whole link covered by customer-controlled

business agreements

03/05/2023

What is the Value Proposition?

• ER = 10Gbps bandwidth • Connects directly to your WAN• Dynamic routing between your network and Microsoft over industry standard

protocols (BGP).• Built-in redundancy in every peering location for higher reliability.• 99.95% Connection uptime SLA.• QoS and support for multiple classes of service for special applications, such as

Skype for Business.

RedundancySLA

Predictable performance

High throughput


03/05/2023

Technical Overview


03/05/2023

ER deployment models

ER partner hands off a LAYER 2 service to the end customer

ER partner hands off a LAYER 3 service to the end customer


03/05/2023

How is this implemented in the point to point model?

Customer Network

Customer Datacenter

Microsoft Network

Azure Datacenter

AzureDatacenter


03/05/2023

Point-to-Point Ethernet connection

Customer Network

Customer Datacenter

Microsoft Network

Azure Datacenter

AzureDatacenter

Customer Microsoft

Customer-owned routers

Microsoft network infrastructure

Routers and connectivity provided

by the ER partner


03/05/2023

Point-to-Point connection BGP Sessions

Customer Network

Customer Datacenter

Microsoft Network

Azure Datacenter

AzureDatacenter

BGP

BGP

Customer establishes BGP sessions

Customer is responsible for addressing, routing and NAT

requirements


03/05/2023

Point-to-Point connection Implementation responsibilities

Customer Network

Customer Datacenter

Microsoft Network

Azure Datacenter

AzureDatacenter

BGP

BGP

Implemented by the customer

Implemented by the ER provider

Implemented by Microsoft


03/05/2023

Point-to-Point connection price structure

Customer Network

Customer Datacenter

Microsoft Network

Azure Datacenter

AzureDatacenter

Customer MicrosoftBGP

BGP1 BILL 1 BILL

Implemented by the customer Implemented by the

ER provider

Implemented by Microsoft


jisc.ac.uk

03/05/2023

Thank you

Federico Guerrini

MS EMEA Azure Networking & [email protected]


http://www.jisc.ac.uk/

The Janet dimensionIan Shepherd

Product manager, Janet Connectivity

03/05/2023All Aboard the Cloud Express

03/05/2023

Why ExpressRoute via Janet?

»Frederico has kindly covered “Why ExpressRoute?”

»So why “via Janet?”›You already have a Janet connection› It’s already fast and reliable›We already have a fast, reliable connection straight into the Azure network

› It’s a no-brainer, It’s what Janet is for›Don’t take my word for it………..


03/05/2023

Significant demand from Jisc members

»Higher Education»Further Education»Local Authorities»Learning Grids»Museums and Galleries»Research Institutes»NHS»Commercials»And More…..


03/05/2023

Demand for What?


03/05/2023

Staffordshire University»Staffs is a leading UK modern university»16,000 students, 85% / 15% under/postgrads, 1,400

staff»Areas of focus:

Business, Leadership and Economics Computing and Digital TechnologiesCreative Arts and Engineering Health and Social CareLaw, Policing and Forensics Life Sciences and Education

»2 year degrees, distance learning, PhD, MBA»Strong industry and international partnerships»Over 150, 3rd party and in-house applications &

services!»Cloud-first strategy


03/05/2023

Staffordshire University»Major project to move to Azure using ExpressRoute»Huge, pioneering commitment»Jisc became involved approximately 12 months ago»Connected Staffs to ExpressRoute service at 2 Gbit/s»Project has gone well (despite some hiccups)»90% of applications and services in the cloud› Finance, HR, VLE, Databases…….

»Little or no disruption and overall good performance »Important lessons learned


03/05/2023

Staffordshire University»Cloud is not a panacea – It won’t fix problems, it just

moves them»It’s a good opportunity to review systems & services»Migrate what you can, keep what you can’t, kill the

zombies»Most issues have been with application migration

rather than networking issues (VM builds, DB architectures etc.)

»Internal network must be up to scratch as well as the WAN

»Have a back-out plan, test it if possibleAll Aboard the Cloud Express

03/05/2023

Staffordshire University»Cloud is not a panacea – It won’t fix problems, it just

moves them»It’s a good opportunity to review systems & services»Migrate what you can, keep what you can’t, kill the

zombies»Most issues have been with application migration

rather than networking issues (VM builds, DB architectures etc.)

»Internal network must be up to scratch as well as the WAN

»Have a back-out plan, test it if possible»Don’t migrate your entire student records system the weekend before Networkshop!


03/05/2023

Jisc’s ExpressRoute Connection Service


03/05/2023

Where does Janet fit in?


Customer Network

Customer Datacentre

Internet

Microsoft Network

Azure DatacentreAzure

Datacentre

ExpressRoute Location

03/05/2023

Member-Janet-Microsoft


Member Janet Microsoft

Public Internet

Private Peering

JanetConnection

03/05/2023

With Janet but without ExpressRoute



Azure Consume

rs

Azure Resource

s

Public Internet

Azure traffic does not touch the public Internet.

Private Peering

JanetConnection

03/05/2023

With Janet and ExpressRoute



Public Internet

Azure traffic is separated from non-Azure traffic and tunnelled straight to the Azure infrastructure

Netpath L2VPNAzure

Consumers

Azure Resource

s

Trunked Janet

Connection

Private Peering

Private Connectio

n

O365 etc.

03/05/2023

Plumbing


Janet Core100%Janet Regions 90%Member Edge ?Member Firewall etc. ?

Must be able to support the required protocols appropriate to their function.

VLAN trunkingL2 VPN802.1Q-in-QBGP

03/05/2023

Plan B



Public Internet

Azure traffic is carried across its own dedicated access circuit and tunnelled straight to the Azure infrastructure

Netpath L2VPNAzure

Consumers

Azure Resource

sSeparate Janet

Connections

Private Peering

Private Connectio

n

O365 etc.

03/05/2023

Service Status

» Pilot Service carrying live member traffic

» Currently 2 x 10 Gbit/s between Janet and Azure

» Investment approved for 2 x 100 Gbit/s and more

» Approved projects in place for:›Full volume production›Additional cloud services providers›Professional Services package


jisc.ac.uk

03/05/2023

Thank You

Ian Shepherd

Product Manager, Janet [email protected]



RCUK Cloud Working Group outcomes

Philip Kershaw, STFC

RCUK Cloud WG

Logo credit: vecteezy.com

RCUK Cloud Working Group:supporting the research community in the application of

cloud computing technologiesNetworkshop45

Nottingham, Wednesday 12th April 2017

Philip KershawTechnical Manager, Centre for Environmental Data Analysis, RAL Space, STFC;

Chair, RCUK Cloud Working Group

RCUK Cloud WG


RCUK Cloud WG


Overview• [What is Cloud?]• Origins of WG• November Workshop• Legal, policy, regulatory issues• Technical Integration• Next steps

RCUK Cloud WG


Cloud 101: need to understand in order to exploit

5 essential characteristicsOn-demand self-

service Broad network access

Resource pooling

Rapid elasticityMeasured

service

3 service models

IaaS (Infrastructure as a Service)

PaaS (Platform as a Service)

SaaS (Software as a Service)

4 deployment models

Private cloud Community

cloudPublic cloud

Hybrid cloud

“Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction.” – NIST SP800-145

RCUK Cloud WG


• Cloud Computing for Research and Innovation - Report Aug 2015, with input from members of research community and contributions from industry– http://bit.ly/pdgcloud

• Where are we with adoption of cloud?– identifies the major technical and policy issues that are seen to be preventing

widespread take up of cloud services

• What needs to be done?– Four high level recommendations

• How do we get there – 5 year roadmap– to investigate these issues and provide closer integration of public and private

sector resources to improve the capability of the UK research community.

RCUK Cloud Working Group Origins

RCUK Cloud WG


Realising the potential of cloud computing for research applications

1. Community building – Cloud Computing Working Group (in place since 2015) to provide a clear community

focus for cloud computing 2. Technical integration

– of the UK National e-Infrastructure resources to promote workload mobility and to reduce technical barriers to entry.

3. Training and Support– Equip the research community with the right skills and support to fully exploit UK

National e-Infrastructure cloud resources.4. Legal, Policy and Regulatory Issues

– Policy changes needed within RCUK to grow the adoption of cloud computing– Policy actions that RCUK can initiate externally on behalf of the UK cloud computing

community.

RCUK Cloud WG


Research Councils UK National e-Infrastructure Group

RCUK Cloud Working Group

Cloud Special Interest Group

RCUK Cloud WG and the Cloud SIG

Task ForceTask

ForceTask Force

Research Community, cloud providers

National e-Infrastructure Project Directors Group

e -Infrastructure Security Access

Management WG

European and wider international initiativesLiaise with

Build relationships with

Reports to

Reports to

Track and collaborate with

Initiated by WG and SIGChallenges and Ideas

RCUK Cloud Working Group terms of reference: http://bit.ly/1NxG5R4

Cloud SIG terms of reference: http://bit.ly/1GuQyxi

http://bit.ly/1NxG5R4

http://bit.ly/1GuQyxi

RCUK Cloud WG


RCUK Cloud WG Membership• David Colling, Imperial College• Tim Cutts, Sanger Institute• David Fergusson, University of Edinburgh• Martin Hamilton, Jisc (Chair of SIG)• Adam Huffman, Francis Crick Institute• Philip Kershaw, CEDA, STFC (Chair)• Steven Newhouse, EMBL-EBI & ELIXIR• David Salmon, Jisc• Simon Thompson, Birmingham University• Jeremy Yates, UCL (Secretary)

RCUK Cloud WG


Community building: workshops• Imperial College, December 2015

– ~40 attendees– Invited presentations

• Crick Institute in November 2016 – ~120 attendees– Combination of invited speakers and talks solicited

from the community• Talks from the big 3 hyper-scale providers• Community and private cloud, OpenStack• Legal, policy and regulatory issues• Lightning talks from the community• Breakout/Interactive session

– Outcomes• Desire for collaboration around task forces e.g. HTC and

HPC compute (need to co-ordinate with HPC-SIG)• Cost and performance issues on-prem. compared with

public cloudPhotos courtesy of Martin Hamilton, Jisc

RCUK Cloud WG


Community building: website• https://cloud.ac.uk/

• Minutes from working group public

• Reports on and presentations from workshops:– https://cloud.ac.uk/2017/03/20/cloud-workshop/– https://cloud.ac.uk/workshops/nov2016/

• A resource to report back on our findings– Technical suitability of workloads for cloud e.g.

HPC and parallel file systems– Legal, regulatory, policy issues, costs

https://cloud.ac.uk/

https://cloud.ac.uk/workshops/nov2016/




RCUK Cloud WG


Legal, Policy and Regulatory Issues

• The goal is to bottom out real and perceived issues around the use of public cloud and to provide guidance to the research community.

– Build on work that has already done– Recognise that this is a changing and evolving area

• Strawman prepared January 2016

• Questionnaire to obtain feedback from the research community (Martin Hamilton)

– https://bit.ly/cloudlegal2016

• Session at November 2016 Workshop at the Crick, input from:– EMBL-EBI – experience from recent public cloud procurement– QMUL Cloud Legal Project

• Briefing note from the WG to provide guidelines currently in preparation

43%

30%

26%

Was legal advice sought?

NoYesNot sure

Response OrganisationsReview is still ongoing 10OK to use public cloud 5Not OK to use public cloud 1Occasional OK but restrictions around data, security and budget 1We have many use cases, hard to find general answer 1N/A – did not need permission for this particular use case 1Inconclusive result 1

https://bit.ly/cloudlegal2016


RCUK Cloud WG


Technical Integration• What are the opportunities, challenges, barriers?

– Managing and tracking costs– Matching research workloads (e.g. HPC) with public cloud architectures– Matching more traditional data access with cloud native e.g POSIX, parallel file

systems and object stores– Hybrid public/private – ability to move data and compute easily between providers

• How can we inform ourselves? - – Interaction and communication within the community to track developments e.g.

OpenStack Scientific WG, contact with representatives from public cloud providers– Dedicated pilots or ‘task forces’ organised through the WG to target particular areas of

interest . . .

RCUK Cloud WG


Technical Integration – current activities• Portability between cloud platforms

– Particle Physics Cloud Pilot– Workshop on use Terraform and Ansible (in planning)

• Use of parallel file systems with on-prem cloud:– https://cloud.ac.uk/reports/spectrumscale/ – Workshop on use of FUSE file system in the summer (in planning)

• Bulk data movement – co-ordinating with work in the NeI Project Directors Group

https://cloud.ac.uk/reports/spectrumscale/


RCUK Cloud WG


Particle Physics Cloud Pilot (an example task force)

• Goal: – Explore strategies for making code deployments interoperable between providers and so avoid

vendor lock-in

• To investigate:– Ability to port given workloads between different public cloud providers with minimal changes– Challenges related to workload needs and cloud topology – Focus on compute aspects rather bulk data movement– Focus on functionality rather than performance

• Domain-specific use case: workloads for Particle Physics CMS and ATLAS experiments– Work carried out be Andrew Lahiff, Scientific Computing Dept., STFC– Started mid-2016

RCUK Cloud WG


Particle Physics Cloud Pilot: technical approach

• Uses container-based solution from the ground up as a means of abstraction for portability – Docker + Kubernetes

• Kubernetes– Supports abstraction e.g. underlying storage with StorageClasses– Powerful for automated deployment and scaling

• On hyper-scale providers: AWS, Azure and Google Cloud Platform– Possible with thanks to the providers for donated free credits

• Target workloads– CMS Monte Carlo simulation - compute intensive– LHCb Monte Carlo just completed, ATLAS jobs planned– Next steps: focus on io intensive workload

RCUK Cloud WG


Particle Physics Cloud Pilot: summary by provider

• Google Compute Platform– Supports Kubernetes out-of-the-box– Supports auto-scaling in response to demand– Web portal Kubernetes interface – very easy to get up and running– Fairly extensive exploration of functionality

• Azure– Out-of-the-box since close 2016 with web portal interface– Supports other container orchestration technologies e.g. DCOS– Current work

• integration of Azure Kubernetes cluster with ATLAS Big PanDA workload management system• Use of Azure Blob storage with DynaFed storage caching system

– Scope for further extensive testing over the coming months

• AWS– No Kubernetes out-of-the-box but 3rd party solutions like StackPoint (https://stackpoint.io/ ) can be used

to overlay Kubernetes cluster across a given cloud provider tenancy– Very limited testing in this pilot to date

https://stackpoint.io/



RCUK Cloud WG


Particle Physics Cloud Pilot: federation across Google and Azure

Courtesy of Andrew Lahiff, STFC

RCUK Cloud WG


Next steps• Technical Integration

– Particle Physics Cloud Pilot: in i/o intensive workloads, close and report findings– HTC and cloud (depending on interest from the community): David Colling (Imperial) and David Salmon (Jisc)– HPC and cloud: discussions underway with HPC-SIG, tracking HPC on public cloud work e.g. NERC NCAS and Azure

• Workshops – training and dissemination– Lustre on cloud (planned): Simon Thompson, Birmingham– Using Terraform and Ansible to make workloads portable (planned): Steven Newhouse

• Legal, policy and regulatory issues, costs– Provide briefing note to provide guidelines to community: David Salmon– Track cost models of public providers

• Community: engage with Boston Open Research Cloud initiative– https://drive.google.com/drive/folders/0B4Y7flFgUgf9dElkaFkwbUhKblU – Attending first meeting in May following OpenStack Summit

https://drive.google.com/drive/folders/0B4Y7flFgUgf9dElkaFkwbUhKblU


RCUK Cloud WG


Further information• Working Group website: https://cloud.ac.uk/

• NIST SP800-145 Cloud definition: http://dx.doi.org/10.6028/NIST.SP.800-145

• RCUK Cloud Working Group terms of reference: http://bit.ly/1NxG5R4

• Cloud SIG terms of reference: http://bit.ly/1GuQyxi

• Cloud Computing for Research and Innovation - Report Aug 2015: http://bit.ly/pdgcloud

• Legal, policy and regulatory issues: https://bit.ly/cloudlegal2016 (further input welcome )

• POSIX, Parallel file systems and cloud: https://cloud.ac.uk/reports/spectrumscale/

• Boston Open Research Cloud initiative– https://drive.google.com/drive/folders/0B4Y7flFgUgf9dElkaFkwbUhKblU

• [email protected], @PhilipJKershaw

https://cloud.ac.uk/

http://dx.doi.org/10.6028/NIST.SP.800-145

http://dx.doi.org/10.6028/NIST.SP.800-145

http://bit.ly/1NxG5R4










mailto:[email protected]

mailto:[email protected]

jisc.ac.uk

03/05/2023

Contact

Philip [email protected]

Title of presentation (Insert > Header & Footer > Slide > Footer > Apply to all)


Thank you

03/05/2023Title of presentation (Insert > Header & Footer > Slide > Footer > Apply to all)

Parallel session: cloud services

Education

Transcript of Parallel session: cloud services