Virtualization Newsand Plans

Luigi GalleraniBE-CO-IN

TC 27 Nov 2014

Virtualization News and Plans2Outline

Motivation:• Improvement: Experience in VPC Consolidation• VPC Limits identified by user feedback• Phase out of IT HyperV Service Openstack

Goal: Design the new best solutions for ACC• Priority on ACC Requirements/ Constraints• Deep Evaluation of CERN OpenStack • Alternatives investigations

Virtualization News and Plans3VPC Improvements experience

• 16 New VPC Servers (Total=29) • 175 Machine migrated to x64bit • Java7 and JWS Migration • All 18 cerntsab* replaced with 32

Virtual Terminal Servers& Clusters• Many bugs fixed and IT issue

solved with strong IT collaboration • Summer Student Project (Marina)

on backend automation

extract from Virtualization@BE-CO Analysis, Strategy , Solutions & Future Plans

L. Gallerani - TC Feb 2013

Lot of experience gainedConfident we can offer more

and better for the future

Virtualization News and Plans4User Feedback survey 2014

Many issue discovered and fixed thanks to your feedback

Tech Student Project started from your feedback comments

Use cases where VPC are not optimal have been clearly identified

VPC user feedback survey page in our Wikiswikis.cern.ch/display/VM/VPC+Feedback+from+user+side+Feb+2014

Virtualization News and Plans5Optimization after feedback survey

Fixes during Technical Student Project (Fotis)

SQL Developer running on NX

Automatic wrong user environment and unused machines detection

Resource upgrade analysis and RAM CPU Network upgrade

Perf tuning based on FESA class Compilation benchmark analysis

slides from Fotis LiatsisSeptember 2014 BE-CO-IN Section Meeting

Virtualization News and Plans6IT Hyper-V Service Phasing out

BE-CO VPC are running on IT Hyper-V CVI service

IT Hyper-V service has been frozen for all users except BE-CO

Hyper-V Replacement is based on OpenStack

TECHNOLOGY UPGRADE

vmm.cern.ch home page – service is phased out

Virtualization News and Plans7Design ACC-Dev infrastructure for

futureMotivation: Improvement: Experience in VPC ConsolidationVPC Limits identified by user feedbackPhase out of IT HyperV Service Openstack

Goal: Find the best solutions for BE-CO:• Priority on ACC Requirements/ Constraints• Deep Evaluation of CERN OpenStack • Alternatives investigations

Virtualization News and Plans8Priority on ACC Requirements

My role is to find the optimal infrastructure solutions for the future considering:– Developers (Java, C++ Fesa, but also WinCC, Siemens,

Labview, Mathlab, PVSS, Schneider Twido…)– Operational support (fast intervention and bug fix, CO,

OP, BI, RF, but also experts of EN cryo, cooling, ventil…) – SysAdmin and DevTools support – Resources (Time, money, technology, performance…. )– CERN IT solutions available and supported

Virtualization News and Plans9ACC Constraints

• No TN in offices• No TN trust if machine not managed by BE-CO• TN access = No Internet connection• Only restricted access to our NFS Servers • Only ACC users in the ACC infrastructure• Limited number officially supported solutions• Migrate dev infrastructure away from the TN

Virtualization News and Plans10Migrate Dev away from TN

Document written by Vito Baggiolini (BE/CO), Alastair Bland (BE/CO), Uwe Epting (EN/CV), Luigi Gallerani (BE/CO), Timo Hakulinen (GS/ASE), Stefan Lüders (CSO), Stephen Page (BE/CO)With comments by Pierre Charrue (BE/CO), Stephen Jackson (BE/BI), Lars Jensen (BE/BI), Chris Roderick (BE/CO), Katarina Sigerud (BE/CO), Wojtek Sliwinski (BE/CO), Andy Butterworth (BE/RF), Jorg Wenninger, Kajetan Fuchsberger (BE/OP)

Fully migrate the current development infrastructure away from the TN and fully decouple them;

Extract from page 3

NEXT TC

Proposal

Virtualization News and Plans11Design ACC-Dev infrastructure for

futureMotivation: Improvement ExperienceVPC Limits identified by user feedbackPhase out of IT HyperV Service Openstack

Goal: Find the best solutions for BE-CO:• Priority on BE-CO Requirements/Constraints• Deep Evaluation of CERN OpenStack • Alternatives investigations

Virtualization News and Plans12CERN Openstack

• Openstack is what IT is offering as replacement of current virtual machine infrastructure(HyperV now obsolete)

• Large portion of IT Computer Center is migrating from physical to OpenStack VirtualSlide from Openstack@CERN Presentation

by Belmiro Moreira (CERN IT)More info at openstack.cern.ch

Virtualization News and Plans13CERN Openstack project scale

Slide from Presentation by Thomas Oulevey (CERN IT )More info at openstack.cern.ch

ACC

Virtualization News and Plans14Scaling up in #cores, not speed

HARDWARE of the FUTURE:More cores vs clock speed Huge RAM available

Service orientedEasy and Cheap to virtualize many “slow” machines for general purpose use : - ideal for IT computer center - ideal for BE-CO Terminal Servers

Performance orientedHard to get high performance virtual machine for compilation(what our developers need)

Virtualization News and Plans15Investigating Open-stack for BE-CO

• Usability Test • Performance analysis

For BE-CO• Migration plan of current

540 VPCs with low impact on users

• Request special config for ACC (8-Cores, 8GB RAM)

The BE-ACC-VPC-TEST OpenStack project home page

Virtualization News and Plans16CPU benchmark: VPC vs Openstack

YOUR BE-CO VPC

OpenStack

Preliminary results - benchmark comparison done by Fotis Liatsis

Average

Integer

Float

Prime Test

Extended SSE

Compression

Performance oriented

Virtualization News and Plans17HDD benchmark: VPC vs OpenStak

YOUR BE-CO VPC

OpenStackAverage

Seq read

Seq Write

Random seek

Performance oriented

Preliminary results - benchmark comparison done by Fotis Liatsis

Virtualization News and Plans18Openstack not yet ready for BE-CO Dev

benchmark comparison done by Fotis Liatsis between Windows BE-CO VPC (in red) vs

Openstack (green& blue)

• Openstack today is not ready for BE-CO dev needs at the moment

• Performance issues for development

• We will lead acceptance tests before saying yes

• IT promise to put in place improved solutions

Performance oriented

Virtualization News and Plans19Openstack is great for BE-CO TS

BE-CO Openstack Virtual Terminal Servers

for experts in cryo, vent, ele, en-ice…

Pilot project driven by BE-CO-IN in collaboration with EN-ICE and IT-OS

To provide better expert application terminal servers TN Trusted

Slides from S. Bukowiec IT-OS, P. Golonka EN-ICE & L. GalleraniTerminal Server Cluster pilot project presentation

service oriented

Virtualization News and Plans20Virtual Terminal Server Clusters

ACCEPTED & RUNNINGcerntsice cerntscryo cerntselcerntscv

slide from S. Bukowiec IT-OS, P. Golonka EN-ICE and L. GalleraniTerminal Server Cluster pilot project presentation (now in production and running)

service oriented

Virtualization News and Plans21Advantages for BE-CO

of the new Openstack clustersfor experts terminal servers

• SERVICE ORIENTED CLUSTERS for many users• Scale horizontally: service overloaded?

more virtual servers added (or duplicated) to the clusters (parallel scaling)

• Upgrades without stopping service in the cluster• HA: If a node goes down service stays up

service oriented

Virtualization News and Plans22Design ACC-Dev infrastructure for

futureMotivation: Improvement ExperienceVPC Limits identified by user feedbackPhase out of IT HyperV Service Openstack

Goal: Find the best solutions for BE-CO:• Priority on BE-CO Requirements/Constraints• Deep Evaluation of CERN OpenStack • Alternatives investigations

Virtualization News and Plans23VPC Alternatives investigation

Alternative to VPC page in our Wikis https://wikis.cern.ch/display/VM/Alternatives+to+BE-CO+VPC?

Virtualization News and Plans24Possible Alternatives under analysis

Openstack is not the unique solution we are evaluating for performance oriented development:

• Physical desktop PC in the GPN not TN Trusted– Nice Windows – Standard CERN Linux with mechanism to get secure NFS– BE-CO linux managed by us for GCC (man power?)

• Physical linux servers for high performanceRemote X11 sessions with xRDP (no nx licence)

• Others: CernVM? VirtualBox? Lightweight virtualization?

Performance oriented

Virtualization News and Plans25Desktop GPN Linux not TN Trusted

ACC Eclipse with FESAPlugin running in GPN not TN trusted

Developer can browse internet

Standard CERN SLC6 in GPN

Screenshot from physical GPN not TN Trusted desktop PC running ACC Eclipse with Fesa PluginTested by BE-BI Developer (M. Ferrari)

Virtualization News and Plans26Ways to mount NFS from GPN

Different way to provide secure access to NFS, and different scenarios analysis

SFTP / SSHFS

PERFORMANCE TESTS?SSHFS Side Effect:

25% CPU taken for encryption

during compilation(can be slower than VPC!)

Secure controlled access to

NFSvia single gateway

using SFTP and SSHFS

from desktop PC

BE-CO Linux only

MORE on ACC

services in

GPN in the

NEXT TC

NFS4

Virtualization News and Plans27High perf physical linux with XRDP

Windows native RDP connecting to BE-CO linux where XRDP linux server is running

Windows native Remote DesktopConnecting to Linux VPC

Could be used as solution for high performance compilation

linux servers

Virtualization News and Plans28Timetable

• HyperV available for BE-CO until new satisfactory solution is found (max Dec 2015)

Possible solutions • Openstack tuning and evaluation (April 2015) • Prototype GPN Desktop (April 2015)• Prototype Linux server (April 2015)• July 2015: Decision • Working solutions in production Dec 2015

Virtualization News and Plans29Conclusions

Ready for changes, motivations and competences – Lot of experience and competences gained in VPC Consolidation– User feedback requests, IT technology changes

Analysis of Best solutions for BE-CO infrastructure illustrated– Priority on ACC Requirements/Constraints– Evaluation of new technology like OpenStack performance and

service oriented , alternatives taken into account– Timetable

We will move in 2015 only when satisfactory solutions are validated and accepted

Questions? Virtualization News and Plans

Luigi.Gallerani@cern.ch

Presentation available in DFS\\cern.ch\dfs\Users\l\lgallera\Public\ TC2014LuigiGallerani.pptx

Virtualization News and Plans31Multiple solutions consideration

MOST OF DEV DONE IN THE TNTN TRUSTED MACHINE ONLY FOR FINAL VALIDATION

TN TRUSTED MACHINE FOR FAST BUG FIX

FOCUSING ON THE GPN SOLUTIONS examples:GCC / driver compilation? BE-CO Linux managed by us, remote desktop to cernts for desktop applications. Clear statement what we support. Java Developer Standard CERN Linux or Windows private machine with ACC Eclipse, remote desktop to cernts for desktop applications when use SLC on local machine. Nice machine are supported by IT ServiceNow with developer as main userLinuxServers BE-CO linux fast remote development and support and bug fix