narhkede-PrivateCloudWipro
Transcript of narhkede-PrivateCloudWipro
-
8/3/2019 narhkede-PrivateCloudWipro
1/17
2009 Wi ro Ltd - Confidential
Private Cloud at Wipro
Cloud computing based on Condor
-
8/3/2019 narhkede-PrivateCloudWipro
2/17
2009 Wi ro Ltd - Confidential2
2009 Wi ro Ltd - Confidential2
3
1 Background2 Wipro Private Cloud
System architecture4 Use of Condor
Agenda
-
8/3/2019 narhkede-PrivateCloudWipro
3/17
2009 Wi ro Ltd - Confidential3
2009 Wi ro Ltd - Confidential3
Need:
Share physical infrastructure between multiple projects and CoE's (Centerof Excellence) to reduce server sprawl and number of physical labs
Provide an environment for evaluating new technologies, developingsolutions and enabling collaboration between multiple labs
Centralize infrastructure procurement and management Reduce infrastructure cost of CoE's by enabling multiple development
environments
Solution:
Setup a private cloud for virtual compute and application infrastructure
Build a self-service portal for on-demand provisioning to reduce processoverheads
Support multiple types of virtualization software
Reuse existing physical infrastructure, procure minimal new infrastructure
Background
-
8/3/2019 narhkede-PrivateCloudWipro
4/17
2009 Wipro Ltd - Confidential4
Wipro Private Cloud
-
8/3/2019 narhkede-PrivateCloudWipro
5/17
2009 Wi ro Ltd - Confidential5
2009 Wi ro Ltd - Confidential5
SaaS Mgmt
Wipro Users
SaaSUser Intranet
SaaSApp
SaaS Enablers
Wipro Cloud Portal / Web Services API Layer
Physical Resource Pool- Servers, Storage, Network
ManagedNetwork
CloudAdmin
C
loud
OA&
MPortal
Wipro Cloud CoreVirtual Machines Shared Storage Virtual Appliances
Application Services
AutomatedProvisioning Multi-tenancy & Isolation Cloud Accounting & Auditing
Performance & Fault Monitoring Automated Network & Security
Developers
Virtual Lab
ManagedNetwork
Wipro Private Cloud
-
8/3/2019 narhkede-PrivateCloudWipro
6/17
2009 Wi ro Ltd - Confidential6
2009 Wi ro Ltd - Confidential6
Service Element Service Feature
Compute Servers
Virtual desktop equivalent to 1.2GHz,512MB RAM,10GB HDD,25Mbps N/w
Low End Server equivalent to 2x1.2GHz, 2GB RAM, 20GB HDD, 25Mbps N/w
High End Server equivalent to 4x1.2GHz, 4GB RAM, 40GB HDD, 25Mbps N/w
OS types Linux (CentOS, RHEL) and Windows XP/Server on Intel x86, x86_64 architecture
StorageISCSI (RAID 5), NFS and CIFS
Data persistence across power-off, suspend & resume of VMs
Public images/appliances
Ready-to-use public images
RHEL 5, Windows XP, LAMP (CentOS 5.2, Apache, Axis, Tomcat, MySQL,PHP, Python)
Preconfigured Software load balancer, firewall appliances
NetworkIsolation between CoE's resources
IPSec, SSL based VPN
Public and Private IP Addresses with NAT support
Private imagesCan Upload VMware Server, VMware ESX and Xen Virtual Machine Imageformats
Reports Reporting on CPU, Storage and memory usage back to user
Cloud Services catalogue
-
8/3/2019 narhkede-PrivateCloudWipro
7/17
2009 Wi ro Ltd - Confidential7 2009 Wi ro Ltd - Confidential7
L1 - Virtual Servers on demand Virtual servers, desktops, storage
Migration assistance
Self-service portal
L2 Application infrastructure on demand
Appliances of standard software
Managed backup, proactive monitoring and help-desk
Itemized billing and charge-back
L3 Business service infrastructure on demand
Scalable business services
Multi-tenant application infrastructure (content management,identity management, database, load balancer, firewall, ...)
Levels of Service
-
8/3/2019 narhkede-PrivateCloudWipro
8/17 2009 Wipro Ltd - Confidential8
System Architecture
-
8/3/2019 narhkede-PrivateCloudWipro
9/17
2009 Wi ro Ltd - Confidential9 2009 Wi ro Ltd - Confidential9
ServiceLayer
Load
Balancer
AppLayer
VirtualMachineLayer
Bare-metalLayer
M
onitoring
CloudManagement
ProvisioningResource mgmtWorkload mgmtAuto recoveryTask & Process
Automation
Service
LB - Active LB - passive
Inst 1 Inst 2 Inst n
VM 1 VM 2 VM n
Provisioning
MonitoringPerformanceAvailabilityAlarmsBilling
ManagementService GovernorPolicy enforcementIncident mgmtOptimizer
Contention
Network Control Alarms
OA&M Portal &Web Service
Gateway
Customer
OA & M Access
Design, TestPackage, Deploy
Bare-metal designStandardizeAutomateRe-provisioning
Virtual M/c designStandardizeAutomateAgileCachingAppliances
Service designSLAs, Policies, rules,prioritiesPackagingCustom agentsShared ServicesBilling parameters
Configuration &Change mgmt
Developers Business UsersOperations
Private Cloud in Action
-
8/3/2019 narhkede-PrivateCloudWipro
10/17
2009 Wi ro Ltd - Confidential10 2009 Wi ro Ltd - Confidential10
Architecture & Service layers
Cloudservice
-
8/3/2019 narhkede-PrivateCloudWipro
11/17
2009 Wi ro Ltd - Confidential11 2009 Wi ro Ltd - Confidential11
IdentityManagement
Bare-metalprovisioning
Storageprovisioning
N/Wprovisioning
VMprovisioning
N/W Plug-in
StoragePlug-in
Bare-metalPlug-in
VM Plug-in
Grid Scheduler
Service Governor
Customer Portal
WorkflowManager Cloud State
VM Caching
Web ServiceGateway
VM Repo
Legend: Developed in Wipro 3rd Party components
Charge-back
N/W (nagios)Monitoring
Nagiosplug-in
Alerts
Metrics Monitor
In Development
System Components
-
8/3/2019 narhkede-PrivateCloudWipro
12/17 2009 Wipro Ltd - Confidential12 2009 Wipro Ltd - Confidential12
Cloud Backbone10.201.72.0/24
Cloud Mgmt192.168.3.0/24
Project X192.168.5.0/24
Project Y192.168.6.0/24
Project Z
192.168.7.0/24
Mgmt ServerHA Pair
Router, firewallVPN Server, IPS, IDS, NAT
Switch FabricCloud physical systems
Storage
VM
VM
VM
Virtual
Machines
VirtualStorage
Isolatednetwork per
project
Deployment Example
-
8/3/2019 narhkede-PrivateCloudWipro
13/17 2009 Wipro Ltd - Confidential13
Use of Condor
-
8/3/2019 narhkede-PrivateCloudWipro
14/17 2009 Wi ro Ltd - Confidential14 2009 Wi ro Ltd - Confidential14
Trusty old features
Flexibility ClassAd mechanism, configurations and policies
Web Services API
High availability
Resource utilization of jobs
Newer features we like VM Universe
Partitionable Slots
Lease management
Integration with Amazon EC2 (public cloud) Proven in large scale deployments
Condor-users and condor-admin support
Open source
Why Condor?
-
8/3/2019 narhkede-PrivateCloudWipro
15/17 2009 Wi ro Ltd - Confidential15 2009 Wi ro Ltd - Confidential15
Mostly standard configuration
A few custom class ads in jobs and machines
Schedd and Collector configured in HA mode
Condor spool for VM persistence
Virtual machine provision request handled by Condor VM job to physical machine match-making, file transfer
Partitionable slots for dynamic partitioning of physicalmachine resources
Customized condor_vm_* files for configuring and startingVM's VLAN control, Swap disk and additional storage creation, ...
Lease management for limiting the number of runninginstances of a licensed image
How are we using Condor?
-
8/3/2019 narhkede-PrivateCloudWipro
16/17 2009 Wi ro Ltd - Confidential16 2009 Wi ro Ltd - Confidential16
Working with Condor: With advanced Condor skills, a lot can be achieved without
modifying condor code
Workarounds: Passing number of virtual CPUs to VMware
Patch to pass proxy username and password to gSOAP for EC2integration
Patch to get VM resource usage details on ESX
Special configuration to handle 2 hour delay in detecting a fewexecute node failures (Thanks Todd!)
Feature wish list: Remote IWD support for VM universe, to avoid any file transfer
Live migration of VM jobs
Observations, Workarounds, Wish list
-
8/3/2019 narhkede-PrivateCloudWipro
17/17
Thank You
mailto:[email protected]:[email protected]