Google Compute Engine

SimonSu@mitac.com.twMiCloud 2014Q1

Prepare

● google cloud project● google-cloud-sdk with gcutil● ssh tool

Reference:https://sites.google.com/a/mitac.com.tw/google-cloud-platform/google-compute-engine/gce---sdk-install-and-auth

● GCE architecture ● GCE web UI to GCE CLI tool

○ Create, Snapshot, Create from Disk or Snapshot● Network & FW

○ 3-tier network implements● Instance option - start script

○ Using start script build a auto scale service

Today’s Objective

Compute Engine Architecture

Network

Firewall

Instances

IP(Static, Dynamic)

L3 Load Balancing

Something about GCE

● Billing: 1 Minute Increments, Minimum 10 Minutes● Security:

○ ISO 27001:2005 Certification for GCE, GAE, and GCS ● Location:

○ Region○ Zone

About the Instances

● Persistent Disk● Network block storage● Max of 16 disks/instance● Created independently of instance● 1 Virtual CPU is a Hyperthread on Processor● Current processor is 2.6 GHz Intel Sandy Bridge Xeon● No GPU or SSD Options

About utility - web ui, gcutil, restful

Compute Engine Web UI

From Web UI to CLI to RESTful

gcutil - Get HELP

➔ gcutil --help➔ gcutil help listinstances

◆ ex: gcutil listinstances --columns=all --format=json➔ https://developers.google.com/compute/docs/gcutil/tips

RESTful APIs

https://developers.google.com/apis-explorer/#p/compute/v1/

Connect to GCE machine

➔ gcutil ssh [instance id]➔ ssh [username]@[instance-ip] -i [path-to-google-ssh-key]

Windows connect GCE

● Prepare ssh private key for project metadata [Ref]

[username]:ssh-rsa [private keys value]

Network & Firewall & Instance Scripts

Sample of create N-Tier

● Security purpose● Permission control● Management purpose● Tiers

○ admin: VPN, management purpose

○ frontend: web server, for public connect

○ db: storing data, sensitive areahttp://gappsnews.blogspot.tw/search?q=n-tier

# service portgcutil addfirewall --allowed_tag_sources=frontend --network=my-network --allowed=tcp:80,tcp:443 myfw-service-port

# ap to dbgcutil addfirewall --allowed_tag_sources=frontend --target_tags=db --network=my-network --allowed=tcp:5984 myfw-couchdb-port

# admin zonegcutil addfirewall --allowed_ip_sources=0.0.0.0/8 --network=my-network --allowed=tcp:22 myfw-admin-ssh

gcutil addfirewall --allowed_tag_sources=admin --target_tags=frontend,db --network=my-network --allowed=tcp:22 myfw-manage-zone

Sample of create N-Tier - Network ACLs

Sample of create N-Tier - VPN & Web servergcutil --project="my-project" addinstance "my-gateway" \

--tags="admin" --zone="us-central1-b" --machine_type="g1-small" \

--network="my-network" --external_ip_address="ephemeral" \

--can_ip_forward="true" \

--image="https://www.googleapis.com/compute/v1/projects/.../global/images/..." \

--persistent_boot_disk="true"

gcutil --project="my-project" addinstance "my-web-01" \

--tags="frontend" --zone="us-central1-b" --machine_type="n1-standard-1" \

--network="my-network" --external_ip_address="ephemeral" \

--can_ip_forward="true" \

--image="https://www.googleapis.com/compute/v1/projects/.../global/images/..." \

--persistent_boot_disk="true"

Instance option - Start Script$ cat -> install-couchdb.sh << EOF

sudo apt-get update -y

sudo apt-get install gcc openssl couchdb -y

EOF

$ gcutil --service_version="v1" \

--project="my-project" addinstance "my-couchdb-01" \

--tags="db" --zone="us-central1-b" --machine_type="n1-highmem-2" \

--network="my-network" --external_ip_address="ephemeral" \

--can_ip_forward="true" \

--image="https://www.googleapis.com/compute/v1/projects/centos-cloud/global/images/centos-6-v20131120" \

--persistent_boot_disk="true"

--metadata_from_file=startup-script:install-couchdb.sh

Share your project

● Is Owner: resource management, project permission● Can Edit: resource management● Can View: resource view

● Add persistent disk● Create image● Bring your own kernel (brief)

Advance operations

Add a Persistent Disk...

➔ gcutil adddisk --zone=us-central1-a testdisk➔ gcutil ssh [instance name]➔ sudo mkdir /mnt/pd0➔ sudo /usr/share/google/safe_format_and_mount \

-m "mkfs.ext4 -F" /dev/disk/by-id/[disk-id] /mnt/pd0

Create a Image...

➔ sudo gcimagebundle -d /dev/sda -o /tmp/ \ --log_file=/tmp/abc.log

➔ gsutil cp /tmp/308...439.image.tar.gz \ gs://arecord-customise-images

➔ gcutil addimage test-image \ gs://arecord-customise-images/308...439.image.tar.gz

Porting recommendation

● Install LAMPsudo yum -y install httpd php php-mysql mysql mysql-serversudo yum install php-mysql php-gd libjpeg* php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc php-mbstring php-mcrypt php-bcmath php-mhash libmcrypt

● FW configure (GCE default enabled the iptables)sudo vi /etc/sysconfig/iptables⇒ Add your port… like 80, 443...

● SELinux setting (GCE default enable the SELinux)sudo vi /etc/sysconfig/selinux⇒ SELINUX=disabled

● Setup boot level servicessudo chkconfig --level 23456 mysqld onsudo chkconfig --level 23456 httpd on

Porting recommendation

● Mount persistence disk when boot$ sudo vi /etc/fstabUUID=a8cf...aaf98 / ext4 defaults,barrier=0 1 1

tmpfs /dev/shm tmpfs defaults 0 0devpts /dev/pts devpts gid=5,mode=620 0 0sysfs /sys sysfs defaults 0 0

proc /proc proc defaults 0 0

/dev/sdb /mnt/pd0 ext4 defaults 1 1

Bring Your Own Image

● Any common Linux distro● Must support some specific kernel settings (e.g.,

specific PCI and ISA bridge, vCPU settings, SCSI settings)

● Must have Python 2.6 or higher & sshd● Must contain some Google packages (startup script

support, google-daemon, gcimagebundle)● Should have other settings configured (e.g. DHCP,

SSH, firewall)