Container communication on lattice #2
-
Upload
kenta-shinohara -
Category
Technology
-
view
551 -
download
2
Transcript of Container communication on lattice #2
Copyright@2016 NTT corp. All Rights Reserved.
2016/2/10第31回PaaS勉強会@sinohara
Container Communication on lattice #2
Copyright@2016 NTT corp. All Rights Reserved.
⾃⼰紹介
@sinoharaNTTでNWの研究開発してます
PaaSたのしい
第28回PaaS勉強会でもLTさせてもらいました!
Copyright@2016 NTT corp. All Rights Reserved.
Copyright@2016 NTT corp. All Rights Reserved.
What is lattice?
• コンテナ管理システム(ルーティング、HM、ロギングetc)• Pivotal社のOSS• Cloud Foundryのdocker-image特化版
Copyright@2016 NTT corp. All Rights Reserved.
今⽇のトピック
latticeでコンテナ間通信
• コンテナ間の通信は未サポート• Dockerはコンテナ間通信の仕組みが沢⼭あるが・・
→latticeはdockerではなくgarden-linux上で動く
Copyright@2016 NTT corp. All Rights Reserved.
(前回)• 同⼀Cell(VM)内でコンテナ間通信は成功• Cell跨りは間に合わず・・
Container #1 Container #2 Container #3 Container #4
LATTICE ARCHITECTURE
Cell-1Cell-0
eth010.0.1.21
w0m6uagc6vub-0
w0m6uagc6vub-110.254.0.1
w0m6uagc6vuc-0
w0m6uagc6vuc-110.254.0.5
wb-0m6uf716cb1010.254.0.2
w0m6uf716lku-0
w0m6uf716lku-110.254.0.1
wb-0m6uf716cb2010.254.0.6
w0m6uf716lkv-0
w0m6uf716lkv-110.254.0.5
Eth010.0.1.193
54.92.69.17854.92.69.172
Region = ap-northeast-1
Virtual Machine Container Network Interfacedescription Network Tunnel
wb-0m6uagc6m51010.254.0.2
wb-0m6uagc6m50010.254.0.6
kshino/lattice-php-cluster kshino/lattice-php-cluster kshino/lattice-php-cluster kshino/lattice-php-cluster
V0.2.5
Cell-brain
54.92.69.42
HTTP_GET 54.92.69.42.xip.io/index.php?ip=10.254.0.5
(php) http_get(http://10.254.0.5/phpinfo.php)
疎通確認方法
Copyright@2016 NTT corp. All Rights Reserved.
今⽇のトピック
latticeでコンテナ間通信
• コンテナ間の通信は未サポート• Dockerはコンテナ間通信の仕組みが沢⼭あるが・・
→latticeはdockerではなくgarden-linux上で動く• Cell間やってきました!
Copyright@2016 NTT corp. All Rights Reserved.
⽅針
• L2 over L3でつなぐ
• Ubuntuと相性の良いCanonical Fanを使⽤
• latticeのソース読むのはしんどいので動いてるモノを触って出来る範囲でトライ
Copyright@2016 NTT corp. All Rights Reserved.
FanNetworking
•同⼀のClass B-NW内でOverlay + Tunneling•コンテナIPのネゴシエーションがVM内で完結•Overlayは/8占有。Cell毎に/24を割り当て•コンテナのIPの共有は別途必要•現在v0.3.0
172.16.0.0/16172.16.0.0/16
172.16.3.5
172.16.23.37
10.3.5.x/24
10.23.37.x/24
VM#1
VM#2
Overlay-NW
Class-B NW
https://wiki.ubuntu.com/FanNetworking
https://launchpad.net/ubuntu/+source/ubuntu-fanhttps://insights.ubuntu.com/2015/06/24/ubuntu-fan-images/
Overlay-NW
Copyright@2016 NTT corp. All Rights Reserved.
流れを追って説明します
Copyright@2016 NTT corp. All Rights Reserved.
Container #1 Container #2
LATTICE ARCHITECTURE
Cell-0 eth0(cell0)10.0.1.21
Veth-host#1
Veth-guest#110.254.0.1
Veth-host#2
Veth-guest#210.254.0.5
Region = ap-northeast-1
Virtual Machine Container Network Interfacedescription Network Tunnel
Vbr#110.254.0.2
Vbr#210.254.0.6
tutum/apache-php tutum/apache-php
lattice=V0.6.0
Fan
Container #1 Container #2
Cell-0 eth0(cell0)10.0.1.21
Veth-host#1
Veth-guest#110.254.0.1
Veth-host#2
Veth-guest#210.254.0.5
Vbr#110.254.0.2
Vbr#210.254.0.6
tutum/apache-php tutum/apache-php
10.0.1.0/24
Step 0. Lattice(初期状態)
Copyright@2016 NTT corp. All Rights Reserved.
Container #1 Container #2
Cell-0 eth0(cell0)10.0.1.21
Veth-host#1
Veth-guest#110.254.0.1
Veth-host#2
Veth-guest#210.254.0.5
Region = ap-northeast-1
Vbr#110.254.0.2
Vbr#210.254.0.6
tutum/apache-php tutum/apache-php
lattice=V0.6.0
fan-br#1250.1.21.1
tunl0250.1.21.1
Container #1 Container #2
Cell-0 eth0(cell0)10.0.1.21
Veth-host#1
Veth-guest#110.254.0.1
Veth-host#2
Veth-guest#210.254.0.5
Vbr#110.254.0.2
Vbr#210.254.0.6
tutum/apache-php tutum/apache-php
fan-br#1250.1.21.1
tunl0250.1.21.1
10.0.1.0/24
L2 over L3 tunnel250.0.0.0/8
Step 1. FanをCellにセットアップ
• カーネル更新(3.19.0-41)• ubuntu-fanのinstall• Fan⽤Overlay-NW作成(250/8)
LATTICE ARCHITECTUREVirtual Machine Container Network Interfacedescription Network Tunnel
Fan
Copyright@2016 NTT corp. All Rights Reserved.
Container #1 Container #2
Cell-0 eth0(cell0)10.0.1.21
Veth-host#1
Veth-guest#110.254.0.1
Veth-host#2
Veth-guest#210.254.0.5
Region = ap-northeast-1
Vbr#110.254.0.2
Vbr#210.254.0.6
tutum/apache-php tutum/apache-php
lattice=V0.6.0
fan-br#1250.1.21.1
fan-veth-h#1 fan-veth-h#2
fan-veth-g#1250.1.21.2
fan-veth-g#2250.1.21.3
tunl0250.1.21.1
Container #1 Container #2
Cell-0 eth0(cell0)10.0.1.21
Veth-host#1
Veth-guest#110.254.0.1
Veth-host#2
Veth-guest#210.254.0.5
Vbr#110.254.0.2
Vbr#210.254.0.6
tutum/apache-php tutum/apache-php
fan-br#1250.1.21.1
fan-veth-h#1 fan-veth-h#2
fan-veth-g#1250.1.21.2
fan-veth-g#2250.1.21.3
tunl0250.1.21.1
10.0.1.0/24
L2 over L3 tunnel250.0.0.0/8
Step 2. 仮想NIC作成
LATTICE ARCHITECTUREVirtual Machine Container Network Interfacedescription Network Tunnel
Fan
• netnsで仮想NICをコンテナ内に繋ぎ込み• コンテナ内のルーティングテーブルに250/8の設定追加
Copyright@2016 NTT corp. All Rights Reserved.
Container #1 Container #2
Cell-0 eth0(cell0)10.0.1.21
Veth-host#1
Veth-guest#110.254.0.1
Veth-host#2
Veth-guest#210.254.0.5
Region = ap-northeast-1
Vbr#110.254.0.2
Vbr#210.254.0.6
tutum/apache-php tutum/apache-php
lattice=V0.6.0
fan-br#1250.1.21.1
fan-veth-h#1 fan-veth-h#2
fan-veth-g#1250.1.21.2
fan-veth-g#2250.1.21.3
tunl0250.1.21.1
Container #1 Container #2
Cell-0 eth0(cell0)10.0.1.21
Veth-host#1
Veth-guest#110.254.0.1
Veth-host#2
Veth-guest#210.254.0.5
Vbr#110.254.0.2
Vbr#210.254.0.6
tutum/apache-php tutum/apache-php
fan-br#1250.1.21.1
fan-veth-h#1 fan-veth-h#2
fan-veth-g#1250.1.21.2
fan-veth-g#2250.1.21.3
tunl0250.1.21.1
10.0.1.0/24
L2 over L3 tunnel250.0.0.0/8
完成!• IPinIP protocol(4)→許容
LATTICE ARCHITECTUREVirtual Machine Container Network Interfacedescription Network Tunnel
Fan
Copyright@2016 NTT corp. All Rights Reserved.
DEMO
riak clusterを動かしてみた
Copyright@2016 NTT corp. All Rights Reserved.
Cell-0
DEMO概要
Router(lattice-brain)
Cell-1
250.0.0.0/8
10.0.0.0/8
sinatra
riak
sinatra
riakclustering
• 250/8でriak cluster構築•外から250/8に直接アクセス不可→Sinatraでインタワークし、latticeのエンドポイントからriak clusterを直接操作
http://52.1.2.3.xip.io:8098/
http://250.1.2.2:8098http://250.1.3.4:8098 InterWork InterWork
tcp-route 8098:8080tcp-route 8098:8080
Copyright@2016 NTT corp. All Rights Reserved.
まとめ
Lattice上でのコンテナ間通信(L2 over L3)の紹介• PoCレベル• netnsとかCellでの作業があります• /8のprivate address(10/8)をlatticeが占有し
ちゃってる問題(IANA未使⽤の250/8で代替)riak cluster on latticeのデモ
Latticeでクラスタリング、あなたなら何に使いますか?