[OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치...
-
Upload
openstack-korea-community -
Category
Technology
-
view
16.674 -
download
0
Transcript of [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치...
![Page 1: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/1.jpg)
OpenNSL으로 브로드콤 기반 네트워크 스위치 제어하기
OpenStack Day Korea 2016
NIC 기술원
SDI Tech. Lab
Manager 서준호2016. 2. 18
![Page 2: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/2.jpg)
1
Contents
본 발표는 지난 한달 간 Facebook이 제공한 Wedge를 가지고 이리 저리
가지고 논 경험을 담고 있음
1. OCP 소개
2. Facebook Wedge H/W 소개
3. Networking Operating System – Open Network Linux (ONL)
4. Broadcom Trident2 & OpenNSL 소개
5. OpenNSL 튜토리얼 (L2 스위칭, L3 라우팅)
![Page 3: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/3.jpg)
2
OCP 소개
OCP (Open Compute Project)
§ Facebook 주도의 데이터센터 인프라 설계 기술 공유를 위한 오픈
하드웨어 프로젝트
§Facebook 은 Oregon 주 Prineville 데이터센터 인프라 스펙을 공개
§IT 기술을 접목시켜 경제적이고 효율적 관리를 통해 $20억 절감
§ 프로젝트
§Server/Storage/Networking/Open Rack/Data Center/Management
§ SKT 는 올해 2월 가입
§OCP Summit 2016 (3월) 에서 발표 예정
![Page 4: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/4.jpg)
3
OCP Networking 소개
Project Chairs: Omar Baldonado | Carlos Cardenas
네트워크 하드웨어 + 네트워킹 소프트웨어
§ Disaggregation + Open
§ 현재는 ToR 스위치와 Spine 스위치 스펙 정의에 집중하고 있음
§ 또한 각 칩 벤더간의 서로 다른 Hardware Abstract Layer (HAL)
영역을 하나의 공통된 인터페이스 Service Abstract Interface (SAI)
로 스펙화 진행 중
![Page 5: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/5.jpg)
4
Facebook이 개발한 1RU(Rack Unit) 크기의 ToR Switch
Facebook Wedge 소개 (1/3)
Modular Design
§ Micro-server module
§Intel Avoton C2550 (Atom)
§ Main switch module
§16 QSFP
§4x Fan module
§ BCM56850 (Trident2)
§ Open BMC module
Spine Switch
![Page 6: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/6.jpg)
5
Facebook Wedge 소개 (2/3)
High Level Block Diagram
기존서버 아키텍처와다를게없음!
![Page 7: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/7.jpg)
6
High Speed Interface Connectivity§ 16x QSFP (each x4 SerDes 10GbE)§ 16x (not used), but for 6-Pack
Facebook Wedge 소개 (3/3)
Main Switching ASIC - BCM56850 (Trident2)
2U Yosemite
Wedge
16 x Yosemite
![Page 8: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/8.jpg)
7
Power on & Boot up Wedge
ONIE: Boot Loader contributed by CumulusNetworks§ GRUB2 (x86) or U-Boot (powerpc or other
architectures)§ Reinstall/Uninstall/Rescue/Boot NOSes on
OCP bare metal switches over IPv6 and HTTP
![Page 9: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/9.jpg)
8
Installing Networking Operating System
Switch Light
Current ONIE compatibleNOS List
We choose Open Network Linux (ONL)
§ OCP 커뮤니티에서 베어 메탈 스위치를 위한 레퍼런스 네트워크 운영체제§ Big Switch가 OCP 커뮤니티 네트워킹 그룹에 오픈소스 기여§ 데비안 리눅스 기반 + 네트워크 ASIC 칩 드라이버 + ɑ (vendor specific)§ 현재 Big Switch Networks, 페이스북, NTT 사용 중§ 현재 베어 메탈 스위치 제조 벤더들이 지원 à 앞으로도 계속 ONL이
지원되는 OCP 표준 스위치 증가할 예정 (OCP summit `15)
![Page 10: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/10.jpg)
9
Installing Networking Operating System
![Page 11: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/11.jpg)
10
네트워크 스위치가 되기 위해서는…
지금까지 한 일
네트워크 스위치가 되기위해서 할 일
Wedge
![Page 12: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/12.jpg)
11
Facebook - the next-generation Facebook data center network
Wedge (ToR 스위치) 6-pack (Spine 스위치)
Clos networks
FBOSS
Facebook ONL Use Case
![Page 13: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/13.jpg)
12
Use Cases: NTT
NTT’s EVPN
NTT ONL Use Case
![Page 14: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/14.jpg)
13
OpenNSL 소개
Broadcom Switching ASIC (i.e., Trident+, Trident2) Chip 제어를 위한오픈소스 기반 Hardware Abstraction Layer (HAL)
Licence Policy§ OEM/ODM Development Package§ Community Development PackageFeatures§ L2/L3/VLAN management§ Port and switch management§ Link monitoring§ Packet transmit and receive§ Trunking§ …Supported Platforms§ Acction Wedge, AS5712§ Broadcom reference platform (SVK)
![Page 15: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/15.jpg)
14
OpenNSL 소개
Broadcom Switching ASIC (i.e., Trident+, Trident2) Chip 제어를 위한오픈소스 기반 Hardware Abstraction Layer (HAL)
OpenNSL Use Cases§ FBOSS (Facebook 주도)§ Open Network Linux (Big Switch 주도)§ BroadView§ Open L3 Routing (NTT 주도)§ OpenSwitch (HP주도)
![Page 16: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/16.jpg)
15
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter(Packet Header Parser)
Packet Processing Pipeline
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
High Level Block Diagram of Broadcom ASIC
OpenNSL 소개
PCI-
E
Kernel SpaceUser Space
OpenNSL
Broadcom SDKKernel BDE
Kernel KNET
DM
A
eth0
eth1
eth2 SRAM
TCAM
DRAM
![Page 17: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/17.jpg)
16
Example – L2 Switching
Basic Three Operations of L2 Switch§ Address learning
§Src MAC matching§Automatically learning
§ L2 Forwarding§ Address ageing
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter(Packet Header Parser)
Address Resolution Logic(ARL)
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
ARL Table
Memory
MAC | VLAN | PORT | TIME
opennsl_l2_addr_add(mac, vlan, port, flags);
flags: OPENNSL_L2_DISCARD_SRCOPENNSL_L2_DISCARD_DSTOPENNSL_L2_L3LOOKUPOPENNSL_L2_MOVE_PORTOPENNSL_L2_COPY_TO_CPUCALLBACK…
opennsl_l2_addr_add()opennsl_l2_addr_delete()opennsl_l2_addr_get()opennsl_l2_addr_register()opennsl_l2_addr_t_init()
![Page 18: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/18.jpg)
17
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter(Packet Header Parser)
ARL
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
Basic Three Operations of L2 Switch§ Address learning§ L2 Forwarding
§Dst MAC matching§Default - Flooding
§ Address ageing
Example – L2 Switching
Forwarding Logic
Forwarding Table
Memory
MAC | MAC_MASK
opennsl_l2_station_add(mac, mask, flags);
flags: OPENNSL_L2_STATION_ARP_RARPOPENNSL_L2_STATION_IPV4
opennsl_l2_station_add()opennsl_l2_station_delete()opennsl_l2_station_get()opennsl_l2_station_t_init()
![Page 19: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/19.jpg)
18
Example – L3 Routing
L3 Routing§ Routing between different sub-networks§ Longest Prefix Matching (LPM)
192.168.1.0/24
192.168.2.0/24
192.168.3.0/24
Subnet 1 Subnet 2 Subnet 3
192.168.1.1
192.168.2.1
192.168.3.1
![Page 20: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/20.jpg)
19
Example – L3 Routing
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter(Packet Header Parser)
L2 Logic
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
L3 Logic
L3 Routing1. Set L3 egress mode2. Create IP interfaces3. Make the address learn on a VLAN and port4. Create egress object5. Add host point6. Add default route
opennsl_switch_control_set(opennslSwitchL3EgressMode);
![Page 21: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/21.jpg)
20
Example – L3 Routing
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter(Packet Header Parser)
L2 Logic
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
L3 Logic
L3 Routing1. Set L3 egress mode2. Create Ingress/Egress IP interfaces3. Make the address learn on a VLAN and port4. Create egress object5. Add host point6. Add default route
opennsl_l3_intf_create(vlan, mac, in_port);opennsl_l3_intf_create(vlan, mac, out_port);
![Page 22: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/22.jpg)
21
Example – L3 Routing
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter(Packet Header Parser)
L2 Logic
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
L3 Logic
L3 Routing1. Set L3 egress mode2. Create Ingress/Egress IP interfaces3. Make the address learn on a VLAN and port4. Create egress object5. Add host point6. Add default route
opennsl_l2_addr_addr(vlan, in_port, mac);opennsl_l2_addr_addr(vlan, out_port, mac);flags |= OPENNSL_L2_L3LOOKUP
ARL Table
Memory
MAC | VLAN | PORT | TIME
![Page 23: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/23.jpg)
22
Example – L3 Routing
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter(Packet Header Parser)
L2 Logic
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
L3 Logic
L3 Routing1. Set L3 egress mode2. Create Ingress/Egress IP interfaces3. Make the address learn on a VLAN and port4. Create egress object5. Add host point6. Add default route
opennsl_l3_egress_create(vlan, out_port, out_intf, nexthop_mac);
L3 Forwarding Table
out_inf | nexthop_mac
![Page 24: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/24.jpg)
23
Example – L3 Routing
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter(Packet Header Parser)
L2 Logic
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
L3 Logic
L3 Routing1. Set L3 egress mode2. Create Ingress/Egress IP interfaces3. Make the address learn on a VLAN and port4. Create egress object5. Add host point6. Add default route
opennsl_l3_host_add(host_ip)
L3 Forwarding Table
host_ip | out_inf | nexthop_mac
![Page 25: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/25.jpg)
24
Example – L3 Routing
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter(Packet Header Parser)
L2 Logic
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
L3 Logic
L3 Routing1. Set L3 egress mode2. Create Ingress/Egress IP interfaces3. Make the address learn on a VLAN and port4. Create egress object5. Add host point6. Add default route
opennsl_l3_route_add(default_ip, out_inf)
L3 Forwarding Table
host_ip | out_inf | nexthop_mac
default_ip | out_inf | nexthop_mac
![Page 26: [OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기](https://reader031.fdocuments.net/reader031/viewer/2022020119/587004e71a28ab427f8b5cf9/html5/thumbnails/26.jpg)
25
Thanks
감사합니다.