whchang@bora.net ISP를 위한 초고속망 구축기술 1

ISP 를 위한 초고속망 구축 기술

장우현whchang@bora.net

데이콤 EC 인터넷사업본부 보라넷팀

whchang@bora.net ISP를 위한 초고속망 구축기술 2

Topics

• LAN

• WAN

• Server System

• Traffic Caching* Access Network (home-to-ISP), Monitoring, Accounting 은

다루지 않았음

whchang@bora.net ISP를 위한 초고속망 구축기술 3

LAN parts

• Bandwidth 의 변화– Shared 10Mbps– Switched 10Mbps– Shared 100Mbps– Switched 100Mbps– Switched 2Nx100Mbps– Switched 1Gbps– Shared N x Gbps

whchang@bora.net ISP를 위한 초고속망 구축기술 4

BORANet Router Port 의 Bandwidth Usage 변화

Bandwidth of a Router port

0

100

200

300

400

500

600

94 95 96 97 98 99

Year

Mb

ps

whchang@bora.net ISP를 위한 초고속망 구축기술 5

High Speed LAN Solutions

• 100Mbps FDDI

• 100Mbps FastEthernet

• ATM LAN Emulation

• FastEtherChannelFastEtherChannel– 2^N x FastEthernet

• Gigabit Ethernet InterfaceGigabit Ethernet Interface

• IP Ring (Gigabit)IP Ring (Gigabit)

whchang@bora.net ISP를 위한 초고속망 구축기술 6

FastEtherChannel

• 2^N x FastEthernet (N = 1,2,3)– Router <-> Switch– Switch <-> Switch– Logical One Interface– 여러업체에서 지원

• MAE-WEST, PAIX, DACOM BORANetMAE-WEST, PAIX, DACOM BORANet 등에서 이용

800M 400M200M

whchang@bora.net ISP를 위한 초고속망 구축기술 7

Gigabit Ethernet Interface

• Most of the famous vendors are providing the Gigabit Interface– ‘98 년중부터 출시됨– Cisco, 3Com, ....– Alteon, Foundry, ...

• FastEtherChannel 보다 성능 , 확장성 , 비용면에서 유리 – FasEtherChannel 은 Gigabit Interface 로 가기

위한 Interim Solution– 가능하다면 100M Interface 에서 Gigabit InterfacGigabit Interfac

ee 로 바로 전이하는 것이 바람직로 바로 전이하는 것이 바람직

whchang@bora.net ISP를 위한 초고속망 구축기술 8

IP Ring• Shared Gigabit Ring• (Wide AreaWide Area) Local Area Network

– Same Network Address• based on SONET/SDH• FDDI 100Mbps 한계를 벗어남 • IP Ring will come to us Soon!

서울 수원

안양

Gigabit Ringbased on SONET/SDH

10.0.0.1 10.0.0.3

10.0.0.2

whchang@bora.net ISP를 위한 초고속망 구축기술 9

WAN Parts

• Bandwidth 의 변화– 9.6/56/64Kbps– Fractional T1/E1– T1/E1– T3– OC3/STM1 (155M)– OC12/STM4 (620M)– OC48/STM16 (2.48G)

whchang@bora.net ISP를 위한 초고속망 구축기술 10

DACOM <-> KT 간 Bandwidth 변화

Bandwidth between DACOM and KT

0

50

100

150

200

250

300

94 95 96 97 98 99Year

Mb

ps

whchang@bora.net ISP를 위한 초고속망 구축기술 11

High Speed WAN Solutions

• ATM

• Packet Over SONET/SDH

whchang@bora.net ISP를 위한 초고속망 구축기술 12

POS (Packet Over SONET/SDH)

• A high-speed WAN transport that leaves LAN traffic in its native formatnative format

• Packet-Over-SONET/SDH is the serial transmission of data over SONET frames through the use of Point-to-Point Protocol (PPP)– RFC 1619, PPP over SONET/SDH– RFC 1662, PPP in HDLC-like Framing

whchang@bora.net ISP를 위한 초고속망 구축기술 13

Benefit of POS

• High utilization of WAN bandwidth– Packet over SONET provides 98% utilization98% utilization

EtherEtherSizeSize

(Byte)(Byte)

SPESPEEff.Eff.%%

SPESPEEff.Eff.%%

POSPOS ATMATM

6464

128128

256256

512512

10241024

15181518

20482048

43524352

44884488

86.886.8

9494

9797

98.698.6

99.399.3

99.599.5

99.699.6

99.899.8

99.899.8

4343

6969

7575

8585

8686

8888

8989

8989

8989

whchang@bora.net ISP를 위한 초고속망 구축기술 14

Which One? POS or ATM

• Consider IP over SONET/SDH when business model or majority of traffic is IP CentricIP Centric

• Consider ATM for aggregation of multiple servicesmultiple services on a common transmission infrastructure

whchang@bora.net ISP를 위한 초고속망 구축기술 15

BORANet Backbone

• T3 now

• We are considering the POS for the national and int’l backbone

• DACOM BORANet <-> KT KORNET – T3 x 3 now-> POS

1 x STM1STM12 x STM11 x STM4

whchang@bora.net ISP를 위한 초고속망 구축기술 16

Server System Parts

• One Big Machine: No way– Use Multiple MachineMultiple Machine– to serve the large and bursty traffic

• Server RackServer Rack

DNS Servers Web ServersVideo Servers

whchang@bora.net ISP를 위한 초고속망 구축기술 17

Share the Disk Storage• Dedicated NFS Server SystemDedicated NFS Server System 이용

– Hot Swappable Disk without Setup– High Speed I/O Operations– Multiple Fast Interfaces

whchang@bora.net ISP를 위한 초고속망 구축기술 18

Server Load 분산

• DNS Round-Robin 이용한 load 분산

• Layer 4 SwitchLayer 4 Switch 이용한 load 분산– Application port number 를 보고 분산시킴– DNS, Web, Telnet, FTP, NNTP, etc.

ip: 10.0.0.1port: 23

10.0.0.2session 10 개

10.0.0.3session 10 개

10.0.0.4session 9 개Layer 2 Switch

Layer 4 Switch

whchang@bora.net ISP를 위한 초고속망 구축기술 19

Off-the-Topic

• Don’t consider only SUN, HP, DEC, ....– They are expensive.

• We can buy the systems having same performance and quality with SUN, HP, DEC at half-price

whchang@bora.net ISP를 위한 초고속망 구축기술 20

Traffic Caching

• Web Traffic Caching– save the cost for WAN connection– improve the performance at the user-view– upto 50% hit ratio– Most of Korea ISPs are caching the traffic to sav

e the int’l circuit– 투자비를 2~3 개월안에 회수 가능

• DNS Request Caching– improve the response time

whchang@bora.net ISP를 위한 초고속망 구축기술 21

Traffic Caching (cont.)

• Many ISPs in the world are installing the cache-farms at their POPs.

SanFrancsico New York

Dallas

cache-farm

cache-farm

cache-farm

whchang@bora.net ISP를 위한 초고속망 구축기술 22

Traffic Caching (cont.)

• Web Caching Servers– Inktomi, Netcache, Cisco Cache, CacheFlow, ...

• L4 Switches– Alteon, Foundry, RAD, ...– Gigabit Interface 지원– Layer2 Switch 로 기능으로도 뛰어남

whchang@bora.net ISP를 위한 초고속망 구축기술 23

Challenges of IP Network

• Accounting

• Class of Services

• Quality of Services

• High-Speed Switching

• VPN provided by the service provider’s network