Team Multiple Network Interfaces (NIC) Into a Single Interface

33
About Forum Howtos & FAQs Low graphics Shell Scripts RSS/Feed nixcraft - insight into linux admin work by LinuxTitli on April 3, 2006 · 96 comments Finally, today I had implemented NIC bounding (bind both NIC so that it works as a single device). Bonding is nothing but Linux kernel feature that allows to aggregate multiple like interfaces (such as eth0, eth1) into a single virtual link such as bond0. The idea is pretty simple get higher data rates and as well as link failover. The following instructions were tested on: RHEL v4 / 5 / 6 amd64 1. CentOS v5 / 6 amd64 2. Fedora Linux 13 amd64 and up. 3. 2 x PCI-e Gigabit Ethernet NICs with Jumbo Frames (MTU 9000) 4. Hardware RAID-10 w/ SAS 15k enterprise grade hard disks. 5. Gigabit switch with Jumbo Frame 6. This server act as an heavy duty ftp, and nfs file server. Each, night a perl script will transfer lots of data from this box to a backup server. Therefore, the network would be setup on a switch using dual network cards. I am using Red Hat enterprise Linux version 4.0. But, the inductions should work on RHEL 5 and 6 too. Linux allows binding of multiple network interfaces into a single channel/NIC using special kernel module called bonding. According to official bonding documentation: RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in... 1 of 33 11/15/2011 5:15 PM

Transcript of Team Multiple Network Interfaces (NIC) Into a Single Interface

Page 1: Team Multiple Network Interfaces (NIC) Into a Single Interface

AboutForumHowtos & FAQsLow graphicsShell ScriptsRSS/Feed

nixcraft - insight into linux admin work

by LinuxTitli on April 3, 2006 · 96 comments

Finally, today I had implemented NIC bounding (bind both NIC so that it works as a single device). Bonding is nothing but Linux kernelfeature that allows to aggregate multiple like interfaces (such as eth0, eth1) into a single virtual link such as bond0. The idea is prettysimple get higher data rates and as well as link failover. The following instructions were tested on:

RHEL v4 / 5 / 6 amd641.CentOS v5 / 6 amd642.Fedora Linux 13 amd64 and up.3.2 x PCI-e Gigabit Ethernet NICs with Jumbo Frames (MTU 9000)4.Hardware RAID-10 w/ SAS 15k enterprise grade hard disks.5.Gigabit switch with Jumbo Frame6.

This server act as an heavy duty ftp, and nfs file server. Each, night a perl script will transfer lots of data from this box to abackup server. Therefore, the network would be setup on a switch using dual network cards. I am using Red Hat enterpriseLinux version 4.0. But, the inductions should work on RHEL 5 and 6 too.

Linux allows binding of multiple network interfaces into a single channel/NIC using special kernel module called bonding. According to officialbonding documentation:

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

1 of 33 11/15/2011 5:15 PM

Page 2: Team Multiple Network Interfaces (NIC) Into a Single Interface

The Linux bonding driver provides a method for aggregating multiple network interfaces into a single logical "bonded" interface. Thebehavior of the bonded interfaces depends upon the mode; generally speaking, modes provide either hot standby or load balancingservices. Additionally, link integrity monitoring may be performed.

Red Hat Enterprise Linux (and its clone such as CentOS) stores network configuration in /etc/sysconfig/network-scripts/ directory. First, you needto create a bond0 config file as follows:# vi /etc/sysconfig/network-scripts/ifcfg-bond0

Append the following linest:

DEVICE=bond0IPADDR=192.168.1.20NETWORK=192.168.1.0NETMASK=255.255.255.0USERCTL=noBOOTPROTO=noneONBOOT=yes

You need to replace IP address with your actual setup. Save and close the file.

Open both configuration using a text editor such as vi/vim, and make sure file read as follows for eth0 interface# vi /etc/sysconfig/network-scripts/ifcfg-eth0

Modify/append directive as follows:DEVICE=eth0

USERCTL=no

ONBOOT=yes

MASTER=bond0

SLAVE=yes

BOOTPROTO=none

Open eth1 configuration file using vi text editor, enter:# vi /etc/sysconfig/network-scripts/ifcfg-eth1

Make sure file read as follows for eth1 interface:

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

2 of 33 11/15/2011 5:15 PM

Page 3: Team Multiple Network Interfaces (NIC) Into a Single Interface

DEVICE=eth1

USERCTL=no

ONBOOT=yes

MASTER=bond0

SLAVE=yes

BOOTPROTO=none

Save and close the file.

Make sure bonding module is loaded when the channel-bonding interface (bond0) is brought up. You need to modify kernel modules configurationfile:# vi /etc/modprobe.confAppend following two lines:alias bond0 bondingoptions bond0 mode=balance-alb miimon=100

Save file and exit to shell prompt. You can learn more about all bounding options by clicking here).

Step # 4: Test configuration

First, load the bonding module, enter:# modprobe bonding

Restart the networking service in order to bring up bond0 interface, enter:# service network restart

Make sure everything is working. Type the following cat command to query the current status of Linux kernel bounding driver, enter:# cat /proc/net/bonding/bond0

Sample outputs:

Bonding Mode: load balancing (round-robin)MII Status: upMII Polling Interval (ms): 100Up Delay (ms): 200Down Delay (ms): 200Slave Interface: eth0MII Status: upLink Failure Count: 0Permanent HW addr: 00:0c:29:c6:be:59Slave Interface: eth1MII Status: upLink Failure Count: 0

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

3 of 33 11/15/2011 5:15 PM

Page 4: Team Multiple Network Interfaces (NIC) Into a Single Interface

Permanent HW addr: 00:0c:29:c6:be:63

To kist all network interfaces, enter:# ifconfig

Sample outputs:

bond0 Link encap:Ethernet HWaddr 00:0C:29:C6:BE:59 inet addr:192.168.1.20 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:2804 errors:0 dropped:0 overruns:0 frame:0 TX packets:1879 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:250825 (244.9 KiB) TX bytes:244683 (238.9 KiB)eth0 Link encap:Ethernet HWaddr 00:0C:29:C6:BE:59 inet addr:192.168.1.20 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fec6:be59/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:2809 errors:0 dropped:0 overruns:0 frame:0 TX packets:1390 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:251161 (245.2 KiB) TX bytes:180289 (176.0 KiB) Interrupt:11 Base address:0x1400eth1 Link encap:Ethernet HWaddr 00:0C:29:C6:BE:59 inet addr:192.168.1.20 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fec6:be59/64 Scope:Link UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:4 errors:0 dropped:0 overruns:0 frame:0 TX packets:502 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:258 (258.0 b) TX bytes:66516 (64.9 KiB) Interrupt:10 Base address:0x1480

Read the official bounding howto which covers the following additional topics:

VLAN ConfigurationCisco switch related configurationAdvanced routing and troubleshooting

This blog post is 1 of 2 in the "Linux NIC Interface Bonding (aggregate multiple links) Tutorial" series. Keep reading the rest of the series:

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

4 of 33 11/15/2011 5:15 PM

Page 5: Team Multiple Network Interfaces (NIC) Into a Single Interface

Table of Contents:

Red Hat (RHEL/CentOS) Linux Bond or Team Multiple Network Interfaces (NIC) into a Single Interface1.Debian / Ubuntu Linux Configure Bonding [ Teaming / Aggregating NIC ]2.

Share this with other sys admins!Download it - Email it - Facebook it - Google+ it - Print it - Tweet it

We're here to help you make the most of sysadmin work. So, subscribe!

{ 96 comments… read them below or add one }

1 Anonymous June 6, 2006

Many thanks for quick instructions. It worked like a charm.

Just one more hint:Enable spanning tree for switches. I had noticed the network speed reduction when spanning tree was not enabled.

Just my two 2 cents

Reply

2 Anonymous September 5, 2006

Thanks for the quick help!! bonding r0x!! Just make sure that eth0 and eth1 are up when you put up the bond0. And be careful if you’redoing this over a remote ssh. *thumbs up*

Reply

3 mattcosa November 30, 2006

Did you notice that your /proc/net/bonding/bond0 was reporting incorrect settings based on your mode and other settings?

I’m having the same problem.

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

5 of 33 11/15/2011 5:15 PM

Page 6: Team Multiple Network Interfaces (NIC) Into a Single Interface

Reply

4 zen January 30, 2007

hmm. did anyone experience what i did?

when i do a ifdown eth0, the ping timed out. it doesn’t seem to be bonding well.

Reply

5 Gaurav February 14, 2007

What will be the behaviour if eth0 and eth1 interfaces does not have IP addresses and they share the IP address of bond0 in a failoverscenario.Will there be any entey for eth0 and eth1 in /proc/net/route????

- gaurav

Reply

6 H8injected March 15, 2007

bond0 config file needed another line in my configuration:GATEWAY=192.168.0.1

Reply

7 seno April 15, 2007

Can you bond more than 2 nics? e.g 4 NICs?

Reply

8 Seno April 15, 2007

Reason why I ask is because I want a big fat trunk of say 4x1Gig Nic’s and then run many virtulized O/S that runs it’s connection virtuallythough bond0.

Reply

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

6 of 33 11/15/2011 5:15 PM

Page 7: Team Multiple Network Interfaces (NIC) Into a Single Interface

9 nixcraft April 15, 2007

Seno,

How many bonding devices can I have?

There is no limit.

How many slaves can a bonding device have? OR Can you bond more than 2 nics?This is limited only by the number of network interfaces (NIC) Linux supports and/or the number of network cards you can place in yoursystem.

HTH

Reply

10 Please May 10, 2007

HI

i have 2 uplink and 1 internal link.i want to use 2 uplink at the same time.i installed core 5.so how can i do?my informationethoip-10.241.215.163nesk-255.255.255.240GW-10.241.215.161

eth2ip-10.255.8.104nesk-255.255.255.252.0GW-10.255.8.1

eth1ip-192.168.1.1nesk-255.255.255.0

Thanks

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

7 of 33 11/15/2011 5:15 PM

Page 8: Team Multiple Network Interfaces (NIC) Into a Single Interface

Please

Reply

11 Andres June 4, 2007

Thanks!It works very well…Just add this :GATEWAY=192.168.1.1 (replace with your gateway)

To the bond config file

thanks!

Reply

12 Rizwan August 29, 2007

I have 4 DSL modems connected with 4 different ISP’s my scenario isa) My FC-2 machine with LAN IP=192.168.10.1 and Bond0 IP=192.168.1.1b) Modem-A LAN IP= 192.168.1.2 , ext IP=xxx.xxx.xxx.xxxc) Modem-B LAN IP=192.168.1.3, ext IP=xxx.xxx.xxx.xxxd) Modem-C LAN IP=192.168.1.4, ext IP=xxx.xxx.xxx.xxxe) Modem-D LAN IP=192.168.1.5, ext IP=xxx.xxx.xxx.xxx

Modem-A, B, C, and D LAN connected with my FC-2 machine, and all 4 interfaces of my machine are in Bond0,Now please help me what default Gateway I should set in my FC-2 machine?>??? or I have to set 4 gateways in my machine??and will this configuration works??my scenario will also listed in my home page, please visit Linux section for better understanding.

Thanks in advance.Rizwan.Karachi, Pakistan.homepage: http://freewaresolutions.150m.com

Reply

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

8 of 33 11/15/2011 5:15 PM

Page 9: Team Multiple Network Interfaces (NIC) Into a Single Interface

13 David Kittell September 21, 2007

Great tutorial however on FC7 and likely other distributions you have to manually assign an IP to the bond0. Not at all a big deal though

Great work.

Reply

14 David Kittell September 21, 2007

What I mean by that is that after reboot on FC7 it appears you have to reassign the bond0 causing you to do the “vi /etc/sysconfig/network-scripts/ifcfg-bond0″ part again and yes I did :wq! :-)

May just be an issue with my build.

Again great work

Reply

15 shyam December 4, 2007

I am using this configuration on centos 5.0 and when i do service network restart i am getting warning hardware address conflicts with eth0change hardware address.

Please suggest me on the same.

Thanks in advanceShyam

Reply

16 nisha December 19, 2007

Perfect guide! My NIC bond went without a single hitch! Thanks and keep the guides coming… ;-)

Reply

17 Kyle February 5, 2008

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

9 of 33 11/15/2011 5:15 PM

Page 10: Team Multiple Network Interfaces (NIC) Into a Single Interface

I did this and it worked on my FC4 machine, but it doesn’t show eth1 in the ifconfig. I am not sure if it is bonded or not. Can you help mewith this problem?

Reply

18 naim February 26, 2008

I did this with FC8, I’m newbie and i am confused that how can i bond two dsl modem together is this the way or this is some thing else? orthis is only bonding two nic for heavy traffic/load balancing in LAN?

Reply

19 p March 31, 2008

This is only for bonding two nics onto the same network, to provide resilience or extra throughput. It’s no use if you want to balance acrossmultiple networks.

Reply

20 krishna April 27, 2008

how to enable MII Status for eth0. Although i have done all the above steps, but its creating problem.

/proc/net/bonding/bond0, snippet :

Slave Interface: eth0MII Status: downLink Failure Count: 0Permanent HW addr: 00:1b:a0:26:cb:78

Reply

21 E71 May 23, 2008

Does anyone know if /proc/net/bonding/bond0 will display ‘MII status: up’ ONLY when your NICs fully support Link Aggregation?

It’s showing as UP for everything and ifconfig produces something similar to what’s shown in this article except the NICs (eth2,eth3) don’tshow an inet value… Is that normal?

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

10 of 33 11/15/2011 5:15 PM

Page 11: Team Multiple Network Interfaces (NIC) Into a Single Interface

Reply

22 Andy July 24, 2008

Great tut but will bonding work with dynamic IP’s as I do not have static IP’s from my ISP

Reply

23 yandy August 6, 2008

Great guide…..managed to configure almost everything but after reboot of server, only managed to see bond0 in up status. I need to executethe network script (service network start) again in order to have eth0 in up status. Kindly advise how can this be resolve??? Thank you.

Reply

24 Mandar August 13, 2008

Very nice explanation!

Reply

25 Mike Baysek September 2, 2008

If you are considering teaming NICs, remember the limitations of the bus itself.

For example, if you think putting 4 Gigabit cards in your PCI bus will get you more speed, you are sadly mistaken! PCI bus is limited tomaximum of around 133 MB/s. Putting two gigabit NICs on PCI will be a waste of time. Better to have at least PCI-X, or even better, PCI-ebefore you think about teaming your adapters.

Remember too, that even on-board NICs usually share the PCI bus with the PCI slots. So adding a gig card to a machine with an onboard gigadapter would also usually be a waste of time unless your bus was, again, PCI-X or better.

Reply

26 Hassan Tariq September 24, 2008

I exactly need the same kind of solution in Windows 2003 Server.Please help me out on this!

Reply

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

11 of 33 11/15/2011 5:15 PM

Page 12: Team Multiple Network Interfaces (NIC) Into a Single Interface

27 mike October 7, 2008

Does anyone know how to trunc two bonding Interfaces to a single one with active-passive mode?

Reply

28 tecnimxv October 21, 2008

I would like to ask if it is possible to bond ethernet connection with a wireless one?

Reply

29 gael November 6, 2008

My OS: xen-dom0 on centos5

I am setting up bonding at the moment and I was wondering why we need:

ONBOOT=yes

I tried with “ONBOOT=no” and my bond interface is brought up at startup with both slaves…

As far as I understand bonding, it seems that when the bonding interface is starting up, it brings up the required slave interfaces.

Please correct me if I am wrong :-) And if you can explain why it would be excellent!

CheersGael

Reply

30 Tri Aji Wibowo October 5, 2011

Hi Gael,

I think you must be hash your config MASTER=bond0.Try this:

DEVICE=eth(x)

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

12 of 33 11/15/2011 5:15 PM

Page 13: Team Multiple Network Interfaces (NIC) Into a Single Interface

USERCTL=noONBOOT=yes#MASTER=bond(x)SLAVE=yesBOOTPROTO=none

-TA-

Reply

31 USMAN December 4, 2008

i m new with linux.i m using redhat linux 9.0 with squid proxy server for my netcafe.i have two broadband conections.the setting of externalinterfaces(broadeband) are following1)192.168.0.1 ip255.255.255.0 mask192.168.0.1 gateay2)192.168.1.1 ip255.255.255.0 mask192.168.1.1 gateway

my internal nic ip is 10.0.0.1 which is for out put.plz send me the bond setting with these ips.i will be very thankfull to you.

Reply

32 AB June 19, 2010

Hi There, I have the same situation, did you get bonding working for your network. Please share the information, would be great.Thanks AB

Reply

33 vijay January 29, 2009

i want to install inbuilt squid proxy in fedora server 9 pls guide me step by step configuration

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

13 of 33 11/15/2011 5:15 PM

Page 14: Team Multiple Network Interfaces (NIC) Into a Single Interface

i am using tata broadband

with regardsvijay

Reply

34 LinuxStudent February 10, 2009

Hey Man,

Solid info provided with minimum efforts…..

Reply

35 nima February 21, 2009

HiI have 2 links (5Mb and 2Mb) from 2 ISP, Does this solution give traffic load balancing possibility as if I have 7Mb bandwidth ?thanks for your any help

Reply

36 Linux-student February 24, 2009

i m using linux fedora 9.0 with squid proxy server for my internet cafe. I have two broadband connections. The setting of externalinterfaces(broadband) are following

1) Bandwidth – 2 Mb uplink10.254.55.16 ip255.255.252.0 mask10.254.55.1 gateay

2)Bandwidth – 3 Mb uplink10.254.22.45 ip255.255.252.0 mask192.168.22.1 gateway

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

14 of 33 11/15/2011 5:15 PM

Page 15: Team Multiple Network Interfaces (NIC) Into a Single Interface

3)LAN IP192.168.1.2255.255.255.0192.168.1.1

What I need to get traffic load balancing. Could you Please advice me.

thank in advanceKA

Reply

37 TJ March 1, 2009

Hi,

How do I connect the slave interface eth0, eth1 to the LAN switch, both interfaces need to connect or just only one ? …

If both eth0 and eth1 are connected, does the LAN switch select one higher priority than other ?

ThanksTJ

Reply

38 Elena March 3, 2009

Hi

I have a bond created. But if I bring both interfaces down, and then back up, they are not making link up; they appear as down.. They willonly come up if I bring up the whole bond. Where can be the problem?

Reply

39 kawaii avon April 3, 2009

how come that u set the mode in balance-alb yet when you type ifconfig, the configuration shows round robin mode..?

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

15 of 33 11/15/2011 5:15 PM

Page 16: Team Multiple Network Interfaces (NIC) Into a Single Interface

Reply

40 shay May 6, 2009

options bond0 mode=balance-alb miimon=100should beoptions bonding mode=balance-alb miimon=100

Reply

41 Dima May 12, 2009

Thanks for great article! ;)

Reply

42 Eldo May 14, 2009

Hi,

I try to configure bond interface in VMWARE with 4 ethernet cards, I follow the steps mentioned above. The problem is only bond interfacehave ip, there is no ip for eth0,eth1,eth2,eth3. I tried this in RHEL 5.1

Reply

43 Mike Roberts May 20, 2009

I am currently having issues with bonding on a RHEL 5 machine. I have the bonding setup correctly and when I run the ifconfig command itlooks to be up, however when I attempt to ping this machine from another computer I get no response. However, if I start a ping from theRHEL 5 comptuer and then attempt to ping from another computer it responds. Any insight would be greatly appreciated.

FYI, I am using “active-backup” mode.

Reply

44 James J August 31, 2010

Did you ever find the fix for this? I’m having the same issue.

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

16 of 33 11/15/2011 5:15 PM

Page 17: Team Multiple Network Interfaces (NIC) Into a Single Interface

Reply

45 Jarvis Semien June 3, 2011

Please let me know if you guys found a fix for this. Thanks in advance

Reply

46 loptos June 24, 2009

Is there any command, which would swith the active interface in the bond?Example: bond0 consists of eth0 and eth1now eth1 is activeenter commandnow eth0 is activeAlso I’m searching for command, which would check, which interface in bond is active.(Suse linux)Thanks, Peter

Reply

47 simon August 5, 2009

Been searching for the answer to loptos above and thought I’d leave it here.

To find which interface is active have a look in /proc/net/bonding/bond0 (mentioned earlier).

To change which is active have a look at the ifenslave command and try something like this:ifenslave -c bond0 eth1

Reply

48 fabrice August 21, 2009

In my case I wasn’t able to start a second bonding interface on a RHEL4.bond0 : eth2 and eth3bond1 : eth4 and eth5

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

17 of 33 11/15/2011 5:15 PM

Page 18: Team Multiple Network Interfaces (NIC) Into a Single Interface

Error message was :“Bringing up interface bond1: bonding device bond1 does not seem to be present, delaying initialization.”

after 2 days of google and test i found a working configuration. My problem was in the modprobe.conf configuration file.

Here are the bonding configuration in the modprobe.conf working for me :

# allow 2 bounding interfacesoptions bonding max_bonds=2

# bond0install bond0 /sbin/modprobe -a eth2 eth3 && /sbin/modprobe bondingalias bond0 bondingoptions bond0 mode=x,miimon=100,primary=eth2,etc.

# bond1install bond1 /sbin/modprobe -a eth4 eth5 && /sbin/modprobe bondingalias bond1 bondingoptions bond1 mode=x,miimon=100,primary=eth4,etc.

Hop it will help someone.

Fabrice

Reply

49 EllisGL June 22, 2011

Thanks. That did the trick. I had issues with it half working and this totally fixed my issues.

Reply

50 Deiveegan August 27, 2009

Hello sir,here we are using 3 squid (proxy )in Redhat 4 and sharing internet for 1) for wifi 2) for Lan and 3) for staff … if one squid dead that peopleare suffering ..if there is any load balancing server for squid ..how share there load if one squid dead…

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

18 of 33 11/15/2011 5:15 PM

Page 19: Team Multiple Network Interfaces (NIC) Into a Single Interface

Thanking youDeiveegan.SSASTRA UniversityThanjavur-Tamilnadu-India

Reply

51 satya September 28, 2009

Thanks a lot fabrice,

we are also facing exactly the same proble(multiple bonding in rhel 4.7).Let me try with the solution provided by you,and will update the comments.

Cheers,satya

Reply

52 jacopo January 12, 2010

Leave a comment… Well, this is more a question because I try to understand how to placemy actual IP address when I am using a client’s TOR network…

Reply

53 ap January 25, 2010

Hello, I have few questionlast week I playing with bonding.At work I making file server, this is ubuntu server box with 2 nic’s also we have windows 7 clients with 2 nic as well & finally cisco switch4500. Becourse we work with huge files, network speed is our main headache, so I try to aggregate network interface. I done it, but speedstay the same. I can utilize only 1 Gb connection (110 Mb /sec), but I want more!Trying different conf, result the same. Even I make connection between win boxes without 802.3 ab cant have more than 110 Mb/sec. Now Ithink may be it is cisco misconfiguration?

Please tell me if some one got multiply speed bandwidth ?

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

19 of 33 11/15/2011 5:15 PM

Page 20: Team Multiple Network Interfaces (NIC) Into a Single Interface

Reply

54 GMB October 7, 2010

@ap

LACP / 802.3ad only allows two end-points to max-out a single channel. Having 2x 1Gbps ports does not give you 2Gbps, but two1Gbps devices can individually consume one each. LACP is also better placed for fail-over; pull a lead and communication continues.It does allow you to add many different speed lines too, so 2x 1Gpbs and 4x 100Mbps into one pool.

Also, are you sure you’re diferrentiating the difference between 1Gbps (1000Mbps) and 110MBps – note the lower / upper “B” forbits and Bytes. Therefore 110MB/s is around 880Mbps, or close to 1Gbps.

Reply

55 GMB October 7, 2010

Just a thought as well – PCI Gigabit cards are limited to the bus speed of the host system and never quite give full Gigabitspeeds. Better to get PCI-Express based NICs.

If it helps, here’s my Cisco C4507 configuration:

interface Port-channel11description Proxy LACP 802.3adswitchportswitchport access vlan 50end

interface GigabitEthernet7/31description proxy 1-2switchport access vlan 50channel-protocol lacpchannel-group 11 mode activeend

interface GigabitEthernet7/32description proxy 2-2switchport access vlan 50

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

20 of 33 11/15/2011 5:15 PM

Page 21: Team Multiple Network Interfaces (NIC) Into a Single Interface

channel-protocol lacpchannel-group 11 mode activeend

With Ubuntu configuration set with Bonding using mode=4 (for 802.3ad LACP)

https://help.ubuntu.com/community/LinkAggregation#mode=4%20%28802.3ad%29

Reply

56 Novajoseph March 5, 2010

Hi Seno,

u metioned above limited only by the number of network interfaces (NIC) Linux supports and/or the number of network cards you can placein your system

I have dought . . How many nw interface Redhat linux support and can bonding two bonding (bond0 ,bond1) …

i have 4 * 1 Gb in one PCI Xpress .How can i bond it..

Reply

57 Novajoseph March 5, 2010

Hi nixcraft ,

u metioned above limited only by the number of network interfaces (NIC) Linux supports and/or the number of network cards you can placein your system

I have dought . . How many nw interface Redhat linux support and can bonding two bonding (bond0 ,bond1) …

i have 4 * 1 Gb in one PCI Xpress .How can i bond it..

Reply

58 Tarik March 23, 2010

Mike Baysek said: “If you are considering teaming NICs, remember the limitations of the bus itself. For example, if you think putting 4

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

21 of 33 11/15/2011 5:15 PM

Page 22: Team Multiple Network Interfaces (NIC) Into a Single Interface

Gigabit cards in your PCI bus will get you more speed, you are sadly mistaken! PCI bus is limited to maximum of around 133 MB/s. Puttingtwo gigabit NICs on PCI will be a waste of time. Better to have at least PCI-X, or even better, PCI-e before you think about teaming youradapters.”As per Wikipedia article on conventional PCI, the capacity of the PCI bus is 133 MB/s (32-bit at 33 MHz), 266 MB/s (32-bit at 66 MHz or64-bit at 33 MHz) and 533 MB/s (64-bit at 66 MHz). Accordingly, PCI 32-bit at 66 MHZ or 64-bit at 33 MHz would only be saturated with3 NICs. 64-bit at 66 MHz would take 7 NICs to be completely saturated.

Reply

59 Xarses March 30, 2010

An additional note is that while there are certain bus limitations as Tarik noted there are constraints of the bonding algorithm. Most notablyis that in most configurations such as those mentioned here the maximum speed of a single connection is determined by the speed of theindividual interface in the bonded group. In an example of 2 x 1Gbit adapters the maximum throughput for a single connection is 1Gbit/s. AsI understand the limitation is that the Ethernet frames for any given conversation will not be moved to another interface unless that interfaceis down. The intent of this design is to prevent out of order frames which would ruin your day create performance issues on the end pointthat would make the extra performance moot. Not to mention that most security software and network devices would drop excessiveamounts of this kind of traffic all together. If you want to use the full capability of the two links in our example you would have to devise amethod to transfer the content using 2 or more conversations. IE cut the data into two groups and re-assemble it on the other side. Otherwiseyou would be better off using 10Gbit or other higher performance interfaces.

Reply

60 ashis Sharma April 2, 2010

Really, very helpfull,

Thanks

Reply

61 Frank May 7, 2010

How can we install mrtg to monitor the traffic comming on bond0

Reply

62 Hammad July 13, 2010

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

22 of 33 11/15/2011 5:15 PM

Page 23: Team Multiple Network Interfaces (NIC) Into a Single Interface

do i have to moidfy the networrk marsk too along with my ip address??

Reply

63 nick July 16, 2010

Thanks nixcraft, always…nice article, just need to add GATEWAY in bond0 file and voila!!

Reply

64 Garfield August 4, 2010

I wonder to konw whether it support mutiple interface bond in IPv6? Can we receive a flow with multiple interfaces?

Reply

65 George September 14, 2010

Great article – thanks for the info.

BTW – you may want to check your banner advertisers – My antivirus (nod32) is blocking some javascript content from your website asbeing malicious.

Reply

66 vinoth September 18, 2010

Thanks for your quick comment about network bonding … you did such a nice job… keep it…….

Reply

67 Rahul October 15, 2010

Hi All,

this bonding automatically remove after the server shutdown or reboot

kindly check the same and let us know

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

23 of 33 11/15/2011 5:15 PM

Page 24: Team Multiple Network Interfaces (NIC) Into a Single Interface

Reply

68 GMB October 16, 2010

@Rahul

If you’re bashing in commands and getting it working, then yes that remains in state so long as the OS is running. You need to edityour config files to make the bonding persistent through reboots. The link I posted above was for Ubuntu, but there should be a guidefor your OS of choice.

Reply

69 tony November 1, 2010

ifconfig does not show the inet addr for both interfaces “eth0 & eth2″ like the example given although bond0 looks good. Am I missingsomething ?

Reply

70 GMB November 9, 2010

@tonyI guess it depends on the mode you’re using. The example given is using mode 6 “balance-alb” which can be used with basic switches.This therefore uses identical IPs and one active / one slave type mode.

Using mode 4 “802.3ad” would require an intelligent switch such as a Cisco and the switch configured for 802.3ad LACP mode also.This is an active / active mode and I guess (as I cannot remote into our server at work to check right now) will only give an IP on thebond interface since the other links are bonded in a truer sense.

Reply

71 sabry January 4, 2011

hello,

if let say i already bound this eth0 and eth1, and later maybe want it to be unbound again, so how can i do that?

thanks…

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

24 of 33 11/15/2011 5:15 PM

Page 25: Team Multiple Network Interfaces (NIC) Into a Single Interface

Reply

72 Rael January 20, 2011

I made a bond on OpenSuse 11.3 through the GUI. You just jump to the Network settings, and click on ADD. Choose ‘Bond’.Then add to the bond the existing Nic-s – as many, as you have! I even added a fiber controller. Make sure the network adapters to be addeddon’t have any IP assigned, and also, in the Global options choose Never wake up! Then they’ll appear in the list of slave devices.Cool, right! :)

Reply

73 paco sanchez January 31, 2011

I am surprised nobody has mentioned testing this “bond” I have the bond setup as described, however if i run ‘ifconfig eth0 down’, all of asudden the link is not pingable, even though /proc/net/bonding/bond0 shows that it is up with eth1…

Another thought, does it seem a little odd that all the MAC addresses are the same?

Reply

74 GMB February 1, 2011

@paco sanchez

It would be my guess that the switch-side isn’t setup correctly if eth1 isn’t activating on fail-over (by you downing eth0 manually inthis case). A real test would be to disconnect an Ethernet lead physically since that or a failed NIC / switch interface is more likely tocause the bond to continue communications over the remaining active links.

The MAC addresses become the same because that is how networks operate at the near-physical level. All NICs in a bond respond torequests to that address, though it is usually down to the switch to control the distribution of requests if it is an intellegent unit (Ciscoet al) in which case it too will be configured for bonding using LACP or similar (and therefore offer round-robin / load-balancing typemethods across the bonded links).

Reply

75 paco sanchez February 1, 2011

Thanks GMB for your prompt reply. It makes sense what you are saying. In my setup, I have a VM running in VirtualBox with

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

25 of 33 11/15/2011 5:15 PM

Page 26: Team Multiple Network Interfaces (NIC) Into a Single Interface

two vnics. I am not sure what the switch architecture is like for vbox but what I am looking for is adapter level redundancy. Iknow in the windows world, the “bond” takes on a virtual MAC which it uses on the network so as to avoid conflicts. If oneadapter is downed, the other takes over at the physical layer using this “spoofed” MAC. I am looking for the same functionalityin a Linux box, is this possible to acheive irrespective of my switch?

Reply

76 GMB February 5, 2011

@paco sanchezI might not of understood correctly, but are you trying to bond NICs within the guest VM? This would be pointless sincevirtual NICs are bridged to a single physical NIC.

I only use VirtualBox at home so I don’t use it in any fault-tolerant way, but at work I use XenServer that can bondphysical NICs and link them to the guest VM as a (single) virtual NIC. The guest VM doesn’t need to manage faulttolerance since it will always have communication via the host’s fault tolerance.

For the mode setting you could try the “active / backup” mode, which doesn’t give you load balancing but does work withmost basic layer 2 switches. If one link fails the other one simply takes over – making it fault tolerant, but only one linkcan be utilised for bandwidth at any time.

Interestingly, the MAC address is taken from the first network card in the bond unless explicitly set to something elseduring the configuration. That’s how you end up with one card’s MAC address spoofing for all of them.

Reply

77 paco sanchez February 6, 2011

GMB, you are correct, it is pointless inside a VM unless you are using it to learn. I am self-learning Linux (andloving every minute of it I might add) and just cannot afford to buy a bunch of physical dual-homed machines to tryall this neat stuff out on. In theory, any VM (whether in virtualbox or another hypervisor) which is presented withmultiple VNics ought to behave nearly similarly to a physical machine with two nics, am I incorrect in assumingthis? I have verified that this works for Windows VM’s using “teaming”, I am simply trying to replicate thisbehavior in Linux. I have some update though.

Reply

78 GMB February 6, 2011

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

26 of 33 11/15/2011 5:15 PM

Page 27: Team Multiple Network Interfaces (NIC) Into a Single Interface

@paco sanchezHello Paco, I’m understanding now. That should certainly work between guest VMs, being the virtual NICdriver doesn’t really go out to the physical NIC if the data remains within the host (so one guest doesn’tcommunicate to the switch and back in to the other guest, so to speak).

However, I’m not sure if the virtual switching would be LACP / 802.3ad aware or compatible, so you maywell still be limited to active / backup mode.

Keep persevering in Linux though, the tide is changing and Microsoft isn’t the mainstay in the back office itonce was. I’ve deployed Ubuntu and FreeBSD servers at my work for services such as MRTG and help desksystem. I’ve been playing with both for over 15 years and I’m still learning!

Reply

79 George February 9, 2011

same problem…2 NICs, mode=1, 2 switches

Reply

80 William B. Peckham February 26, 2011

nice!Thank you for presenting this.

With gig NIC speed should not really be the issue. Load balancing helps, if the load justifies, but it is the failover capability that presents thereal value.

Reply

81 Arjun Kapoor April 25, 2011

nice tutorial… please tell me how do i share this bond’s internet connection on the whole network via 3rd nic which will connect to theswitch.

Reply

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

27 of 33 11/15/2011 5:15 PM

Page 28: Team Multiple Network Interfaces (NIC) Into a Single Interface

82 Arjun Kapoor April 25, 2011

also please tell that if my 2 nics’ ip is dynamic the will it work….. please justify the bond’s ip address (is it the static ip for the network ?).thank you

Reply

83 Tarun May 4, 2011

Thanks dear….Good document…

Reply

84 Bret June 9, 2011

Hi Nixcraft,

I have got one linux server (CentOS 5 installed) with two NIC interface (eth0 & eth1) on it.

I want to configure server so that- Only http and ssh traffic goes to/from eth0 interface (with IP-filtering) and- Only https (external access) via eth1 interface. No other traffic allow on eth1 interface.

Is this possible to configure?

Thanks in advance,Bret

Reply

85 Vivek Gite June 9, 2011eth1 – 202.54.1.1 bound https port to this ipeth0 – 202.54.1.5 bound ssh and http port to this ip

Use plain old good IPtables to deny everything on eth1 interface except port 443 (https). Same goes for eth0 interface, blockeverything except port 80 (http), 22 (ssh) and other required port.

Reply

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

28 of 33 11/15/2011 5:15 PM

Page 29: Team Multiple Network Interfaces (NIC) Into a Single Interface

86 William Peckham June 9, 2011

BRETInteresting question, but WAY off topic. That has nothing to do with port binding.

Reply

87 Ethan Case July 5, 2011

LinuxTitli:Thank you for this!! You made it super easy! (or maybe I’m finally learning linux)To Bret, and others wanting to do Protocol Binding:I’m pretty sure there is a method to do this in Linux, but you can also do this using a hardware-firewall. I use a Netgear SRX5308 – it’s aquad-WAN/LAN firewall. I have multiple static ip’s from different isp’s – using Protocol Binding on the firewall/router, I can set specifictraffic to pass through certain WAN ips.

Reply

88 kumar July 6, 2011

HiI am facing some sort of a peculiar issue. As per the step 1,2,3 i have done the bonding. Issue here is when i remove the eth1 physical port.,the pinging will continue, but if i remove eth0 after eth1 is fixed back…GONE.. my network ping shows REQUEST TIME OUT. if i fixback the n/w cable no way still RTO.Below is the h/w details for better understanding.BL 460c G7, embedded 553i emulex 10G nic, connected to 1GB Ethernet Pass Thru, RHEL 5.6 x64.. Can anybody share your inputs toresolve this issue at the earliest…Thanks … Vijay

Reply

89 kumar July 6, 2011

Hi AllAnother point to be added… Both the nics are connected to single Cisco 4500 switch and the round robin was also changed to fault tolerant(active-standby). Request your helpReg Kumar

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

29 of 33 11/15/2011 5:15 PM

Page 30: Team Multiple Network Interfaces (NIC) Into a Single Interface

Reply

90 Mostafa July 8, 2011

I have a PC and DSL connection. It`s OS is OpenSuse and router is from DLink. My PC can connect to Internet throw DSL and it is possibleto ping it (it has static IP) but it is not possible to access SSH server from it.It has 2 network interface, eth0 and dsl0. eth0 make a connection to the DLink router and dsl0 (which tack static ip) connect to the internet.I need to access My PC from internet but it is not possible. Is it possible to solve my problem with bounded interfaces?I`m new in linux, please help me. Thanks.

Reply

91 William Peckham July 10, 2011

Mostafa: That is a router issue, you need to configure your router to pass outside traffic on port 22 ( and any other service ports you want touse for access from ‘outside’) to your server. It has nothing (IMOHO) to do with port bonding.

Reply

92 Nick July 22, 2011

Is it possible to apply this technique to a couple of USB data modems but use it to stream to another box – does the data stream have to bere-combined at the other end somehow to retain sync?I’m trying to make a location video streaming solution.

Thanks

Reply

93 Vivek M Garg August 9, 2011

Configuration was simple & worked perfectly well.Got High throughput (HT) as well as High availability (HA) with the bond.However after a reboot did not get HT or HA.

ifconfig is showing only bond0 & eth0.eth1 is not visible in ifconfig. However it’s visible in ifconfig -a.cat /proc/net/bonding/bond0 is also showing only bond0 & eth0.

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

30 of 33 11/15/2011 5:15 PM

Page 31: Team Multiple Network Interfaces (NIC) Into a Single Interface

What to do to sustain the bond after a reboot ? Please help !

Reply

94 Kevin Pham September 6, 2011

Thanks for the great doc! May I add a note please…

When the “modprobe bonding” command is issued, it may unload and reload the network modules and you’ll loose connection to the remoteserver and you may not be able to issue the next command. To make sure things work without loosing connection to the server, we can putboth commands together in the same line

# modprobe bonding; /sbin/service network restart

Kevin

Reply

95 Arno September 10, 2011

I see that is posible to bound more than 2 NIC`s with eachother, but my questions as following.I have TP-Link TL-SG1016 (http://www.tp-link.com/uk/products/details/?categoryid=1588&model=TL-SG1016) and my mainboard is anAsus P5Q Premium that has 4 NIC`s onboard. i use it as a server with raid 5. I ask you, does the switch need also the trunk option to boundyour NIC`s?

Reply

96 RK_RAJ October 24, 2011

Hi friends,

By following this howto, I configured load balancing method of bonding with my 2 NICS in CentOS 5.5 X64 it works well without a singlehitch, thanks for your tutorial

Reply

Leave a Comment

Name *

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

31 of 33 11/15/2011 5:15 PM

Page 32: Team Multiple Network Interfaces (NIC) Into a Single Interface

E-mail *

Website

You can use these HTML tags and attributes for your code and commands: <strong> <em> <ol> <li> <u> <ul> <blockquote> <pre> <a href=""title="">

Notify me of followup comments via e-mail.

Tagged as: /proc/net/bonding/bond0, cat command, config files, configuration file, driver linux, enterprise linux, fedora linux, heavy duty, how to create a bond in linux, ifconfigbond, kernel feature, kernel module, linest, Linux bond, linux bonded interfaces, Linux bonding, Linux nic teaming, load balancing services, network cards, network interfaces,network scripts, NIC bonding linux, red hat enterprise, single channel, single device, virtual link

Previous post: Force sendmail to deliver a message in sendmail’s mail queue

Next post: Can I use SSL certificate for my sub domains as well?

Make the most of Linux Sysadmin work!

42k+ Subscribers | Twitter | Google +

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

32 of 33 11/15/2011 5:15 PM

Page 33: Team Multiple Network Interfaces (NIC) Into a Single Interface

C

Related Posts

What Can You Do With A Second Linux Server Ethernet port?Best Linux / UNIX Posts of 2007: Part ~ IHow to Publish Multiple Websites Using a Single Tomcat Web Application

©2004-2011 nixCraft. All rights reserved. Cannot be reproduced without written permission.Privacy Policy | Terms of Service | Questions or Comments | Copyright Info | Sitemap

RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-in...

33 of 33 11/15/2011 5:15 PM