How to Setup LACP

Discussion in 'Tomato Firmware' started by eangulus, Jul 31, 2013.

  1. eangulus

    eangulus Network Guru Member


    Completely new to setting up LACP Trunks and am having some small issues with it so I thought I would ask here to see if I may have missed something.

    I currently have a RT-N66U running Shibby's Tomato v111. I have turned on STP option in the LAN Settings.

    From here I have 2 CAT6 cables running to my detached home office to a HP ProCurve 1810G-24 J9450A Switch. In this I have setup a Trunk on ports 23/24.

    This seems to work fine until a device needs DHCP. It doesn't seem to be able to find the Router in the house untill I unplug 1 of those Trunk Cables. I have also tried setting the ProCurve Trunk to Static (incase STP wasn't working in the Asus) and it still does the same thing.

    And for anyone that asks why I need or want a 2x1Gb links, I have 5 kids each with 5 computers, and a XBMC PC in the house, with my work Computer and a Linux Server with Samba using 2 1Gb Intel NIC's in a Bond setup.

    I figured this should give me a little more breathing room with the 6 house PC's when they are using Media and Backing up to the Server in the Office.
  2. eangulus

    eangulus Network Guru Member

    Can anyone shed some light?
  3. koitsu

    koitsu Network Guru Member

    I need to see a physical cabling/wiring topology diagram, with all devices shown as well. As of right now I'm about 80% sure sure I know what your problem is, but I can't be sure without a full topology diagram.
  4. eangulus

    eangulus Network Guru Member

    No probs here it is.

    Attached Files:

  5. eangulus

    eangulus Network Guru Member

    Not quite everything there, but the core network is there.
  6. eangulus

    eangulus Network Guru Member

    Just for some more info if it matters, I have tried both Static On and Off for the Trunk settings in the ProCurve.
  7. plgvie

    plgvie Networkin' Nut Member

    Does Shibby's firmware offer link aggregation within RT-N66U and did you turn it on?

    Kind rgds
  8. koitsu

    koitsu Network Guru Member

    Lots to say here:

    1. You're using the term "trunking" incorrectly here. I understand it's confusing because ProCurve vs. Cisco differ in how they use this term.

    With Ciscos, trunking refers to using multiple switch ports to create "one big port" between two switches (e.g. two switches connected to one another via 4 100mbit ports, with all 4 of those ports trunked, which would be treated as "one big 400mbit port"), meaning it's a form of "port bonding" to multiply/increase available bandwidth.

    With ProCurves, a trunk is something very very different and does not have anything to do with this. It involves use of 802.1Q (VLAN tagging), which the Asus side does support (through "VLAN" builds of Tomato firmwares), but is a separate beast and I will not discuss it/cover it here -- I also do not discuss/involve myself with the VLAN stuff on Asus routers. I am familiar with VLANs on ProCurve switches (I ran a hosting organisation for 18 years and used VLANs to segregate traffic from each server we had in our datacenter; 2 NICs per server, one being "WAN" or "Internet-facing" the other being "LAN" or "private network", and used VLANs on a ProCurve 2510G). It's very possible that some of these probably-802.1Q-tagged frames are landing on your 8-port generic switches and being ignored (you would need to be told by the switch vendor if their switches understand or pass through 802.1Q headers or not -- you would be surprised how many don't).

    2. The configuration you're using effectively requires the Asus device (more specifically: the switching IC/fabric on the Asus, as well as the software side of things (Linux kernel and proper drivers)) to understand LACP. Meaning: for LACP to work, both sides need to understand the protocol. LACP packets (called LACPDUs) are at layer 2.

    As far as I know, none of the Asus devices/firmwares nor software running on them support LACP. The only way to control the behaviour of the switching fabric on the Asus device is through a proprietary Broadcom command called brctl, and on my Asus RT-N16 there is no LACP support mentioned here. LACP cannot just "magically work", you have to configure it on both switches.

    If you're using LACP between your Ubuntu box and the ProCurve, and that works, then that's fine. What isn't going to work is LACP between the ProCurve and the Asus.

    This is a general reference for my statements (read the comments):

    3. All of this this is separate from STP, which is for a different/unrelated problem (specifically physical cabling loops where by creating a cabling loop, frames would literally go out of one interface, across the cable, and back into an interface on the same device, which would then be processed and go back out an interface, rinse lather repeat, causing the switch to get stuck in an infinite loop). STP is also very, very slow (even RSTP is slow, just a lot less slow than STP); duplex and speed negotiation with STP can sometimes take up to a full 30 seconds (!). In general I tell people to avoid use of STP, RSTP, and things like LLDP given how they can badly affect bringing up a network link (when a machine reboots, etc.).

    I will mention that "passive mode" LACP does "sort of" act like STP, but you need to understand they're separate technologies unrelated to one another; what you want is "active mode" LACP.

    So in summary, what's happening is that you're probably confused the living hell out of the Asus device, which is probably treating each physical connection to the 1810G separate, and the reason things "intermittently" work is because of how switches "cache" MAC addresses and internally (within the firmware code itself at runtime) associate them with a physical Ethernet port. So what's happening is that frames coming from the ProCurve are probably "intermittently" landing on a different port on the Asus (this is purely timing, there's nothing you can do about this), so things randomly stop working until you pull one of the Ethernet cables, forcing ARP and all related protocols to reassociate/discover across the only remaining port.

    Options as I see them:

    1. Buy yourself a decent router, not consumer-grade crap, that supports LACP and does what you need. Consider reaching out to Juniper or Cisco. Expect to pay a large sum of money, but you will get what you need.

    2. Use a single network connection (one Ethernet port) between the Asus and the ProCurve. This means 1000mbit (gigE) but that's that.

    In fact, to be honest with you, I don't even understand why you're screwing around with LACP or bonding to begin with. There isn't anything on your network in that diagram that would justify from having 2000mbit worth of traffic -- the PC to the ProCurve is 1000mbit, the Asus to the generic 8-port switch is 1000mbit, etc... Not to mention, the Asus's WAN port is limited to 1000mbit as well, so unless you're doing this just "to learn" and "for fun", then I don't see what the purpose is. No offence intended of course.
  9. Malitiacurt

    Malitiacurt Networkin' Nut Member

    Overkill for your home setup. Streaming media does not require more than 50-100Mbps max. Even 20-25Mbps is enough for most 10GB 1080p x264 movies. You only require more if your streaming a raw copy of the blu-ray.

    And if they're copying files across the network, well it's not like adding a few extra seconds/minutes to the copy time is going to matter, plus it's pretty sure the chances are low that a bunch of them doing it at the same time AND require it immediately/time-critical files.
  10. eangulus

    eangulus Network Guru Member

    First it is not overkill for my home setup. My home is also my business. So its not a typical home setup at all.

    From the help files in the ProCurve it states the Trunking is the LACP stuff, and by trying to set it as Static should in effect mean I don't need LACP support on the other end. Thats what the help files say anyway. Thats why I was trying it.
  11. eangulus

    eangulus Network Guru Member

    I am doing this primarily for learning too. And all that info you have given has cleared a few things for me, thanks.

    There is a situation that will use the full speed of a LACP 2Gb link but is not critical. Meaning I won't spend money or a great deal of time on trying to get it to work, but just thought if its there and I can get it to work, why not.

    The linux server uses Bonding on the Box side so no need for support on the switches. I also have a similar box in the house as well as a duplicate backup. THis is the only case in which I could utilize the link, but in saying that it would only be certain drives to certain drives. I do have a couple of RAID setups using SSD's and they would be the only things fast enough to utilise the link. The main HDD RAID setup maxes out at around 60Mb/s so would never saturate the link.

    More importantly thou, I have 3 cables from the house to the office. Is there a suitable way for me to utilize 2 of these cables either in LACP or if not, as a simple backup line, incase a rat or the like chews thru one of the lines?
  12. eangulus

    eangulus Network Guru Member

    Just for reference, here are the Help files I have on the ProCurve.

    Trunking Configuration Page:

    This page allows creation, deletion, and modification of Trunk Configurations.
    • Create - Select to create Trunk.
    • Trunk Name - Displays the Trunk name, if configured. Otherwise, the user should configure a Trunk name. (Range: 1 to 15 Alphanumeric Characters)
    • Number of Trunks created - Displays the current number of Trunks created.
    • Interface - Displays the list of all Trunks created.
    • Trunk Members - Displays the configured Trunk members participating in the specified Trunk.
    • Admin Mode - Select to configure the admin mode for the specified Trunk. When the Trunk is disabled, no traffic will flow and LACPDUs will be dropped. But the links that form the Trunk will not be released. By default, when the user configures a Trunk name Admin Mode is enabled.
    • Static Capability - Select to enable a Trunk to operate in static mode. A static Trunk interface does not require a partner system to be able to aggregate its member ports. In this mode, it does not transmit or process received LACPDUs i.e. the member ports do not transmit LACPDUs and all the LACPDUs it may receive are dropped. ( Default: Disabled )
    • Modify - Select to modify the Trunk configuration. The user can modify the Admin Mode and Static Capability only if this field is selected.
    • Delete - Select to delete the specified Trunk.
    • A newly created trunk is by default dynamic i.e. Trunk will be initiated and maintained by the periodic exchanges of Link Aggregation Control PDUs (LACPDUs). If you explicitly configure Enable Static Mode for a trunk it will become a static trunk.
    • The traffic is distributed among trunk members based on hashing algorithm. The hashing algorithm considers source MAC, destination MAC, VLAN ID and ethertype field values from the each incoming packet and selects at a trunk member based on a Hash function. The incoming packet will be forwarded onto the hash selected trunk member.
    • Loop protection is not supported on dynamic trunks. Loop protection will be auto disabled if it was previously enabled on static trunk that is now being configured as dynamic.

    There are two main buttons associated with this page:

    • Help - Represented by a ? sign at the right corner of the web page. Click to display the help window.
    • Apply - Updates the switch configuration. No changes are made to the configuration until this button is pressed.

    Trunk Membership Page:

    In this page, ports can be added to or deleted from trunks. Also LACP can be configured on trunk members.
    Trunk Membership
      • Trunk Id - Displays the list of all trunks created. The user can select the trunk from the drop down menu, as well as the corresponding interfaces to be included as members of this trunk.
      • Port - Port Number.
      • Trunk - A port can be added to a selected trunk by clicking multiple times on the icon.


      • : Indicates that port is not a member of any trunk.


      • : Indicates that port is member of selected trunk.


      • : Indicates that port is not allowed to be configured. It is either a member of already existing Trunk or it should have been configured as a member (destination or source port) of the mirroring configuration. The user is not allowed to perform any trunk membership configuration on this port until the port is removed from the existing Trunk or mirroring configuration.

    Note: A maximum of eight trunks can be created, with a maximum of four ports per trunk.
    • LACP - LACP can be enabled/disabled on a port by clicking multiple times on the icon.

    [​IMG] : Indicates that LACP is disabled. LACP mode configuration is not allowed to perform on this interface if the port is either a member of already existing Trunk or it should have been configured as a member of the mirroring configuration. User is not allowed to perform any LACP configuration on the interface until the interface is removed from the existing Trunk or mirroring configuration.
    [​IMG] : Indicates that LACP is enabled.

    Note: Ports added to Trunk looses port VLAN memberships and get assigned to Trunk group VLAN membership. Ports removed from Trunk are assigned membership to the default VLAN.
    Note: It is mandatory that all the ports participating in a trunk have the same speed.
    Note: LACP mode configuration is not valid on a Trunk with static capability enabled. LACP mode configuration will not be displayed for static Trunks.

    There are two main buttons associated with this page:

    • Help - Represented by a ? sign at the right corner of the web page. Click to display the help window.
    • Apply - Updates the switch configuration. No changes are made to the configuration until this button is pressed.

    Both these pages talk about the LACP. So Still a little confused. Based on what I have read, even if the STP stuff in the Router is not the same as the LACP stuff in the ProCurve, then by setting the Procurve Trunk to Static means I don't need support on the other end.
  13. eangulus

    eangulus Network Guru Member

    Oh and to answer why I have got bonding.

    The Server with Bonding stores all our media. I have 5 Kids that watch different things all the time via their PC's or via either or both the Media Centres in the house. Since I setup the Bonding all that is working much better, I now get double the bandwidth to 2 devices at once. Prior to the setup we were saturating a single link often and now my limiting factor is the HDD's.
  14. koitsu

    koitsu Network Guru Member

    I would suggest you begin posting your dilemma over on the HP/ProCurve support forum (not on this site), giving them the same information you've given here. I completely cease to understand how "link bonding" would work without both switches being configured for this capability, meaning the "Static Capability" option sounds more to me like it has to do with 802.1Q tagged Ethernet frames than it does anything else (this is sort of implied by the "Note" parts of the LACP section) -- and for that (specifically "bonded links") to work, the switch on the remote end (the Asus) would need to know of this fact. I really don't see how else it could work. The folks on the HP/ProCurve forum can help you more. But the Asus devices, unless you're running the VLAN builds of Tomato, do not understand 802.1Q, just FYI.

    But all in all I still do not see what the bonding gets you, because nothing else on your network is able to utilise >1000mbit speeds, given the topology diagram you provided. At present the only benefit you have is 2000mbit between your Ubuntu box and a ProCurve switch itself, and if you were to get LACP working between the ProCurve and the Asus, all it'd get you is 2000mbit between the Ubuntu box, through the ProCurve, and to/from the Asus device. Home vs. business has no bearing on the situation either; I'm just not seeing the point of what you're doing, but I guess that's separate from the technical side of things. Setting up bonding on the Ubuntu box to the ProCurve switch absolutely cannot explain a "performance boost" when multiple people are using the PCs listed off in the upper right of your topology diagram -- think about it, the max bandwidth you can get due to the 8-port generic switch is 1000mbit!
  15. eangulus

    eangulus Network Guru Member

    I know there is probably no need, but I hate letting things beat me.

    Anyway just letting everyone know I got it working. I set up the Trunk as Static on the ProCurve then, based on the VLAN info, as I am using the AIO Tomato, under VLAN I checked the Trunk VLAN Support Override.

    Both cables are connected and so far I am not getting the issues I had before.

    ALso I am doing up a full network map in Visio and would love some ideas on how I could possibly configure it to obtain the best performance out of it. WIll post as soon as its done.

    Thank you for everyones opinions and information. It has all been really helpful.
  16. eangulus

    eangulus Network Guru Member

    Spoke too soon. The changes above, did and has worked allot better, as connected devices didn't drop off, but anything trying to get a DHCP IP, seems to timeout, and they didn't work until I unplugged the second line.

    Also heres a full detailed Network Map.

    Obviously to utilize the 2x1Gb links properly I think I need to replace the Generic 8 Port switches to something that supports trunking.

    Is there any other changes that should improve the network? Also changes now and after new switches too.

    Attached Files:

  17. plgvie

    plgvie Networkin' Nut Member

    Assuming that your internet-connection isn't above 1Gb;), I'd suggest to leave poor little RT-N66U alone and let him be just Router/Gateway and (if necessary) AP; So please just one cable from Asus' Lan-port to a (smart-)managed switch capable of 802.3ad. From this switch you can create as many trunks to as many other switches, servers, NAS,... as you see fit and as long as they support LACP either.
  18. eangulus

    eangulus Network Guru Member

    Would it be worth while to buy another ProCurve for the house end? This way I can utalise the bonding on the linux box, with my PC getting full speed and any 1 of the house PC's getting full speed?
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice