Tomato ignores IPv6 Router Advertisements

Discussion in 'Tomato Firmware' started by Marcin R., Jan 8, 2017.

  1. Marcin R.

    Marcin R. Network Newbie Member

    Hi All,

    I'm using the latest Tomato build for my ASUS RT-AC3200 (1.28.0000 -138 K26ARM USB AIO-64K)

    My ISP is Comcast Business with dynamic IP - the same as residental.

    I've configured my router to use DHCPv6 with 64 bit prefix delegation. This mostly works - the router gets a /128 IP address on the wan, then it is delegated a /64 prefix for the internal network, all this works well. However, the router is missing the IPv6 default route. I've enabled "Accept RA from WAN". This version of Tomato doesn't have the spurious ipv6 default route bug, I simply don't have an ipv6 default route at all!

    So, what I did was tcpdump all the IPv6 packets on the WAN interface, and the RA's are definitely there. So, I manually added a default route like this:
    route add -A inet6 default gw fe80::201:5cff:fe62:5446 vlan2, based on what I saw in the captured RA's and IPV6 works great for a while, until the router address changes.

    Any tips on how to make Tomato automatically accept those RA's and configure the default gw?
  2. Sean B.

    Sean B. LI Guru Member

    Interesting. I'm wondering if there's something different regarding the Business class. I use Comcast residential and it's working as it should, although it will fail to obtain the routes if "Request PD only" is used. Have you confirmed the 64 prefix is correct for business class? Also, are you using the provided Comcast gateway/modem combination or are you using your own modem?
  3. Marcin R.

    Marcin R. Network Newbie Member

    No, it turns out that it's a kernel bug that was fixed in 2.6.37, but this tomato build is off 2.6.36.

    Here's a link to another forum post about this.

    I'm using my own Arris modem, not their business gateway (which does some strange stuff over RIP and static IPv6). The IP addresses are correct. This is misbehavior purely on Tomato's side.

    I previously had the static IPv6 from Comcast with their modem, and there's no automatic way to get that working. They assign you a /56, so you just pick a /64 from out of there and manually configure Tomato with some hard coded addresses. I had to do it via WAN up scripts since there's no way to set the default gw via UI, and with the business gateway modem, that never changed.
  4. Sean B.

    Sean B. LI Guru Member

    Hmm, IMHO I don't think this is a kernel issue. If it was, this would be a much more wide spread and prevalent issue.. not a " sometimes works sometimes doesn't " issue as that post states. The direct cause, once again IMHO, is likely a configuration miss-match in either the Tomato IPv6 settings/Cable modem/or ISP dhcpv6 server. I see that Comcast business issues a /56 and expects their supplied gateway/modem to delegate /64's for internal subnetting. If this is the case, you'd be missing a route if your configured for a 64 prefix. Or if using a gateway/modem combo and not in bridge mode, sense ( if properly configured ) your supplied comcast gateway needs to pass through to your tomato router and not try to handle routing itself.. otherwise it will conflict. If you're set that it's a kernel issue, all good. If you'd be interested in a bit of a debugging process to be sure this isn't something we can get working I'd be happy to run through it with you. Up to you :)
  5. Marcin R.

    Marcin R. Network Newbie Member

    Comcast only uses a /56 is you have static IP addresses, which I do not. The dynamic IP based business class is identical to consumer (I will be switching soon). The router delegates a specific /64 and send an RA advertising itself. Tomato was ignoring it. I don't think it's a modem incompatibility - the modem dutifully forwards evertything as tcpdump showed. The link above suggested disabling forwarding on the interface where you want RA's so that the kernel doesn't drop them, and that indeed fixed it for me. This is a kernel behavior which the UI needs to consider when generating its config scripts. The Tomato config behavior is correct for 2.6.37+. I only have one router, so maybe other routers run a later kernel version and don't have this issue.
  6. Sean B.

    Sean B. LI Guru Member

    I'd suggest configuring the Basic->IPv6 section with a prefix length of 56, and then delegating a 64 to your LAN interface in dnsmasq custom section. I have a feeling this may solve the routing issue. Worth a shot.
  7. Sean B.

    Sean B. LI Guru Member

    Ah I see, not familiar with Business class. All tomato firmware runs either 2.4.34 (MIPS) or 2.6.36 (ARM). The broadcom drivers are tied to these versions and it cannot be changed.
  8. koitsu

    koitsu Network Guru Member

    @Marcin R. So, um, did you try the workaround (disabling IPv6 packet forwarding for the WAN interface) in the thread you posted? I'll provide actual links to remove confusion:

    This supposedly worked for some people:

    But not others:

    And one is undetermined:

    Also, when someone online tells me something like "it's a kernel bug which was fixed in 2.6.37 or newer", I always ignore it/reject it unless actual commits that confirm/reference the problem itself are provided. It's not that I don't "believe" (the poster of that thread or you) someone, it's that "blame the kernel" is something I've heard throughout the decades more times than I can count and is often used as a defunct go-to when someone can't explain something (think magic smoke). Can you find the correlating Linux kernel commits that fix it? I haven't been able to (here's another reference). Because such things can be backported to 2.6.36. We cannot upgrade/change the kernel version due to reliance on binary blob wireless drivers from Broadcom.
  9. Marcin R.

    Marcin R. Network Newbie Member

    Yes, this works for me, that's why I responded when I found that thread, since I wasn't aware of it before posting originally.

    I was just repeating what that first thread says, don't have time to dig into this myself, I accepted this is a behavior with changed w.r.t. forwarding interfaces in 2.6.37, don't have time to vet everything myself. Check out OpenWRT issue 12882, it has more context, but it does appear that accept_ra = 2 doesn't work before kernel 2.6.37, but that setting is used in the 2.6.36 on my router - though I have no idea if it's been backported. It clearly works for some people.

    What I know is that in my case:
    - WAN RA's were ignored with accept_ra = 2 for vlan2
    - The Arris modem forwarded these to my WAN port, because tcpdump on the router showed them
    - Setting forwarding= 0 fixed the problem, and it explains why I was able to manually make a default route which fixed everything.

    It really does sound like that behavior change. Is there a list somewhere of which kernel patches have been ported back into Tomato's source, or do I have to dig through git?
  10. koitsu

    koitsu Network Guru Member


    OpenWRT ticket 12882 talks about how as of Linux 2.6.37, a new value to the accept_ra tunable was added: 2. Prior to that, only values 0 and 1 were accepted. I have talked about this tunable recently (within the past 3-4 months). With IPv6, it has to do with SLAAC.

    There is a good reference for this matter: (I hate the way this information is presented, but it's there -- note, we do not have sysctl on Tomato)

    As for Tomato code: you'll have to dig through git, I'm sorry to say -- and it's very painful because of all the forks and integration/sharing of commits (many things have to be manually backported/migrated) between forks.

    There are a couple places which tweak/touch accept_ra. I'm more familiar with MIPS (2.6.22) than I am ARM, but I'll provide the following as a quick point of reference. God, the more I look at this the more text I have to write out. There is no documentation for Tomato, so reverse-engineering must be done every time.

    I am referring below to the Toastman-ARM branch -- I do not look at Shibby branches.

    NVRAM variable ipv6_accept_ra is the tunable (an integer) that controls accept_ra behaviour for the WAN interface and the LAN interface (yes, in one variable). Bit 0 controls the capability for the WAN interface, and bit 1 controls the capability for the LAN interface. You can see how the code in router/www/basic-ipv6.asp ends up OR'ing the relevant bit/value (per interface), that's how the checkbox works in this case.

    I'm using the word "capability" because this variable does not contain the actual value stuffed into the /proc/sys/net/ipv6/conf/XXX/accept_ra tunable.

    The default value for this NVRAM variable is 1 (per router/shared/defaults.c).

    There is a function called accept_ra() (router/rc/network.c) which accepts an interface name (ex. vlan2, br0, etc.) and then tweaks the appropriate /proc variables for that interface name like so:

     753 void accept_ra(const char *ifname)
     754 {
     755         char s[256];
     757         sprintf(s, "/proc/sys/net/ipv6/conf/%s/accept_ra", ifname);
     758         f_write_string(s, "2", 0, 0);
     760         sprintf(s, "/proc/sys/net/ipv6/conf/%s/forwarding", ifname);
     761         f_write_string(s, "2", 0, 0);
     762 }
    From this we can see that /proc/sys/net/ipv6/conf/XXX/accept_ra gets set to 2, and /proc/sys/net/ipv6/conf/XXX/forwarding also gets set to 2 whenever this function is called.

    But wait, didn't we just determine that Linux 2.6.37 is when the value of 2 got introduced/supported, yet ARM is running 2.6.36 (technically I think)? Yeah, now you're getting the picture. Tomato is not a normal Linux distro, it's a very kludgy hodgepodge hacked-up mix of things because we can't change the kernel version due to binary blob drivers.

    So, this may be one of those things that didn't get backported into Tomato -- or maybe it did. I simply don't know. The only way to know for certain is to actually dig through all the code and see if the accept_ra value of 2 is truly supported or not in the Linux kernel (digging through the Tomato Makefile infrastructure may also be required, as for all I know it might be a patch that gets applied at make-time, but I doubt it).

    Anyway, onward:

    accept_ra() is called in a couple different places.

    router/rc/wan.c has a start_wan6_done() function which is passed the WAN interface name. I haven't looked what calls this function. It has the following code in it:

     860         if (service != IPV6_DISABLED) {
     861                 if ((nvram_get_int("ipv6_accept_ra") & 1) != 0)
     862                         accept_ra(wan_ifname);
     863         }
    In other words: if bit 0 is set to 1 (i.e. accept RAs on WAN in the GUI is enabled), then (let's just assume vlan2 is the WAN interface) you'll end up with /proc/sys/net/ipv6/conf/vlan2/accept_ra=2 and /proc/sys/net/ipv6/conf/vlan2/forwarding=2.

    router/rc/services.c is another place, function start_ipv6(), which does the following:

     886         if (service != IPV6_DISABLED) {
     887                 if ((nvram_get_int("ipv6_accept_ra") & 2) != 0 && !nvram_get_int("ipv6_radvd"))
     888                         accept_ra(nvram_safe_get("lan_ifname"));
     889         }
    In other words: if bit 1 is set to 1 (i.e. accept RAs on LAN in the GUI is enabled), and ipv6_radvd == 0, then (let's just assume br0 is the LAN interface) /proc/sys/net/ipv6/conf/br0/accept_ra=2 and /proc/sys/net/ipv6/conf/br0/forwarding=2.

    This situation becomes more complicated when we get into discussing the forwarding tunable, and here's why:

    router/rc/firewall.c has the following code in it, where if IPv6 is enabled (per ipv6_service in NVRAM -- meaning if it's set to something, vs. being empty) then forwarding=1 else forwarding=0 (but keep reading):

     147 void enable_ip6_forward(void)
     148 {
     149         if (ipv6_enabled()) {
     150                 f_write_string("/proc/sys/net/ipv6/conf/default/forwarding", "1", 0, 0);
     151                 f_write_string("/proc/sys/net/ipv6/conf/all/forwarding", "1", 0, 0);
     152         }
     153         else {
     154                 f_write_string("/proc/sys/net/ipv6/conf/default/forwarding", "0", 0, 0);
     155                 f_write_string("/proc/sys/net/ipv6/conf/all/forwarding", "0", 0, 0);
     156         }
     157 }
    The "default" and "all" fake interfaces in Linux appear obvious at first, but in actuality they aren't. In fact, they're way more convoluted than their names allude to -- read the question and the top-voted answer (which talks about IPv4, but I want to assume it's the same for IPv6):

    So the reason I mention all this is because for a "proper fix" to be applied, one is going to have to REALLY sift through all the code and figure out what is calling what and when/where. You have to get familiar with the services bits too, because when clicking Save in some parts of the GUI, Tomato could end up running a function like the above that might stomp or override a forwarding=0 workaround.

    My opinion is that if the forwarding=0 workaround is actually working, then someone needs to figure out when and where Linux actually fixed the real problem, so that preferably none of the Tomato code (not the kernel, but Tomato) has to be changed.
    Monk E. Boy likes this.
  11. Marcin R.

    Marcin R. Network Newbie Member

    Damn, thanks for digging into that stuff. It's a mess indeed, and I didn't realize that the devs had to jump through so many hoops to stay compatible with closed binary drivers.
  12. koitsu

    koitsu Network Guru Member

    And now you understand one of the many reasons why OpenWRT doesn't support binary blob drivers. :)
  13. Sean B.

    Sean B. LI Guru Member

    Perhaps I've looked right over it, but I'm not clear on what is the cause of the issue for the OP when the IPv6 tuneables in their stock form are working for myself and many others on Comcast residential? Is this an actual Tomato issue, or a mis-match configuration between something different about Comcast Business?
  14. Sean B.

    Sean B. LI Guru Member

    It seems rather similar to what happens if you "Request PD only" on a Comcast connection. Your LAN interface will receive a /64 delegated prefix.. and dnsmasq will hand out IP's to clients properly.. but there will be no internet connectivity do to the missing route. Of which is a route that would have been through the NA-PD /128 given to the WAN interface had PD Only not been used. Not saying PD Only is the cause of this issue, but really seems like a basic configuration issue.
  15. koitsu

    koitsu Network Guru Member

    Wow, Sean... this may be a first (LOL), but I think your last post flew right over my head. Welcome to my inexperience with IPv6 (I "get it" but it's horribly overcomplicated, and why I have books :p). :D

    As for why the OP's situation might be different: I wouldn't be surprised if there was slight differences between the services. We know for a FACT there are differences in IPv6 behaviour between ISPs, so it wouldn't surprise me if there were slight anomalies between service classifications within a single ISP.
    M_ars likes this.
  16. Sean B.

    Sean B. LI Guru Member

    Lol you just made me need to verify gravity hasn't reversed and the sky isn't falling ;)
    M_ars likes this.
  17. Sinopsys

    Sinopsys Reformed Router Member


    I was very exited to find that thread as I am struggling to make IPv6 working on my Tomato box with the exact same situation but the forwarding trick isn't working.

    My config:
    - box: Netgear R8000 (ARM)
    - firmware: fresh tomato 2018.4 (based on kernel
    - ISP: NETVIGATOR (HK) with IPv6 native and DHCPv6-PD support
    - Connection type: FTTH through a huawei modem/router with no open doors

    My setup:
    - IPv6 activated with DHCPv6-PD
    - prefix lenght: 56 (seems to be the requirement as per some local forums)
    - Request PD Only set: no DHCPv6 prefix provided otherwise
    - Static DNS: blank
    - Accept RA from WAN:set LAN:unset
    - Announce IPv6 on LAN (SLAAC): set
    - Announce IPv6 on LAN (DHCP): set

    I have no special tweaks applied as seen in forums (forwarding, accept_ra) as it looks like this firmware has already integrated these changes with some patches in linux core sources.

    So here is what is happening:
    - DHCPv6 negociation is successfull thanks to dhcp6c:
    /56 prefix is provided
    DNS are provided and updated
    br0 (LAN) is assigned a /64 address that is advertised and validated
    Here is the "cleaned" log of dhcp6c:
    thanks to that I have addresses set on interfaces
    So far so good.

    Now the routing table isn't updated with default route through provided gateway fe80::200:5eff:fe00:102
    So this is almost normal out of DHCPv6 part.

    Nevertheless, I do have a ICMPv6 RA sent by the gateway on vlan2:
    But this RA is not triggering the ND processing as it should.

    To try to figure out what is happening on this RA, I've set a logging ACCEPT rule in ip6tables specifically for this type of icmp (134) message: no trace of it in syslog !?

    looks like it is trashed/droped/?? before.

    edit: ip6tables do log it !

    I would really appreciate your wise advises and I am keeping investigating...

    again the accept_ra/forwarding stuff seems to be managed natively with the famous accept_ra = 2 to workaround dependency on forwarding state.
    Last edited: Nov 7, 2018
  18. Sean B.

    Sean B. LI Guru Member

    Has the huawei modem/router been set to transparent bridge mode? I just skimmed through your post as I'm headed to bed, but initial impression is that the modem/router hasn't been set to transparently bridge your connection and is therfor receiving the IPv6 address meant for your WAN interface ( VLAN2 ) when PD-only is not selected. If modem/router is not configured as stated, I suggest doing so and then attempt connection using DHCPv6 w/ prefix delegation and do not select PD-only.
  19. Sinopsys

    Sinopsys Reformed Router Member


    Thank you for answering on wake up :)

    I have no hands on the Huawei device despite having it in my flat as the fiber end-point.

    I’ve tried to contact my ISP to get further info on the setup but they told me that they cannot provide me support as my personal router is not among their ipv6 compliant certified devices.

    I will try to investigate myself on it but from my previous attempt it looks pretty much black box.

    I’ll also review more in details that final RA that looks to be what is supposed to finalise the linkage from what I
    understand ! Any confusion ?
  20. Sinopsys

    Sinopsys Reformed Router Member

    Early research is that the bridge is Huawei hg8040h which is said on paper to support “Transparent transmission of IPv6 packets at Layer 2”

    Going to have a look on any config capabilities...
  21. Sean B.

    Sean B. LI Guru Member

    Here is a section from an IETF document entitled "Route Problem at Relay during DHCPv6 Prefix Delegation " :

    I believe that without transparent bridge mode enabled on the modem/router.. the issue is the same.
  22. Sinopsys

    Sinopsys Reformed Router Member

    Hello again,

    Thank you for the great reference.

    well difficult to tell as again I have no way to get into that huawei GPON/ONT/...

    But from what I can read is that it is a fully capable router with DHCP server, DHCPv6, SLAAC...

    Then I'm a bit confused on what mode it is configured actually.

    But apparently the Huawei device apply as the Delegating router.

    I'll try again with PD-Only unset and see what I get tonight and keep you posted...

    But it is very frustrating to see that if I manually set the default gateway in the routing table, I'm all set and have full ipv6 live and running :(
  23. Sean B.

    Sean B. LI Guru Member

    When you have time, give this a try and see if there's any change for IPv6 connectivity.

    In the routers web interface under Advanced->DHCP/DNS:
    • Disable ( uncheck ) the option "Announce IPv6 on LAN" for both SLAAC and DHCPv6.
    • Place the following lines in the Custom Config box -
    • Click save.
    • Under Status->Logs click the "View last 100 lines" link. Make sure there aren't repeating error lines from dnsmasq at the tail end of the log ( I'm not sure on exactly what version of dnsmasq the off-link directive was added, and therefor am not sure if it's supported by the version running under your build of Tomato ). If repeating error logs from dnsmasq are present, revert the changes made above. If no repeating error logs from dnsmasq are found, reboot the router.
    • While the router is rebooting, reboot all LAN clients as well.
    • Once the router and clients are back up, check for any change in IPv6 connectivity for LAN clients.
    Last edited: Nov 8, 2018 at 1:05 PM
  24. Sinopsys

    Sinopsys Reformed Router Member

    Feedback on the test with PD-only not set: it is not improving the situation. Delegating router is answering the DHCPv6 process the same way as with PD-only so I do get /56 prefix, DNS, a local address for WAN and a local+global address is assigned to LAN.
    But for the IA part, Delegating router is rejecting with an error:
    Then dhcp6c run in endless loop re-sending SOLICIT for IA that is not answered by the delegating router.

    :( So PD-Only seems the better option to accommodate my ISP setup !?

    If I sum'up my understanding: my router is performing the DHCPv6 protocol pretty much correctly for the WAN part to get the prefix and then taking the delegation to create an address for the LAN and advertising it to make it global while The WAN do not get its own global address. but nothing else is expected from there actually is it ?

    From the various reading I got, the default route isn't supposed to be set in this process !? It's not exactly the problem described in the ietf as communication between WAN and delegating router is working !?

    The missing bit is the "default" route between WAN and Gateway that is more a matter of Neighbor Discovery !?

    I do the second test right away to check...
  25. Sinopsys

    Sinopsys Reformed Router Member

    ... done

    well I'm not sure what this is supposed to bring this second change ?

    Looks like the current dnsmasq (version 2.80test6) is supporting the off-link directive.

    With PD-Only set (which seems to be no different from unset) and customized dnsmasq I have IPv6 LAN working properly in local network: I can ping my router or any other device in the network. While connectivity with WAN remains broken:
    This is not different from the previous situation, route table is unchanged:
    I would be curious to have ND driver with debug logs to investigate this area and to understand why LAN do not discover properly route to WAN !? known ways to do it or that needs dedicated build ?
  26. Sean B.

    Sean B. LI Guru Member

    This failure is almost certainly due to the modem/router. While some service providers do use just "pd-only", it's mainly for commercial/enterprise customers that are expected to be configuring large internal infrastructure and handling their own routing anyway. With IPv6, a client/host/router interface that receives a prefix delegation must be on-link with the delegating router in order to auto set the default route. The reason for this is technical and long to explain, google if interested. Here'e the catch, your routers WAN interface counts as a hop between your LAN and the delegating router. Therefor your LAN is not on-link with that router.. hense no route. That is where the IA comes in. PD only is only the prefix, when an IA is requested as well it gives your WAN interface a /128 IPv6 IP and then the prefix passes through it to the LAN. Now your WAN interface becomes the "delegating router" and is on-link with your LAN.. and route is made. The modem/router is almost certainly causing the IA to fail.
  27. Sinopsys

    Sinopsys Reformed Router Member

    Well if that so, I'm screwed as I cannot change the router/modem and ISP isn't willing to help.
    Then I don't understand how standard routers (netgear, asus, linksys with stock firmware) can connect in IPv6 on their network ????

    I will try to get further info on the modem with ISP...
  28. Sean B.

    Sean B. LI Guru Member

    The same type of thing happens with an IPv4 modem/router. If it's not put into bridge mode your routers WAN interface gets an internal IP such as 192.168.100.X or 10.1.10.X . Speaking of which, what IPv4 IP address is your router getting from that modem/router? Is it your actual global IP, or an internal?
  29. Sinopsys

    Sinopsys Reformed Router Member

    On IPv4 i do have a public ip:

    I had a « technical expert » from ISP, feedback is;
    - modem is set as pure transparent bridge
    - their IPv6 is setup fully standard with no specifics (if that means anything)
    - it is working with most personal routers in « autoconfig » mode
    - they cannot provide further support as none of their customers ask for such technical elements (prefix length, Pf-only or not...) and I am using an exotic setup (device out of their short list + custom firmware)

    I’ll have to dig further in theoretical to then dig and explore more in details what is happening.

    Thank you anyway for your explanations and I am very open to any other advices for investigating.
  30. Sean B.

    Sean B. LI Guru Member

    Under Basic->IPv6, did you set the prefix length to 64 or 56? If 56, then try setting it to 64 and do not enable PD-only. Most provider systems will assign a requested prefix size as long as it's bigger ( smaller network size ) than what it would provide by default. If their system does allow for it and assigns a 64, I'm curious to see if it affects the issue at all.
    Last edited: Nov 9, 2018 at 4:59 PM
  31. Sinopsys

    Sinopsys Reformed Router Member

    I tried both already w and w/o PD-Only.

    I will retry with full reboot of modem & router...
  32. Sinopsys

    Sinopsys Reformed Router Member

    No change :(

    Actually this is what is confusing me: in IPv4, I have:
    - WAN (vlan2): assigned with public IP address allocated by ISP dhcp (42.2.33.XX)
    - LAN (br0): set with internal IP address I choose (
    - a default route to tell that all other addresses out of LAN should go to WAN
    So I wonder why in IPv6 it is different;
    - WAN remains with local link only as I would have expected that it get global link (public) through ND/SLAAC at first
    - LAN would have been set first with local link getting global link once PD is done

    LAN is currently reachable from outside.

    I wanted to try an old tweak forcing forwarding to 0 on WAN: surprisingly it is automatically reset to 1 :(

    I’ll try to build my R8000 package with NDP trace enabled to understand what is happening inside...
    Last edited: Nov 11, 2018 at 8:32 AM
  33. Sean B.

    Sean B. LI Guru Member

    I don't recall if the option affects IPv6 or not, but I'd suggest enabling "DHCP routes" under Advanced->Routing and see if anything changes. If you haven't already.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice