1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

High pings on Asus RT-AC66U with Tomato by Shibby

Discussion in 'Tomato Firmware' started by mahi2003, Aug 28, 2013.

  1. mahi2003

    mahi2003 Reformed Router Member

    Recently I got myself the Asus RT-AC66U wireless router. Because I did't like the Asus user web interface I flashed Tomato by Shibby on it (version 1.28.0000 MIPSR2-112 K26 USB AIO-64K). Wow, what an improvement! Tons of features, intuitive web interface,... The Tomato developers sure did a great job!

    I do, however, have one major issue with the Tomato firmware: I get high pings everywhere. Not just through the router or with wireless connections, but also on wired gigabit connections between router and LAN.

    AC66U router: 192.168.1.1
    LAN computer: 192.168.1.10

    The computer is attached directly to one of the LAN ports on the router using a CAT6 cable and 1 Gbit/s link speed. The CPU load on both LAN computer and router was close to zero and no other devices were using the network during the tests.

    From LAN computer to router:
    Code:
    # ping -c8 192.168.1.1
    PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
    64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.385 ms
    64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=10.2 ms
    64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=8.93 ms
    64 bytes from 192.168.1.1: icmp_seq=4 ttl=64 time=7.06 ms
    64 bytes from 192.168.1.1: icmp_seq=5 ttl=64 time=0.327 ms
    64 bytes from 192.168.1.1: icmp_seq=6 ttl=64 time=6.92 ms
    64 bytes from 192.168.1.1: icmp_seq=7 ttl=64 time=4.95 ms
    64 bytes from 192.168.1.1: icmp_seq=8 ttl=64 time=0.312 ms
    
    --- 192.168.1.1 ping statistics ---
    8 packets transmitted, 8 received, 0% packet loss, time 7006ms
    rtt min/avg/max/mdev = 0.312/4.891/10.230/3.805 ms
    Code:
    # traceroute 192.168.1.1
    traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 60 byte packets
     1  hephaestus (192.168.1.1)  0.944 ms * *
    The pings are all over the place. The traceroute even timed out on the second and third query.

    From router to LAN computer:
    Code:
    # ping -c8 192.168.1.10
    PING 192.168.1.10 (192.168.1.10): 56 data bytes
    64 bytes from 192.168.1.10: seq=0 ttl=64 time=0.887 ms
    64 bytes from 192.168.1.10: seq=1 ttl=64 time=9.923 ms
    64 bytes from 192.168.1.10: seq=2 ttl=64 time=10.065 ms
    64 bytes from 192.168.1.10: seq=3 ttl=64 time=10.065 ms
    64 bytes from 192.168.1.10: seq=4 ttl=64 time=10.048 ms
    64 bytes from 192.168.1.10: seq=5 ttl=64 time=3.464 ms
    64 bytes from 192.168.1.10: seq=6 ttl=64 time=10.461 ms
    64 bytes from 192.168.1.10: seq=7 ttl=64 time=0.960 ms
    
    --- 192.168.1.10 ping statistics ---
    8 packets transmitted, 8 packets received, 0% packet loss
    round-trip min/avg/max = 0.887/6.984/10.461 ms
    Code:
    # traceroute 192.168.1.10
    traceroute to 192.168.1.10 (192.168.1.10), 30 hops max, 38 byte packets
     1  hades (192.168.1.10)  0.698 ms  8.910 ms  9.606 ms
    More of the same...

    I tried enabling/disabling QoS, disabling unecessary processes, clearing the NVRAM, reflashing Tomato, flashing the older Tomato by Shibby release (1.28.0000 MIPSR2-110 K26 USB AIO-64K without 5 GHz support),... Nothing improved the network latency.

    Then I decided to go back to the Asus firmware, or more precisely the Asuswrt-Merlin firmware. I flashed Asuswrt-Merlin version 3.0.0.4.374.32 (RT-AC66U_3.0.0.4_374.32_0) onto the router and repeated the tests:
    Code:
    # ping -c8 192.168.1.1
    PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
    64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.267 ms
    64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.293 ms
    64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=0.287 ms
    64 bytes from 192.168.1.1: icmp_seq=4 ttl=64 time=0.328 ms
    64 bytes from 192.168.1.1: icmp_seq=5 ttl=64 time=0.318 ms
    64 bytes from 192.168.1.1: icmp_seq=6 ttl=64 time=0.323 ms
    64 bytes from 192.168.1.1: icmp_seq=7 ttl=64 time=0.349 ms
    64 bytes from 192.168.1.1: icmp_seq=8 ttl=64 time=0.383 ms
    
    --- 192.168.1.1 ping statistics ---
    8 packets transmitted, 8 received, 0% packet loss, time 6995ms
    rtt min/avg/max/mdev = 0.267/0.318/0.383/0.038 ms
    Problem fixed! But now I'm back to the Asus web interface I don't like.

    I do understand that by using third part firmwares like Tomato I might not get the same performance out of the router as the original manufacturer, but this can't be right... Overall performance of Tomato on the RT-AC66U is actually very good. I can download large files from the Internet at speeds very near the limits of my 120 Mbit/s connection, wired LAN large file transfers are just as fast as going over my dedicated switch and the wireless performance blows my old router away in both speed and range (although I only use the 2.4 GHz band). It's just the latency that is much, much higher.

    Can other RT-AC66U/Tomato owners confirm this issue? Can RT-N66U/Tomato owners confirm their router does not show this issue?

    I'm quite sure this issue is not inherent to Tomato or even Tomato by Shibby. I found a topic at smallnetbuilder.com, High ping times to RT-AC66U with 3.0.0.354.27, where 2 users experience a similar or perhaps even the same issue but with a stock Asus firmware. I couldn't find this specific firmware version, but I tried the nearest Asuswrt-Merlin firmware (RT-AC66U_3.0.0.4_354.29-BETA1) and got the same results.

    So to summarize:
    • Asus/Merlin 3.0.0.4.270.24 with driver 6.30.39.31 (r341183): OK
    • Asus/Merlin 3.0.0.4.354.29 with driver 6.30.102.9 (r366174): HIGH PINGS
    • Asus/Merlin 3.0.0.4.374.32 with driver 6.30.102.9 (r366174): OK
    • Tomato/Shibby 1.28 MIPSR2-110 with driver 6.30.39.31 (r341183): HIGH PINGS
    • Tomato/Shibby 1.28 MIPSR2-112 with driver 6.30.102.9 (r366174): HIGH PINGS
    First I thought the issue could be related to the driver version of the Broadcom chip, but apparently Asus, Merlin and Shibby all use the same driver version in their latest stable release.

    Any ideas what could be going on?
     
  2. RMerlin

    RMerlin Network Guru Member

    I know Asus resolved the high ping issue that was introduced with the 3xx FW line, however I'm not sure what the actual fix was. I would start by replacing the Ethernet switch driver to see if it helps.

    The version number doesn't tell the whole tale, I suspect it could be a tweak that Asus did to either the code or to how they compile it. The version typically only change when they get newer code from Broadcom from what I understand.
     
  3. koitsu

    koitsu Network Guru Member

    I cannot reproduce this problem using Toastman's firmware tomato-K26USB-1.28.0502.8MIPSR2Toastman-RT-N-Ext.trx for my Asus RT-N16, however I am using a specific GUI feature and some /proc tunable adjustments that may play a role (keep reading):

    Code:
    root@gw:/tmp/home/root# ping 192.168.1.50
    PING 192.168.1.50 (192.168.1.50): 56 data bytes
    64 bytes from 192.168.1.50: seq=0 ttl=64 time=0.448 ms
    64 bytes from 192.168.1.50: seq=1 ttl=64 time=0.396 ms
    64 bytes from 192.168.1.50: seq=2 ttl=64 time=0.363 ms
    64 bytes from 192.168.1.50: seq=3 ttl=64 time=0.353 ms
    64 bytes from 192.168.1.50: seq=4 ttl=64 time=0.359 ms
    64 bytes from 192.168.1.50: seq=5 ttl=64 time=0.355 ms
    64 bytes from 192.168.1.50: seq=6 ttl=64 time=0.358 ms
    64 bytes from 192.168.1.50: seq=7 ttl=64 time=0.354 ms
    64 bytes from 192.168.1.50: seq=8 ttl=64 time=0.358 ms
    64 bytes from 192.168.1.50: seq=9 ttl=64 time=0.354 ms
    64 bytes from 192.168.1.50: seq=10 ttl=64 time=0.360 ms
    64 bytes from 192.168.1.50: seq=11 ttl=64 time=0.433 ms
    64 bytes from 192.168.1.50: seq=12 ttl=64 time=0.357 ms
    64 bytes from 192.168.1.50: seq=13 ttl=64 time=0.362 ms
    64 bytes from 192.168.1.50: seq=14 ttl=64 time=0.361 ms
    64 bytes from 192.168.1.50: seq=15 ttl=64 time=0.361 ms
    64 bytes from 192.168.1.50: seq=16 ttl=64 time=0.360 ms
    64 bytes from 192.168.1.50: seq=17 ttl=64 time=0.360 ms
    64 bytes from 192.168.1.50: seq=18 ttl=64 time=0.362 ms
    
    --- 192.168.1.50 ping statistics ---
    19 packets transmitted, 19 packets received, 0% packet loss
    round-trip min/avg/max = 0.353/0.369/0.448 ms
    
    The fact that you get really good throughput (120mbit/sec, etc.) for Internet-bound transfers indicates to me that the router may be either applying rate-limiting to ICMP packets, or may be prioritising forwarded traffic (i.e. between a LAN system and the Internet).

    So, that said:

    a) Go to Advanced / Firewall and uncheck the box labelled Limit PPS, click Save, then re-do your ping test,

    b) If there is no such box, please telnet into the router and issue the following commands the re-do your ping test:

    echo 0 > /proc/sys/net/ipv4/icmp_ratelimit
    echo 0 > /proc/sys/net/ipv6/icmp/ratelimit

    Note the 2nd line is not a typo; the path really is different. If your firmware does not have IPv6 then that command may result in an error which you can ignore.

    And if you do (b), please also provide output from the following command: iptables -L -n -v --line-numbers

    My gut feeling is that (a) will fix your problem, and if it does, I can explain in further detail how/why (I was one of the few people who came across this ordeal in the first place :)) if you ask me (otherwise I will not explain).

    Also, you do not need to test pinging from PC->router. Pinging from router->PC is all you need in this case because 1) ping is bidirectional in the sense that the ping client sends ICMP ECHO and the recipient sends back an ICMP ECHO_REPLY and 2) (more important) we know there is no asymmetric routing involved on your network.
     
  4. mahi2003

    mahi2003 Reformed Router Member

    Thank you for your replies!
    I don't think I can do this without recompiling everything, right? It's highly unlikely simply replacing the driver module (is it et.ko?) with one from another firmware (like yours) is going to work, isn't it? If this means recompiling, I'm afraid that even with your and Shibby's sources, I would not get very far... :(

    I'm not an expert in this matter, but doesn't the ICMP rate limit apply only to the WAN interface? For now I concentrated on the local LAN (less external/uncontrollable factors).

    Anyway, the "Limit per second" is greyed out because "Respond to ICMP ping" is unchecked. I checked the values of /proc/sys/net/ipv4/icmp_ratelimit and /proc/sys/net/ipv6/icmp/ratelimit and both were set to 100. Reset to 0 and tested again. No changes - still high pings.

    This is right after an NVRAM reset (and reboot obviously):
    Code:
    # iptables -L -n -v --line-numbers
    Chain INPUT (policy DROP 1 packets, 1400 bytes)
    num   pkts bytes target     prot opt in     out     source               destination        
    1        0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           state INVALID
    2       37  5772 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED
    3        1    48 shlimit    tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:22 state NEW
    4        0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0          
    5       26  2340 ACCEPT     all  --  br0    *       0.0.0.0/0            0.0.0.0/0          
    6        0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp spt:67 dpt:68
    
    Chain FORWARD (policy DROP 0 packets, 0 bytes)
    num   pkts bytes target     prot opt in     out     source               destination        
    1        2    86            all  --  *      *       0.0.0.0/0            0.0.0.0/0           account: network/netmask: 192.168.1.0/255.255.255.0 name: lan
    2        0     0 ACCEPT     all  --  br0    br0     0.0.0.0/0            0.0.0.0/0          
    3        0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           state INVALID
    4        0     0 TCPMSS     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp flags:0x06/0x02 TCPMSS clamp to PMTU
    5        2    86 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED
    6        0     0 wanin      all  --  vlan2  *       0.0.0.0/0            0.0.0.0/0          
    7        0     0 wanout     all  --  *      vlan2   0.0.0.0/0            0.0.0.0/0          
    8        0     0 ACCEPT     all  --  br0    *       0.0.0.0/0            0.0.0.0/0          
    
    Chain OUTPUT (policy ACCEPT 75 packets, 8118 bytes)
    num   pkts bytes target     prot opt in     out     source               destination        
    
    Chain shlimit (1 references)
    num   pkts bytes target     prot opt in     out     source               destination        
    1        1    48            all  --  *      *       0.0.0.0/0            0.0.0.0/0           recent: SET name: shlimit side: source
    2        0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           recent: UPDATE seconds: 60 hit_count: 4 name: shlimit side: source
    
    Chain wanin (1 references)
    num   pkts bytes target     prot opt in     out     source               destination        
    
    Chain wanout (1 references)
    num   pkts bytes target     prot opt in     out     source               destination
     
  5. Victek

    Victek Network Guru Member

    I'm now merging RT-AC66U in Tomato RAF, not ready yet (some problems with ac BW selection box...), ping test from one beta user on his local network:

    Code:
    root@RT-AC66:/tmp/home/root# ping 10.10.2.2
    PING 10.10.2.2 (10.10.2.2): 56 data bytes
    64 bytes from 10.10.2.2: seq=0 ttl=64 time=0.421 ms
    64 bytes from 10.10.2.2: seq=1 ttl=64 time=0.404 ms
    64 bytes from 10.10.2.2: seq=2 ttl=64 time=0.365 ms
    64 bytes from 10.10.2.2: seq=3 ttl=64 time=0.363 ms
    64 bytes from 10.10.2.2: seq=4 ttl=64 time=0.357 ms
    64 bytes from 10.10.2.2: seq=5 ttl=64 time=0.357 ms
    64 bytes from 10.10.2.2: seq=6 ttl=64 time=0.357 ms
    
    --- 10.10.2.2 ping statistics ---
    7 packets transmitted, 7 packets received, 0% packet loss
    round-trip min/avg/max = 0.357/0.374/0.421 ms
    
     
    Last edited: Aug 28, 2013
  6. koitsu

    koitsu Network Guru Member

    The GUI option I'm describing applies only to the WAN interface, and the rate-limiting there is done via an iptables rule. I had forgotten that the GUI option is greyed out unless the "Response to ICMP ping" box is checked. My apologies.

    However, the kernel itself also has ICMP rate-limiting capabilities, and those apply universally (not per-interface, but to how the kernel handles ICMP universally (i.e. all interfaces)). But this doesn't matter, because:

    Okay, good to know. There's nothing indicative of problems in your iptables output either (no sign of any rate-limiting either, though that wouldn't show up given the aforementioned GUI stuff), so the issue is elsewhere.

    I'm sorry to say I do not know where to look. I would suggest following other peoples' recommendations at this point -- examples include trying an older/different Shibby firmware version, trying a different branch (ex. Toastman or Victek), or what @RMerlin said.

    Sorry I can't be of much help -- I was hoping this was one of the above two situations, but it's not.

    Only other ideas I can think of in the meantime:

    How long is the Ethernet cable between the router and the PC? Metres or feet/inches would be helpful. I'll note up front you might be thinking "why would that matter if I switch back to the Asus firmware and the problem is gone?" -- the issue may be related to the switching fabric driver using features like "Green Ethernet" (not the same as PoE) or other such driver and IC-level features, where the amount of power pushed across the wire is diminished, so long cable runs are more likely to be impacted.

    If the cable length is less than 250 feet then the issue is unrelated to that.
     
  7. RMerlin

    RMerlin Network Guru Member

    The issue he's referring to was specific to the RT-AC66U. It was an actual bug that I discussed with Asus at the time, and which they have since fixed. It did not affect the N16/N66U, which are on SDK 5.x.
     
    koitsu likes this.
  8. RMerlin

    RMerlin Network Guru Member

    Correct. My reply was more targeted at Shibby/Victek than you. Just confirming that this was a known issue in past Asus versions.
     
  9. mahi2003

    mahi2003 Reformed Router Member

    Everyone, thanks again for your replies!
    That looks perfect and exactly what one would expect from a local LAN!

    Is there a chance I can get access to your RT-AC66U test build? For certain features I prefer to stay with Shibby's build (sorry), but I'd like to make sure the problem is really with the Shibby build and not my router/network. If your firmware fixes the high ping issue for me, I'm sure Shibby can fix it too in a future release!
    Well, my house is not exactly a mansion ;). The tests were performed with a ~2 meter CAT6 cable directly between computer and router. Because that's a cable I crimped myself I also briefly tested with the weird "flat" Ethernet cable that came with the RT-AC66U (~1 meter, unknown spec but probably CAT5E) but the results were exactly the same. I do not want to rule out my hardware (I've had some weird stuff in the past with certain network component combinations), but at this moment my guess is that Shibby overlooked a driver change that Merlin and Victek did not.
     
  10. Victek

    Victek Network Guru Member

    Hope to release it this week..
     
  11. mahi2003

    mahi2003 Reformed Router Member

    Thanks! I'm looking forward to it!
     
  12. Victek

    Victek Network Guru Member

  13. mahi2003

    mahi2003 Reformed Router Member

    Victek: I flashed your firmware, but the router refuses to boot afterwards. I made sure to erase the NVRAM, put the router in recovery mode and flashed the firmware using the Asus Firmware Restoration utility (same tool I use for switching between Tomato by Shibby and Asuswrt-Merlin). Immediately after the flashing completes the router restarts but seems to hang during the boot. Even after 10 minutes the router is not listening at 192.168.1.1 and the only LEDs that are on are power, LAN1 and WAN (actually those are the same LEDs that show when in recovery mode but I'm not sure that's related). I then decided to power cycle it manually, but it got stuck in the same condition. Finally I tried reflashing your firmware, but still no go :(...

    Back to Shibby's now. If you need any information from the router, let me know!
     
  14. Victek

    Victek Network Guru Member

    Thanks, have to check ....weird!
     
  15. shibby20

    shibby20 Network Guru Member

    i even didn`t know about this issue. Thx for info. I checked myself and i also have ping about 6-9ms over LAN. I will try to fix it.
     
  16. mahi2003

    mahi2003 Reformed Router Member

    Thank you Shibby!

    PS: It's not really of any importance, but the Flash Size is reported incorrectly on the Status > Overview page. It says 1MB...
     
  17. shibby20

    shibby20 Network Guru Member

    this router has double-flash bone (sflash 2MB and NAND flash 128MB). This is why flash size is wrong detected.
     
  18. mahi2003

    mahi2003 Reformed Router Member

    Quick update to this thread: Victek's latest beta Tomato RAF for the RT-AC66U (1.2b at the moment of writing) fixes the high ping issue! :cool:
     
  19. Sp0wn971

    Sp0wn971 Reformed Router Member

    Hello Shibby20, do you know when you will be able to fix this high ping issue ?

    regards
     
  20. shibby20

    shibby20 Network Guru Member

    Victek solved this problem. I already have his code and it`s really working. It will be fixed in next release.
     
    Last edited: Sep 22, 2013
  21. Sp0wn971

    Sp0wn971 Reformed Router Member

    Great news :) , thx
     
  22. CowMix

    CowMix Addicted to LI Member

    Hey shibby,

    I was just wondering when we can expect a build with the high ping fixed, No rush just wondering. Thanks :)
     

Share This Page