1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Trying 1.us.pool.ntp.org []:Timeout (ping OK)

Discussion in 'Tomato Firmware' started by windozer, Feb 20, 2013.

  1. windozer

    windozer LI Guru Member

    I can't seem to get my asus RT-N16 router time updated while using any tomato mod since i think last year. It makes logging into openvpn problematic.
    • I have tried Toastman, Shibby, and Victec K26 MIPSR2 - same result.
    • Everytime I flash, I reset nvram, update, and reset nvram again.
    • I have flashed/upgraded tomato mods more than 5-6 times atleast
    • I tried dd-wrt and asuswrt-merlin and time updates - no problem!

    Below, pinging ntp server works, but ntpc command results in timeout.
    root@unknown:/tmp# ping google.com
    PING google.com ( 56 data bytes
    64 bytes from seq=0 ttl=45 time=282.487 ms
    64 bytes from seq=1 ttl=45 time=154.510 ms
    64 bytes from seq=2 ttl=45 time=154.625 ms
    64 bytes from seq=3 ttl=45 time=153.564 ms
    64 bytes from seq=4 ttl=45 time=154.728 ms
    64 bytes from seq=5 ttl=45 time=278.313 ms
    64 bytes from seq=6 ttl=45 time=154.545 ms
    64 bytes from seq=7 ttl=45 time=277.910 ms
    --- google.com ping statistics ---
    8 packets transmitted, 8 packets received, 0% packet loss
    round-trip min/avg/max = 153.564/201.335/282.487 ms
    root@unknown:/tmp# ntps
    -sh: ntps: not found
    root@unknown:/tmp# ntpc
    Usage: ntpc <server>
    root@unknown:/tmp# ping 0.pool.ntp.org
    PING 0.pool.ntp.org ( 56 data bytes
    64 bytes from seq=0 ttl=49 time=215.367 ms
    64 bytes from seq=1 ttl=49 time=216.961 ms
    64 bytes from seq=2 ttl=49 time=214.737 ms
    --- 0.pool.ntp.org ping statistics ---
    3 packets transmitted, 3 packets received, 0% packet loss
    round-trip min/avg/max = 214.737/215.688/216.961 ms
    root@unknown:/tmp# ntpc 0.pool.ntp.org
    Trying 0.pool.ntp.org []:Timeout
    root@unknown:/tmp# ping asia.pool.ntp.org
    PING asia.pool.ntp.org ( 56 data bytes
    64 bytes from seq=0 ttl=47 time=101.323 ms
    64 bytes from seq=1 ttl=47 time=97.875 ms
    64 bytes from seq=2 ttl=47 time=100.766 ms
    64 bytes from seq=3 ttl=47 time=100.191 ms
    --- asia.pool.ntp.org ping statistics ---
    4 packets transmitted, 4 packets received, 0% packet loss
    round-trip min/avg/max = 97.875/100.038/101.323 ms
    root@unknown:/tmp# ntpc asia.pool.ntp.org
    Trying asia.pool.ntp.org []:Timeout
    root@unknown:/tmp# ping 1.us.pool.ntp.org
    PING 1.us.pool.ntp.org ( 56 data bytes
    64 bytes from seq=0 ttl=53 time=226.519 ms
    64 bytes from seq=1 ttl=53 time=226.350 ms
    64 bytes from seq=2 ttl=53 time=238.075 ms
    --- 1.us.pool.ntp.org ping statistics ---
    3 packets transmitted, 3 packets received, 0% packet loss
    round-trip min/avg/max = 226.350/230.314/238.075 ms
    root@unknown:/tmp# ntpc 1.us.pool.ntp.org
    Trying 1.us.pool.ntp.org []:Timeout
  2. quihong

    quihong Networkin' Nut Member

    I just did a quick test and have no issues with ntpc asia.pool.ntp.org.

    Since it's not working with any firmware, sounds like your ISP is blocking it? How about trying it from a computer itself?
  3. gfunkdave

    gfunkdave LI Guru Member

    Perhaps your ISP blocks the NTP connection (UDP port 123)?

    Also, it's probably best to use the generic USA resolver (us.pool.ntp.org) unless you have some specific reason for using the one you're using.

    You have pretty awful latency.
  4. quihong

    quihong Networkin' Nut Member

    Had to look this one up, but here is how you can test if UDP Port 123 is opened.

    root@r2d2:/tmp/home/root# opkg install nmap
    Installing nmap (6.01-4) to root...
    Downloading http://wl500g-repo.googlecode.com/svn/ipkg/openwrt/nmap_6.01-4_entware.ipk.
    Installing libpcap (1.1.1-2) to root...
    Downloading http://wl500g-repo.googlecode.com/svn/ipkg/openwrt/libpcap_1.1.1-2_entware.ipk.
    Configuring libpcap.
    Configuring nmap.
    root@r2d2:/tmp/home/root# nmap -p 123 -sU -P0 us.pool.ntp.org
    Starting Nmap 6.01 ( http://nmap.org ) at 2013-02-20 15:51 PST
    Nmap scan report for us.pool.ntp.org (
    Host is up (0.054s latency).
    Other addresses for us.pool.ntp.org (not scanned):
    123/udp open  ntp
    Nmap done: 1 IP address (1 host up) scanned in 3.56 seconds
    windozer likes this.
  5. Monk E. Boy

    Monk E. Boy Network Guru Member

    Are you using a satellite ISP by any chance?
  6. windozer

    windozer LI Guru Member

    @quihong Time simply works from other firmware on my N16; also window pc and linux satellite box
    @Mark : P no my ISP has a censorship firewall.
    @gfunkdave Yes my ISP sucks

    @all I don't know about the NTP port being blocked because (in my experience) non-Tomato firmwares on this router N16 have no trouble updating time. My (linux)satellite receiver and windows machine is setup to receive NTP time, and works.

    Can you guys see my screenshot above? The router hasn't received time in like 4 days... or how-much ever long I have it on : / I've been having this issue from last year as far as I can remember. I'm using Tomato for 10+ years : D
  7. koitsu

    koitsu Network Guru Member

    We all see the screenshot quite clearly, but there is no easy way to debug this problem with the information you've provided. In fact, I can't even advocate tcpdumps/packet captures because the traffic is purely UDP (stateless); my guess is that all it would capture is outbound UDP packets, but no response. There's no way to debug that without network engineers at your ISP assisting. Really/honest/truly.

    I can assure you I have no problem with Toastman firmware tomato-K26USB-1.28.0501.2MIPSR2Toastman-RT-N-Ext.trx on an RT-N16. ntpc works here, regardless of ntp pool and so on. My ISP is Comcast, so I do not use PPPoE (I use native DHCP).

    I can think of a few things at the firewall layer that might be jacking up your UDP packets, but this would only happen if you have some strange configuration setup (such as using a VPN, manually adjusting iptables rules, etc.). It's also possible that if you're using PPPoE for your WAN connection something there may be going on that impacts packets exceeding a certain size.

    There is always the possibility that your ISP has a filtering device of some sort upstream that's tagging packets generated by ntpc as invalid. I would really appreciate use of ntpq or ntpdc except that isn't available on the router.

    So, I would start by providing a write-up of every setting you've changed in the router (as in every setting that's changed in the GUI, or anything you're doing via the CLI) after factory defaults. You should also be doing a thorough NVRAM reset (this is not the same thing as the "clear NVRAM" checkbox when choosing Upgrade in Tomato/TomatoUSB -- I'm talking about Administration -> Configuration -> Restore Default Configuration -> Erase all data in NVRAM memory (thorough)) and manually reconfiguring the router (not via Config Backup/Restore! This will just reapply NVRAM variables that might be causing your problems!) every time you switch firmwares -- because you've obviously changed firmwares a bunch. You can, obviously, exclude SSIDs, passwords, and other things from the list. Port forwards are important, same with port triggers, so please do not exclude those.

    You also need to provide a concise explanation of your physical network topology, i.e. RT-N16 WAN port connected to SB6121 cable modem, etc.. It matters. If you're doing something like NAT-behind-NAT (i.e. your ISP gives you a router, which then connects to your RT-N16) that matters. The same if you've set up the router your ISP gave you to DMZ to your RT-N16, which then does your NAT. Like I said: all this matters.

    If you want to provide me with a packet capture anyway, that's fine. A static tcpdump binary can be found here, which you can wget + chmod 755 and use: http://multics.minidns.net/tomato/PRECOMPILED-static/tcpdump

    Assuming your WAN interface is vlan2 (if it isn't, you've obviously tinkered about with a bunch of settings in the GUI), you want to run this in one terminal window (telnet/ssh session): tcpdump -p -i vlan2 -l -n -s 0 "host"

    Then in another, issue ntpc and wait for the timeout. Once the timeout happens, you can Ctrl-C the tcpdump, and provide the full output here in a code block to retain formatting.

    DO NOT EDIT/OBFUSCATE ANY OF THE IP ADDRESSES OR INFORMATION SHOWN, INCLUDING YOUR WAN IP, FROM THE CAPTURE. I will immediately discredit whatever is shown if you edit it. I cannot stress this point enough when it comes to packet captures.

    The reason I'm using explicitly for this test is because it's one of the servers listed per us.pool.ntp.org that's currently responding (as of this writing) and I know does work with ntpc on TomatoUSB:

    root@gw:/tmp/home/root# ntpc
    Time Updated: Wed, 20 Feb 2013 22:46:30 -0800 [+1s]
    And from a FreeBSD box where I can use ntpq to look at's configured server list (this is also behind a TomatoUSB router, so it shows that NAT'ing of NTP UDP packets works fine as well):

    $ ntpq -n -c peers
        remote          refid      st t when poll reach  delay  offset  jitter
    +    2 u  352 1024  377  269.297  -10.139  14.500
    -    2 u  751 1024  377  291.203    8.852  13.188
    +    3 u  493 1024  377    1.310  -1.701  6.125
    -    3 ?  411 1024  377    2.917    2.935  2.099
    *    2 u  933 1024  377  264.853  -7.082  13.890 is also one which you reported as showing a timeout earlier, so let's stick with that one.

    This is what your packet capture, assuming you get a response, should look like (with your IP involved of course):

    22:53:47.301474 IP > NTPv4, Client, length 48
    22:53:47.565897 IP > NTPv4, Server, length 48
    You can see quite clearly that there's an outbound UDP request from my WAN IP on a dynamically allocated port number, to port 123. Roughly 0.26 seconds later a response comes back from that server with the NTP servers' response.

    My guess is that your packet capture will show the outbound packet, but no response packet.

    Anyway, provide all of this information here, concisely, and we'll do what we can to help. If it turns out to be your ISP, you should talk to them to find out why they're dropping some of your packets but not others. (Again, I can think of a few reasons for this, especially if PPPoE is used, but we do not have enough information at this time so speculation does us no good)
    windozer likes this.
  8. windozer

    windozer LI Guru Member

    I have PPPoE/A DSL internet and my own DSL modem. The RT-N16 wan ( ppp0) port goes to bridged mode TP-link TD-8816 DSL modem ( vlan2). Modem settings have VPI 1, VCI 32, CBR QoS, 1483 Bridged IP LLC.

    I chose tomato-K26USB-1.28.7501.2MIPSR2Toastman-RT-Lite.trx to rule out openvpn, dlna, etc related problems.
    After clearing nvram, I setup PPPoE user/pass, enabled dhcp & ssh, double-checked Time NTP setting is enabled, copied & chmod tcpdump. Everything else is default. (I always use own dns servers but this time I left it default)

    ip addr command:
    root@unknown:/tmp/home/root# ip addr
    1: lo: <LOOPBACK,MULTICAST,UP,10000> mtu 16436 qdisc noqueue
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet brd scope host lo
    2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
        link/ether 48:5b:39:15:40:42 brd ff:ff:ff:ff:ff:ff
    3: eth1: <BROADCAST,MULTICAST,ALLMULTI,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
        link/ether 48:5b:39:15:40:44 brd ff:ff:ff:ff:ff:ff
    7: imq0: <NOARP> mtu 1500 qdisc noop qlen 30
    8: imq1: <NOARP> mtu 1500 qdisc noop qlen 30
    9: imq2: <NOARP> mtu 1500 qdisc noop qlen 30
    22: vlan1@eth0: <BROADCAST,MULTICAST,ALLMULTI,UP,10000> mtu 1500 qdisc noqueue
        link/ether 48:5b:39:15:40:42 brd ff:ff:ff:ff:ff:ff
    23: vlan2@eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc noqueue
        link/ether 48:5b:39:15:40:43 brd ff:ff:ff:ff:ff:ff
        inet peer scope global vlan2
    24: br0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc noqueue
        link/ether 48:5b:39:15:40:42 brd ff:ff:ff:ff:ff:ff
        inet brd scope global br0
    25: ppp0: <POINTOPOINT,MULTICAST,UP,10000> mtu 1492 qdisc pfifo_fast qlen 3
        inet peer brd scope global ppp0
    tcpdump -p -i ppp0 -l -n -s 0 "host" and ntpc
    root@unknown:/tmp# tcpdump -p -i ppp0 -l -n -s 0 "host"
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on ppp0, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
    -1:-23:-56.8195 IP > NTPv4, Client, length 48
    1 packets captured
    1 packets received by filter
    0 packets dropped by kernel
  9. windozer

    windozer LI Guru Member

    Today I flashed Asuswrt-Merlin, DD-WRT, and unofficial OpenWRT !! In all 3 firmwares, I only entered pppoe user/pass and nothing else, and BAM it works.



  10. Monk E. Boy

    Monk E. Boy Network Guru Member

    Have you tried one of Teddy's builds?

    Maybe a busybox update threw things out of whack.
  11. koitsu

    koitsu Network Guru Member

    You're going to need to provide the same output as you did for TomatoUSB, but from the other firmwares, as a form of comparison. Because PPPoE is involved, this already throws a wrench into things -- I would not be surprised if this turns out to be a PPPoE-related bug. You should probably also provide iptables -L -n -v output on TomatoUSB, once before the ntpc, and once after. (The reason for before and after is that I'm looking for an increase in dropped packets counted by the INPUT chain).
  12. windozer

    windozer LI Guru Member

    I've pasted the command outputs separated by a line, like this:

    tcpdump -p -i ppp0 -l -n -s 0 "host"
    ip addr
    iptables -L -n -v
    iptables -L -n -v

    Pastebin links-
  13. koitsu

    koitsu Network Guru Member

    Err... against OpenWRT this isn't going to help track down the problem. OpenWRT not only has a completely different kernel, a completely different set of iptables/nat rules, but doesn't use use ntpc (it uses ntpclient, which is completely different/separate). I'm certain its PPPoE bits are completely different too.

    DD-WRT might be a better comparison against Tomato.

    On the other hand, the output for TomatoUSB's iptables shows that at least the firewalling stack isn't dropping any packets, so it's something else.

    Troubleshooting this would be a lot easier if I was physically there. My initial guess is that the problem is at the PPPoE layer, which I do not know how to troubleshoot in any way.
  14. windozer

    windozer LI Guru Member

    Trying 0.pool.ntp.org []:Timeout 
    Trying 0.us.pool.ntp.org []:Timeout 
  15. windozer

    windozer LI Guru Member

    This time I was testing Victek mod tomato-K26USB-1.28.9013MIPSR2-R1.0--RAF-VPN-NOCAT.trx
    Today I setup the dsl modem as pppoe with dhcp server. The RT-N16 wan is set to DHCP client. The dsl modem shows correct time and date from NTP, while Tomato has the same issue as stated above.

    rt-n16 tomato
    root@unknown:/tmp/home/root# ping 1.us.pool.ntp.org
    PING 1.us.pool.ntp.org ( 56 data bytes
    64 bytes from seq=0 ttl=41 time=309.225 ms
    64 bytes from seq=1 ttl=41 time=310.202 ms
    64 bytes from seq=2 ttl=41 time=310.426 ms
    64 bytes from seq=3 ttl=41 time=310.240 ms
    64 bytes from seq=4 ttl=41 time=309.724 ms
    --- 1.us.pool.ntp.org ping statistics ---
    5 packets transmitted, 5 packets received, 0% packet loss
    round-trip min/avg/max = 309.225/309.963/310.426 ms
    root@unknown:/tmp/home/root# ntpc 1.us.pool.ntp.org
    Trying 1.us.pool.ntp.org []:Timeout
  16. koitsu

    koitsu Network Guru Member

    None of us using RT-N16s with DHCP can reproduce this behaviour, I'm sorry to say. Verification is below.

    root@gw:/tmp/home/root# ntpc
    Time Updated: Sun, 24 Feb 2013 00:38:26 -0800 [+1s]
    root@gw:/tmp/home/root# ntpc
    Time Updated: Sun, 24 Feb 2013 00:38:46 -0800 [-1s]
    root@gw:/tmp/home/root# ntpc
    Time Updated: Sun, 24 Feb 2013 00:48:56 -0800 [+1s]
    root@gw:/tmp/home/root# ntpc
    Time Updated: Sun, 24 Feb 2013 00:49:06 -0800 [-1s]
    Note I'm using IPs here because I want to show that all the NTP servers which time out for you work fine for me. You keep using the FQDNs which return multiple A records and thus rotate every N seconds. I don't think you're paying attention to that fact -- example: your ping of 1.us.pool.ntp.org hit, while your subsequent ntpc hit This is why I've told you for testing to stick with a single IP address and not an ntp.org FQDN.

    I do appreciate you trying to rule out the PPPoE portion though -- that really does help, thank you very much.

    I think you should engage your ISP on this matter, because as I said, nobody in this thread so far is able to reproduce this issue. It may be something as simple as them having a layer 7 filtering device that has rules which are dropping packets to you, or that your ISP has very strange/odd network routing issues (i.e. your outgoing packet never makes it to the NTP server you're querying, indicating your ISP or one of their uplinks dropped it). You're going to need to reach out to actual network engineering resources at your ISP (getting through to them via technical support is difficult, I know). You should show them the tcpdumps/packet captures -- those are the most useful evidence. Those engineers should be able to see where your packets are going within their network and see if a response packet is being received by them and sent back to you.

    All I can tell you is that the physical network layer is not receiving a response to the UDP packet that is sent via your ISP (either the NTP server does not receive it, or the response packet from the NTP server never makes it to you).

    I say this with full knowledge that OpenWRT, DD-WRT, and the stock firmware work for you.

    Ultimately if this is not something you want to engage your ISP on, then use a different firmware. You have at least 3 others which you can use that do work for you, regardless of what the root cause is.
  17. koitsu

    koitsu Network Guru Member

    BTW: another option is for you to provide an actual packet capture from a working firmware (I would prefer DD-WRT), as well as a packet capture from a non-working firmware (e.g. TomatoUSB) to a consistent/static NTP server (i.e. The captures I'd need would be similar to what you provided above, except I need the actual raw packet capture data and not the decoded/summarised version. Getting that is easy as long as you can get a working tcpdump binary for both firmwares. You'd do this:

    tcpdump -p -i {waninterfacename} -s 0 -w /tmp/capture.pcap "host"

    Then in another window initiate ntpc or ntpclient (depends on the firmware of course). Once completion or a timeout happens, hit Ctrl-C in the window with tcpdump. You should then have a file called /tmp/capture.pcap on the router -- I need that file from each firmware (the one which timed out, and the one which was successful). You can offload the file on to another machine using scp, stick it on a USB drive, use a CIFS/SMB mount, or whatever other means you have available -- I simply need the raw captures from both firmwares. How you get them off the router is, politely, your own problem. :)

    Once I have that, I will need to spend some dedicated time comparing all of the IP and UDP headers and relevant data to see what the difference is. That takes me some time to do, and it may not be definitive because I imagine some of the packet contents between ntps and ntpclient differ.

    Other than this, there is only one other choice: and that's to do the exact same procedure as above, but against a public NTP server which someone on this forum runs who can also do a packet capture from their side (specifically looking for your IP address; e.g. they would do "tcpdump -p -i {interface} -l -n -s 0 "host {yourwanip}") before you initiate the ntps/ntpclient). If they never see the NTP client request packet, then your ISP or one of your ISPs uplinks is dropping the packet for whatever reason. If they do see the NTP request packet (and respond to it -- that should also show up in the server capture), but you see a timeout, then your ISP (or one of their peering providers) is dropping the response packet for whatever reason.

    I do run a public server (I used to run a co-located hosting org since roughly 1993), but I do not run a public NTP daemon. I could set one up, but I would have to be very specific with my firewall rules and know what your WAN IP is (and it may change if you change firmwares). This would take me a lot of time and I would need to spend an afternoon working with you (in real-time) on it. Most engineers charge for this sort of time -- so ask yourself if it's really worth it.
  18. windozer

    windozer LI Guru Member

    Issue solved : ) I was messing about with the router and I remember doing the following steps: Red-button reset, flashed dd-wrt(rt-n16) via ASUS firmware tool, flashed tomato through webgui , again Red-button reset.
    By red button reset I mean powering off the router, holding the red button, and turning it back on and releasing the button after 4-5 seconds (equivalent of the famous 30-30-30 reset). I guess Red-button reset is what I should have been using to clear nvram instead of doing it via web-gui.
  19. Monk E. Boy

    Monk E. Boy Network Guru Member

    Yeah the red button on an RT-N16 is WPS, which functions roughly like the reset button of yore with everything except the OEM firmware. Even then it functions like reset during power-on, it's just once the OEM firmware is loaded pressing the button will actually start a (blech) WPS operation.

Share This Page