ICMP from WAN to router's LAN IP (Tomato fw)

Discussion in 'Tomato Firmware' started by roterado, Jan 26, 2014.

  1. roterado

    roterado Reformed Router Member


    I am seeing a pattern in my firewall log that I don't recognize, and I'd like to ask for your help in identifying it. It starts with a TCP SYN to my public IP to port 33434, which is dropped, and followed by 20 seconds of DROPped PINGs from the same source netblock to the routers LAN (private side) address, ie,

    Jan 26 16:49:23 x-x-x-x.isp.com kernel: DROP IN=vlan2 OUT= MACSRC=... MACDST=... MACPROTO=0800 SRC= DST=<mypublicIP> LEN=40 TOS=0x00 PREC=0x00 TTL=55 ID=55141 PROTO=TCP SPT=18009 DPT=33434 SEQ=0 ACK=0 WINDOW=5840 RES=0x00 SYN URGP=0
    Jan 26 16:49:31 x-x-x-x.isp.com kernel: DROP IN=vlan2 OUT= MACSRC=... MACDST=... MACPROTO=0800 SRC= DST= LEN=48 TOS=0x00 PREC=0x00 TTL=55 ID=55151 PROTO=ICMP TYPE=8 CODE=0 ID=20349 SEQ=33434
    Jan 26 16:49:31 x-x-x-x.isp.com kernel: DROP IN=vlan2 OUT= MACSRC=... MACDST=... MACPROTO=0800 SRC= DST= LEN=48 TOS=0x00 PREC=0x00 TTL=54 ID=55152 PROTO=ICMP TYPE=8 CODE=0 ID=34217 SEQ=33434
    Jan 26 16:49:32 x-x-x-x.isp.com kernel: DROP IN=vlan2 OUT= MACSRC=... MACDST=... MACPROTO=0800 SRC= DST= LEN=48 TOS=0x00 PREC=0x00 TTL=54 ID=55153 PROTO=ICMP TYPE=8 CODE=0 ID=34218 SEQ=33434
    Jan 26 16:49:32 x-x-x-x.isp.com kernel: DROP IN=vlan2 OUT= MACSRC=... MACDST=... MACPROTO=0800 SRC= DST= LEN=48 TOS=0x00 PREC=0x00 TTL=50 ID=55154 PROTO=ICMP TYPE=8 CODE=0 ID=11124 SEQ=33434
    Jan 26 16:49:32 x-x-x-x.isp.com kernel: DROP IN=vlan2 OUT= MACSRC=... MACDST=... MACPROTO=0800 SRC= DST= LEN=48 TOS=0x00 PREC=0x00 TTL=54 ID=55155 PROTO=ICMP TYPE=8 CODE=0 ID=34219 SEQ=33434
    ... ICMP type 8 DROPs continue for ~30s
    I have DMZ disabled, and only a handful of UDP and TCP ports forwarded to a server on the LAN, but not TCP port 33434, and none forwarded to the router itself ( I understand that linux uses UDP port 33434 for traceroutes.

    The initial TCP packet has a TTL=55, so I don't believe this is sent by tcproute, which would have a TTL=1 on the initial TCP packet.

    1. What mechanism would cause ICMP's from the WAN to be forwarded to the LAN interface, where they were
    ultimately dropped? ICMP source routed somehow?

    2. How can duplicate this type of scan from a VPS on the net?

    3. How can I close this hole? I will verify it's closed by running the scan myself from a VPS.

    I don't know who is or what their intentions are, but I would like to defend myself against such scans from any other IP. This netblock belongs to a VPS hosting company in Montreal, so it's possibly a script kiddie, or worse.

    I use Tomato Firmware v1.28.7498 MIPSR2-Toastman-VLAN-RT K26 USB Ext

    Thank you kindly
  2. koitsu

    koitsu Network Guru Member

    TCP port 33434 is often used by TCP-based traceroute as a starting port number in some operating systems. It looks to me like someone ran a TCP-based traceroute to you, followed possibly by some ICMP pings. Read everything I have written below.

    First the TCP port 33434 "probe": what are you worried about? The packets in question are being dropped. "I want to defend myself against such scans". If these type of logged events bother you, then stop logging denies in your iptables rules/Tomato configuration. Really, it's that simple. This kind of thing happens on the Internet constantly. I've recommended this to other people here as well: stop being so OCD about it, else if it bothers you **that much**, then contact abuse@planethoster.net (per ARIN) and complain. But before doing that, keep reading.

    Regarding the subsequent ICMPs: ask yourself how an Internet router would properly route an IP packet with a IP destination within an IANA registered network block ( The question then becomes how exactly did such a packet actually arrive on your WAN interface. Step back for a moment and think about it. How would the router/gateway for know to send packets with a destination address of to _you_?

    The "DST" field in iptables log is sometimes confusing, as certain kinds of logging events are logged either with the wrong destination or with an "unexpected" destination as a result of certain kinds of NAT translation being applied. I remember reading/hearing something about this somewhere, and I believe it relates to the older 2.6 kernels or iptables implementation; there is some quirk/bug/oddity with regards to certain kinds of logged traffic having wrong info in it.

    iptables logging is not sufficient evidence of anything. What you actually need are packet captures (with tcpdump) being done on your WAN interface (-i parameter).

    If you can get packet captures and have confirmed that the IP dst field does in fact contain IANA-reserved addresses, then you absolutely should contact your own ISP and discuss this matter with them. Someone somewhere may need to be introduced to BCP38, as if the packets arriving on your WAN really do have as a destination address in them, then you need to ask your ISP how exactly these packets are making it across your WAN connection (not "to you WAN IP", but across the physical wire of your WAN).

    If you think this is impossible, let me assure you it's not: I had co-located servers with a co-lo provider in California. One day, I started seeing ~20-30mbit/sec of traffic inbound to my switch (connected to them via an uplink cable), except my switch was discarding all of the traffic since it wasn't really destined to an IP address that was part of my related netblocks (re: ARP/MAC caching). I set up port mirroring to a spare port and a spare system and captured the packets coming across the uplink interface to find out what this massive amount of traffic was that didn't actually make it to any of my servers. The packets were TCP packets destined to some completely unrelated IP address (not part of my netblock), but were part of a network block for another customer within the same co-location. I informed my co-location provider of this, and they investigated. Their first response was "that's impossible", so I gave them the packet captures. A few days went by and they admitted to having misconfigured a VLAN on their own switches, so I was getting traffic that was intended for a completely different customer (some company that did image hosting).

    Otherwise if packet captures do not show what I describe, then you have some convoluted Tomato router setup/configuration that isn't being disclosed, e.g. you use a VPN, you run some kind of strange setup, VLANs, or who knows what else. I can only speculate.

    My guess is that the traffic you're seeing dropped might actually have been induced by something a system on your own LAN (assuming does exist and is a legitimate machine) had instigated, i.e. you're running some program that connects to some service, and there are certain kinds of traffic that's NAT'd (tracked within the Linux iptables conntrack or the NAT layer -- ICMP tracking is supported!) that it's able to cross-reference/refer to hence the DST being within your LAN, but the response packets are getting dropped for whatever reason (see previous paragraph).

    One such thing I can think of that might induce this kind of "probe" are certain IRC networks which run a type of "scan" (from either the server itself or a separate/dedicated server or server pool elsewhere) to any IRC client that's connecting.

    You may have to do packet captures across your WAN and LAN interfaces (requiring two tcpdump captures simultaneously) and start analysing all of the traffic yourself. This is very time-consuming and tedious, FYI, and you will need to have decent familiarity with how IP works ditto with NAT. It's very, very easy to get lost very quickly.

    I cannot help past this point. Good luck.
    Last edited: Jan 27, 2014
    roterado likes this.
  3. roterado

    roterado Reformed Router Member


    Thank you very much for your reply. It's exactly your kind of insight I was hoping to find.

    The TCP DROPs in the log are very helpful, they are not a reason to be bothered, I mentioned the TCP port 33434 drop because it provides context, as the follow-up ICMPs are from the same source.

    I do plan on setting up a capture on the WAN side, to help explain what the dropped ICMP's contain, it will take some rewiring with a mirror port, not unlike your setup.

    The address does exist, it is the LAN-side address of the router (br0 interface). The router itself runs the administrative interface (Tomato UI) on this address, and this is why I'm interested to see how another host on the WAN side can reach it at all.

    Thanks for your mention of a kernel quirk. Using your explanation as keywords, I found another post by a user here with the same problem as mine, and one you also helped with in Oct 2012: "Firewall log and ICMP dest - why local IP address?". You used very similar wording in your response then, which is how google was able to zero in on it. That thread mentions two online pingers that also cause the same kind of firewall drops as I mentioned initially. Great, now I can easily reproduce the DROPs.

    In that thread, USNetboy explained that the ICMP traffic from the WAN side gets translated by the WANPREROUTING chain of the nat table. My router is set up to respond to ICMP, but the DROPs may be due to the limit of 1 ICMP per second being exceeded.

    Using your feedback above as well as that other thread I figured out a set of ping parameters (7 packets at 0.2s intervals) that, when directed to my public IP from elsewhere, generate exactly one DENY, so now I have a way to reproduce the condition.

    Knowing what I've learnt, I don't think these scans are reaching my LAN, and also I agree it's unlikely that my ISP is sending me IANA private traffic.

    Thank you very much for your help
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice