1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Services behind OpenVPN client on second router unreachable from outside

Discussion in 'Tomato Firmware' started by MassiveCollision, Sep 14, 2013.

  1. MassiveCollision

    MassiveCollision Reformed Router Member

    I want to be able to connect to my ISP IP address of the first router and then forward that traffic to the NAS behind an OpenVPN client on a second router. It does work after disabling the OpenVPN client but that would defeat the purpose. It looks like the router is sending the packets back through the OpenVPN connection. I ultimately want to reach services on TCP ports 5001 and 8080 on the NAS.

    Situation

    [​IMG]


    My Fritz is connected to my ISP and it runs a local wired and wireless network.

    The LAN1 on the Fritz is connected to the WAN on the ASUS.

    The ASUS router runs the recent Tomato-K26USB-1.28.7502.1MIPSR2Toastman-RT-VPN-NOCAT firmware. The Fritz runs stock firmware.

    The whole purpose of my setup is to run one extra network that goes through a VPN service. This way I can browse anonymously, circumvent geo restrictions and all the rest. I can switch networks with my wireless devices in order to change from my ISP IP to my VPN IP.

    The Fritz hands out IP addresses in the 192.168.178.x range.
    The ASUS in the 172.16.1.x range.

    Connection between networks

    When connected to the ASUS I can access the web GUI of the Fritz by going to 192.168.178.1, I can also connect to or ssh into my Apple TV on the other network by connecting to 192.168.178.3.

    Similarly, when connected to my main Fritz network, I can connect to my NAS by going to 172.16.1.2:5001. Also, the Apple TV can stream from the XBMC library that is on the NAS fine. I can't however connect to the web GUI of the ASUS on 172.16.1.1. This just hangs for a while and then stops.

    Port forwarding and reaching my NAS
    • The ASUS Tomato router is on 'Router' mode.
    • NAT loopback on the Tomato is enabled for 'All'.
    • Accept DNS configuration on the Tomato is on 'strict'.
    • There's a static ip routing table entry in my Fritz for the 172.16.1.0 network with a 192.168.178.2 gateway and 255.255.255.0 subnet mask. This enables devices on both networks to reach each other.
    • I port forward TCP 5001 and 8080 from the Fritz to the ASUS router, and then again from that router to the NAS on 172.16.1.2.
    • DNS on the ASUS Tomato is now on 4.2.2.1 or 8.8.8.8.
    • I also forward UDP 1194 in my Fritz to the ASUS which permanently runs an OpenVPN client in Tomato. No problems here.
    • Port forwarding to other devices on my Fritz network works fine too.
    Trying to reach my NAS from the outside using this setup now just hangs for a while and then stops, giving this dropped package warning in logs:
    Code:
    unknown user.warn kernel: DROP IN=vlan2 OUT= MACSRC=c0:25:06:7a:55:36 MACDST=60:a4:4c:65:fd:e9 MACPROTO=0800 SRC=<redacted VPN IP> DST=192.168.178.2 LEN=64 TOS=0x00 PREC=0x00 TTL=54 ID=23004 DF PROTO=TCP SPT=61122 DPT=5001 SEQ=2689429389 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (02040558010303040101080A3CB714E10000000004020000)
    Traffic comes in through vlan2 (WAN) but gets dropped.

    Do I maybe need UPnP or NAT-PMP configured?

    Scripts

    I left my Firewall script empty. I'd like to know what I need to put there, if anything. Should I put something like:
    Code:
    iptables -I FORWARD -i tun11 -p tcp -d 172.16.1.2 --dport 5001 -j ACCEPT
    iptables -I FORWARD -i tun11 -p udp -d 172.16.1.2 --dport 5001 -j ACCEPT
    iptables -t nat -I PREROUTING -i tun11 -p tcp --dport 5001 -j DNAT --to-destination 172.16.1.2
    iptables -t nat -I PREROUTING -i tun11 -p udp --dport 5001 -j DNAT --to-destination 172.16.1.2
    
    Or should I use a different IP here? Or are these wrong rules? There are lots or different ones posted online.

    I've tried several WAN up scripts, including the most promising one:
    http://www.linksysinfo.org/index.php?threads/route-only-specific-ports-through-vpn-openvpn.37240/

    I had hoped it would work by letting the ports I needed bypass the VPN.

    This is my script:
    Code:
    #
    # First it is necessary to disable Reverse Path Filtering on all
    # current and future network interfaces:
    #
    for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do
      echo 0 > $i
    done
    
    #
    # Delete and table 100 and flush any existing rules if they exist.
    #
    ip route flush table 100
    ip route del default table 100
    ip rule del fwmark 1 table 100
    ip route flush cache
    iptables -t mangle -F PREROUTING
    
    #
    # Copy all non-default and non-VPN related routes from the main table into table 100.
    # Then configure table 100 to route all traffic out the WAN gateway and assign it mark "1"
    #
    # NOTE: Here I assume the OpenVPN tunnel is named "tun11".
    #
    #
    ip route show table main | grep -Ev ^default | grep -Ev tun11 \
      | while read ROUTE ; do
          ip route add table 100 $ROUTE
    done
    ip route add default table 100 via $(nvram get wan_gateway)
    ip rule add fwmark 1 table 100
    ip route flush cache
    
    #  Ports 5001 and 8080 will bypass the VPN
    iptables -t mangle -A PREROUTING -i br0 -p tcp -m multiport --dport 5001,8080 -j MARK --set-mark 1
    I think it's bypassing those ports, but it still shows the same error as above.

    I've also tried bypassing all traffic with the following line to test if this would work, which it doesn't.
    Code:
    # By default all traffic bypasses the VPN
    iptables -t mangle -A PREROUTING -i br0 -j MARK --set-mark 1
    Any ideas? I've been pulling my hair out over this for a couple of weeks now, it's driving my nuts. I've tried many of the things here on the forums, but I don't understand iptables enough to hack something together that works for me. If there is any more info I need to provide, let me know.
     
  2. Almaz

    Almaz Serious Server Member

    someone might help you if you provide the output of the command of ifconfig
     
  3. MassiveCollision

    MassiveCollision Reformed Router Member

    This is my ifconfig:
    Code:
    br0        Link encap:Ethernet  HWaddr 60:A4:4C:65:FD:E8
              inet addr:172.16.1.1  Bcast:172.16.1.255  Mask:255.255.255.0
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:4496 errors:0 dropped:0 overruns:0 frame:0
              TX packets:3519 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:806733 (787.8 KiB)  TX bytes:1604654 (1.5 MiB)
    
    eth0      Link encap:Ethernet  HWaddr 60:A4:4C:65:FD:E8
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:4776 errors:0 dropped:0 overruns:0 frame:0
              TX packets:4604 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:1836509 (1.7 MiB)  TX bytes:1018706 (994.8 KiB)
              Interrupt:4 Base address:0x2000
    
    eth1      Link encap:Ethernet  HWaddr 60:A4:4C:65:FD:EA
              UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
              RX packets:4197 errors:0 dropped:0 overruns:0 frame:219876
              TX packets:4289 errors:5 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:796647 (777.9 KiB)  TX bytes:1904192 (1.8 MiB)
              Interrupt:3 Base address:0x1000
    
    lo        Link encap:Local Loopback
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING MULTICAST  MTU:16436  Metric:1
              RX packets:15 errors:0 dropped:0 overruns:0 frame:0
              TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:2244 (2.1 KiB)  TX bytes:2244 (2.1 KiB)
    
    tun11      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
              inet addr:10.186.1.6  P-t-P:10.186.1.5  Mask:255.255.255.255
              UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
              RX packets:2394 errors:0 dropped:0 overruns:0 frame:0
              TX packets:2215 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:100
              RX bytes:1262656 (1.2 MiB)  TX bytes:380189 (371.2 KiB)
    
    vlan1      Link encap:Ethernet  HWaddr 60:A4:4C:65:FD:E8
              UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
              RX packets:329 errors:0 dropped:0 overruns:0 frame:0
              TX packets:1766 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:74014 (72.2 KiB)  TX bytes:350088 (341.8 KiB)
    
    vlan2      Link encap:Ethernet  HWaddr 60:A4:4C:65:FD:E9
              inet addr:192.168.178.2  Bcast:192.168.178.255  Mask:255.255.255.0
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:4447 errors:0 dropped:0 overruns:0 frame:0
              TX packets:2838 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:1676527 (1.5 MiB)  TX bytes:668618 (652.9 KiB)
    And this is the output of 'iptables -t mangle -L PREROUTING':
    Code:
    Chain PREROUTING (policy ACCEPT)
    target    prot opt source              destination      
    MARK      tcp  --  anywhere            anywhere            multiport dports 5001,webcache MARK set 0x1 
    Hope that is more informative. Thanks.
     
  4. dc361

    dc361 LI Guru Member

    That's a lot of frame errors on ETH1 -- bad cable?
     
  5. koitsu

    koitsu Network Guru Member

    eth1 is the wireless interface; you should expect some number of errors, and an even larger number of framing errors. Reason: there is nothing reliable about wireless. Ever. For example, on my RT-N16 with a 101 day uptime:

    Code:
    root@gw:/tmp/home/root# uptime
    08:33:42 up 101 days, 13:22,  load average: 0.00, 0.00, 0.00
    
    root@gw:/tmp/home/root# ifconfig eth1
    eth1  Link encap:Ethernet  HWaddr 10:BF:48:E6:F4:6B
      UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
      RX packets:420510 errors:88 dropped:0 overruns:0 frame:620126468
      TX packets:1683587 errors:126 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000
      RX bytes:112815563 (107.5 MiB)  TX bytes:1041610314 (993.3 MiB)
      Interrupt:3 Base address:0x1000
    
    Proof of what I say about eth1 being wireless:

    Code:
    root@gw:/tmp/home/root# dmesg | egrep ^eth1
    eth1: Broadcom BCM4329 802.11 Wireless Controller 5.100.138.20
    
    TL;DR -- The TX error count on his eth1 has nothing to do with the issue being discussed.
     
    dc361 likes this.
  6. MassiveCollision

    MassiveCollision Reformed Router Member

    Appreciate the info and pointers so far, but I'm still pulling my hair out over this.

    I've now reflashed the firmware, did a full 30/30/30 and everything. Also tried Shibby's build for a bit without success.

    After doing the reset of the Tomato router and starting again I've ran into more problems. I want to try and get this working without OpenVPN first now. So just a normal 2 router setup, 2 subnets, port forwarding from the outside to devices behind the 2nd router. No WAN up or Firewall scripts yet.

    Without anything in my Firewall script, I get this error when connecting to 83.xxx.xxx.xxx:5001:
    Code:
    unknown user.warn kernel: DROP IN=vlan2 OUT= MACSRC=c0:25:06:7a:55:36 MACDST=60:a4:4c:65:fd:e9 MACPROTO=0800 SRC=83.xxx.xxx.xxx DST=192.168.178.2 LEN=64 TOS=0x00 PREC=0x00 TTL=61 ID=30867 DF PROTO=TCP SPT=60109 DPT=5001 SEQ=260652994 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020405AC010303040101080A3F9F1CB30000000004020000)
    The following lines in my Firewall script get rid of this error when trying to connect but the problem persists:
    Code:
    iptables -I INPUT -p tcp --dport 5001 -j ACCEPT
    iptables -I INPUT -p tcp --dport 8080 -j ACCEPT
    Main router:
    - The main router has TCP ports 5001 and 8080 forwarded to 192.168.178.2 (WAN IP of Tomato).
    - The main router has a static route to network 172.16.1.0, subnetmask 255.255.255.0, and gateway of 192.168.178.2 (WAN IP of Tomato router).

    Tomato:
    - The Tomato is in 'Router' mode. (NAT disabled, right?)
    - Advanced > Firewall on the Tomato has 'NAT Loopback: ALL' and 'NAT target: MASQUERADE'.
    - The Tomato has TCP ports 5001 and 8080 forwarded to LAN IP of destination (172.16.1.2)
    - I also can't remotely access the Tomato's GUI from the main LAN. (I have remote access turned on, HTTP on port 8082, 192.168.178.2:8082). Gives me this error in connection logs:
    Code:
    unknown user.warn kernel: DROP IN=vlan2 OUT= MACSRC=68:a8:6d:47:90:12 MACDST=60:a4:4c:65:fd:e9 MACPROTO=0800 SRC=192.168.178.21 DST=192.168.178.2 LEN=64 TOS=0x00 PREC=0x00 TTL=64 ID=51912 DF PROTO=TCP SPT=59717 DPT=8082 SEQ=4264908586 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020405B4010303040101080A3F92CDBC0000000004020000)
    Is it possible here that my first router can't do NAT for other subnets, only for its own subnet? Or is any other problem with my first router a possibility to begin with?
     
  7. somms

    somms Network Guru Member

    Why not just attempt to use a OpenVPN TAP connection first to see if it works before moving to the more difficult TUN configuration?
     
  8. MassiveCollision

    MassiveCollision Reformed Router Member

    I'm actually now first trying without OpenVPN altogether, haven't even configured it after reflashing.

    So just 2 routers, 2 subnets and port forwarding. Not working. Can't reach the second routers remote access or devices behind it.
     
  9. jochen

    jochen LI Guru Member

  10. Vi Lo

    Vi Lo Connected Client Member

Share This Page