1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Route only specific ports through VPN (openvpn)

Discussion in 'Tomato Firmware' started by ethaniel, Apr 1, 2012.

  1. theredmoose

    theredmoose Network Newbie Member

    Thanks John. These addresses are AWS ranges so in theory we may be routing traffic to another site in those ranges however I think this is by far the best answer. Ideally we would be able to get the plex list of ip addresses.
    On a separate note, how do you find the AC68? Are you using an updated firmware such as : https://github.com/RMerl/asuswrt-merlin/blob/master/release/src/router/rc/firewall.c

    Thanks again for your help.
     
  2. theredmoose

    theredmoose Network Newbie Member

    @john9527 do you find your connection stable? I still get the "You have successfully signed your server in to Plex, but we were unable to reach it from outside your network." message. It is intermittent like I mentioned before.

    Would you mind sharing your iptables rules cfg file? I am also wondering if it matters where you are located. I am on the east coast. There is a possibility that services route to different hosts depending on your location.
     
  3. john9527

    john9527 Network Guru Member

    A couple of things that I had to contend with that may help you out....things seem stable to me, but I must admit I don't give this Plex feature a lot of use. Just got it working to prove it could be done.

    (1) The Plex DLNA server is doing something strange with UPNP, and floods the router log with this message if UPNP is enabled. (reported to Plex, but not a lot of interest in getting to the bottom of it)

    miniupnpd[11368]: SendSSDPResponse(): sendto(udp): Operation not permitted

    So I run with UPNP disabled, and set up manual port forwarding on the router where I need it - probably better for security anyway. I chose port 17827, and have that forwarded to 32400 for TCP on the plex server address. In Plex, I specify the 17827 port.

    (2) Although I always thought it may be overkill, I have a version of the rules that were previously posted also active. Here's my complete rule set for Plex (remember I use mark 2 instead of mark 1 for my VPN bypass).

    # Bypass VPN for myPlex
    iptables -t mangle -A PREROUTING -i br0 -p tcp -m multiport --sport 32400,32443 -j MARK --set-mark 2
    iptables -t mangle -A OUTPUT -p udp -m multiport --dport 17827,32400,32443,32410,32412,32413,32414,32469 -j MARK --set-mark 2
    iptables -t mangle -A PREROUTING -i br0 -d 184.72.0.0/16 -j MARK --set-mark 2
    iptables -t mangle -A PREROUTING -i br0 -d 50.18.0.0/16 -j MARK --set-mark 2
    iptables -t mangle -A PREROUTING -i br0 -d 184.169.0.0/16 -j MARK --set-mark 2
    iptables -t mangle -A PREROUTING -i br0 -d 54.241.0.0/16 -j MARK --set-mark 2

    Hope this helps you out.
     
    Last edited: May 30, 2014
  4. theredmoose

    theredmoose Network Newbie Member

    Thanks John that helped a lot. I added my Plex rules below. Note that I added one line to your AWS servers, 54.176.0.0/16. Plus after some testing I noticed that the following rule has to be in place but I am not sure why. Shouldn't the AWS server rules cover this one rule? Unless plex is sending the info out to yet another server not covered by the AWS rules. I am sure these could be restricted further but this is what works for me for now. If anyone else discovers how to restrict these further please let us know.

    #Allows plex server to send packets from wooster from port 32400 to any address
    iptables -t mangle -A PREROUTING -i br0 -s 192.168.1.13 -p tcp -m multiport --sport 32400,32443 -j MARK --set-mark 1​



    # PLEX RULES
    # Enables VPN for one specific machine, the plex server
    iptables -t mangle -A PREROUTING -i br0 -s 192.168.1.13 -j MARK --set-mark 0

    #Allows plex server to register/communicate with plex.tv, all protocols, bypassing the VPN.
    iptables -t mangle -A PREROUTING -i br0 -s 192.168.1.13 -d plex.tv -j MARK --set-mark 1

    #Allows plex server to send packets from wooster from port 32400 to any address
    iptables -t mangle -A PREROUTING -i br0 -s 192.168.1.13 -p tcp -m multiport --sport 32400,32443 -j MARK --set-mark 1

    #Allow all traffic to plex ip addresses to bypass VPN (note potentially bypasses for other AWS sites)
    iptables -t mangle -A PREROUTING -i br0 -d 184.72.0.0/16 -j MARK --set-mark 1
    iptables -t mangle -A PREROUTING -i br0 -d 50.18.0.0/16 -j MARK --set-mark 1
    iptables -t mangle -A PREROUTING -i br0 -d 184.169.0.0/16 -j MARK --set-mark 1
    iptables -t mangle -A PREROUTING -i br0 -d 54.241.0.0/16 -j MARK --set-mark 1
    iptables -t mangle -A PREROUTING -i br0 -d 54.176.0.0/16 -j MARK --set-mark 1

    #Allow ports used by plex to bypass VPN
    iptables -t mangle -A OUTPUT -p udp -m multiport --dport 17827,32400,32443,32410,32412,32413,32414,32469 -j MARK --set-mark 1
     
  5. alwaysbless

    alwaysbless Network Newbie Member

    Great thread with tons of useful info! Big thx to everyone contributing! Ive been using this on ASUSWRT with my RTN66U - Im not sure if anyone else has or if this applies to other peoples units but is anyone else using the script in post #9 noticing that once routes are flushed using command

    "iptables -t mangle -F PREROUTING"

    if the connection is disabled or dropped it still tries to mark packets for table 100? I had issues with NAT loopback breaking after disconnecting the VPN so I compared routes while loopback was working with routes after the VPN was disconnected and they are different. In order to fix its quite simple I can just restore the default route using command

    iptables -t mangle -A PREROUTING -i br0 --destination $(nvram get wan_ipaddr) -j MARK --set-mark 0xd001

    I havent spent much time on this outside of figuring that out but it would be nice to automate this so when tun11 goes down it automatically restores the default routes.
     
    Last edited: Jun 13, 2014
  6. frontalot

    frontalot Serious Server Member

    I'm trying the script in post 9 but it doesn't seem to be working for me. My script:

    for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do
    echo 0 > $i
    done

    ip route flush table 100
    ip route del default table 100
    ip rule del fwmark 1 table 100
    ip route flush cache
    iptables -t mangle -F PREROUTING

    ip route show table main | grep -Ev ^default | grep -Ev tun12 \
    | while read ROUTE ; do
    ip route add table 100 $ROUTE
    done
    ip route add default table 100 via $(nvram get wan_gateway)
    ip rule add fwmark 1 table 100
    ip route flush cache

    iptables -t mangle -A PREROUTING -i br0 -j MARK --set-mark 1
    iptables -t mangle -A PREROUTING -i br0 -p tcp -m multiport --dport 80,443 -j MARK --set-mark 0
    Everything appears to go through successfully other than the error "RTNETLINK answers: No such process." When I run "ip route show table 100" I see my default rules And when I run "iptables --list -t mangle" I see the marks and the rules from my script.

    If I check my IP, though, it's still my ISP and not my VPN. Any ideas on how to troubleshoot the issue? If it makes any difference my router is running in router mode as it's behind a separate gateway.
     
  7. ShinichiYao

    ShinichiYao Reformed Router Member

    Finally I have this script worked, Thanks everyone working on it.
    Still one question: After Connected to VPN I lost every service listen on WAN of the router itself. such as Asterisk, Aria2... just could not access it from WAN
     
    Last edited: Jun 22, 2014
  8. Jaeran

    Jaeran Network Newbie Member

    I just want to thank everyone for providing the information in this thread, it helped me immensely.
     
  9. frontalot

    frontalot Serious Server Member

    I think the problem is related to "routing" vs. "gateway" mode because switching to gateway and everything works. Still no idea how to make it work with routing mode, though...
     
  10. kamaaina

    kamaaina Serious Server Member

    Wow, what a great thread. If you can make custom ports and apps work then I should be able to exclude a single IP address as well, I hope.

    I am trying to exclude the OBIhai SIP adapter from the OpenVPN client traffic. I tried putting the OBI in the DMZ but that does not seem to exclude the traffic from the VPN tunnel. I gave the OBI a steady LAN IP via static DHCP. Putting that address in the DMZ still prevents the adapter from registering with the provider. I found another thread for DD-WRT where one can add some policy based routing and I guess that's what you guys did here but this seems more complex than what I had hoped for/might be capable of.

    Is there a simple way in Tomato v120 to excludes one LAN IP (all incoming/outgoing traffic) from the OpenVPN client traffic and bypass it? Thanks.
     
  11. frontalot

    frontalot Serious Server Member

    Finally got the script to work. Here's how:

    1. Do NOT use "route-nopull" in the OpenVPN config.
    2. DO enable "create NAT on tunnel" in the OpenVPN config.
    3. All traffic will go through the VPN by default, so use mark 1 to exclude ports from the VPN (basically the reverse of how the script was intended to be used).
    4. Routing vs. Gateway mode makes no difference in whether the script works.
     
  12. frontalot

    frontalot Serious Server Member

    Actually, using default mark 0 and selectively applying mark 1 works as well. Guess the root problem was using "route-nopull" and disabling "create NAT on tunnel." Hope this helps someone else!
     
  13. frontalot

    frontalot Serious Server Member

    Now I'm having the same problem as others with not being able to remotely access the router (web interface, SSH, or OpenVPN server). I've tried adding:

    iptables -t mangle -A OUTPUT -p tcp -m multiport --sport 22 -j MARK --set-mark 1

    Without any luck. Anyone know how to configure iptables so that traffic coming from the router itself bypasses the VPN and thus servers like SSH can work correctly?
     
  14. john9527

    john9527 Network Guru Member

    Try the following.....

    # Allow SSH to bypass the VPN (Also need router option set to allow WAN access)
    iptables -t mangle -A PREROUTING -i br0 -p tcp --dport 22 -j MARK --set-mark 1
    iptables -t mangle -A OUTPUT -p tcp --sport 22 -j MARK --set-mark 1

    And if your VPN blocks FTP ports, the following will allow FTP to bypass the VPN

    # Allow FTP to bypass the VPN, turn OFF passive FTP in application
    iptables -t mangle -A PREROUTING -i br0 -p tcp -m multiport --dport 20,21 -j MARK --set-mark 1
    iptables -t mangle -A OUTPUT -p tcp -m multiport --sport 20,21 -j MARK --set-mark 1
     
  15. frontalot

    frontalot Serious Server Member

    For a VPN server running on the same router should I add:

    iptables -t mangle -A PREROUTING -i tun21 -j MARK --set-mark 1
     
  16. k-man

    k-man Network Newbie Member

    @kk5000 were you able to figure out the IP range for WWE Network?

    Thanks!
     
  17. k-man

    k-man Network Newbie Member

    Not to hijack the OPs question, but I get the remote admin page accessible by adding this:

    # OUTPUT for Admin page
    iptables -t mangle -A OUTPUT -p tcp -m multiport --sport 8080 -j MARK --set-mark 1

    However I am unable to access my PPTP VPN server setup on the router. I tried with 1723

    iptables -t mangle -A OUTPUT -p tcp -m multiport --sport 1723 -j MARK --set-mark 1

    but that didn't work.
     
  18. drawab

    drawab Network Newbie Member

    Using IP table config to bypass VPN

    I use StrongVPN for browsing the internet - but while watching Popcorn streaming (popcorntime.io) ive inadvertently received DMCA warnings from strongvpn for some torrent seeding - I have one option is to change the VPN provider - alternatively I've thought of setting up an IP table routing one particular port (randomly configured on popcorn advanced settings to 43763 for eg) to pass through the VPN and use my ISP (non-US based) DSL Modem router IP - 192.168.1.1

    I would like to use that example to modify the code to use on Popcorn - Would I be correct to use this specific code in the terminal of dd-wrt

    Code:
    ip route add default table 100 via 192.168.1.1
    ip rule add fwmark 1 table 100
    ip route flush cache
    iptables -t mangle -I PREROUTING -p tcp --dport 43763 -j MARK --set-mark 1
    
    is this correct or am I wrong somewhere -
    Thanks in advance
     
  19. TronixA

    TronixA Reformed Router Member

    I had some issues with Bittorrent not going through the VPN as well. I believe this has something to do with the fact that Bittorrent actually uses multiple ports to send data to the remote hosts. The port specified in Bittorrent only pertains to the port that it uses to pull in data to. This is based on what I saw by analyzing the ports on the local system.

    One solution I found is to use a secondary NIC on your computer and give it a different IP address than the one used on your primary. You can then download/install a program called ForceIPBind which allows you to bind a program on your computer to a specified interface. You also need to make sure that the second NIC is not the preferred device in Windows. For the WANUP script you want to create a rule that sends all traffic from that internal IP address on the second NIC to the VPN. This will force Bittorrent only to use the VPN.

    Note: I had a problem with the entire WANUP script even working. I found that the script doesn't execute properly as I check the tables via SSH and they aren't setup properly. I ended up breaking the WANUP script into 3 parts using the vi editor under the SSH root account and running them in manually. Only after doing were the tables properly set up. I don't know what the cause is because I can't see what errors the WANUP script is returning. Another annoying thing is that the internet died on my entire network after being in use for awhile. The only thing I found in the log was that the VPN was reporting back that it couldn't resolve the IP address for the VPN. It may have something to do with the fact I'm using Advanced Tomato that is built on top of the Shibby build of Tomato.

    To fix this once and for all I may end up building a dedicated router PC and install PFsense on it. Tomato seems to work great if you use just the webgui tools provided but it seems to balk when you start messing with stuff under the hood like the routing tables. I have no idea why the internet would cut out.Instead of rebooting the device, I killed the VPN connection on the router and ran the clear tables portion of the script and the internet connection was restored. I have no idea what's going on...any ideas of how to find out what's going on with the traffic?
     
  20. MassiveCollision

    MassiveCollision Reformed Router Member

    I still can't get this to work as desired in the following network.

    ................PC1
    ..................^
    Internet > R1 > R2 (Tomato) > PC2


    I have a service listening on port 5001 on PC2 that I want to access from the internet, this now just times out after a while with the way I have it setup now. With VPN not running, everything is accessible, indicating that ports are forwarded correctly and it should work along with the script in this thread.

    Relevant info:

    - Port 5001 is forwarded on R1 to R2, and also on R2 to IP address of PC2

    - R1's LAN is connected to R2's WAN

    - With VPN running and the first iptable rule to let all traffic bypass the VPN (iptables -t mangle -A PREROUTING -i br0 -j MARK --set-mark 1) nothing really bypasses the VPN. Everything I do on PC2 just times out and internet basically stops working, until I mark it 0 again.

    - Making port 5001 bypass the VPN (iptables -t mangle -A PREROUTING -i br0 -p tcp --dport 5001 -j MARK --set-mark 1) I can access the port on PC2 from PC1 by going to <ip of R2>:5001, indicating that these rules DO work but only from R1's network, not the internet

    - I use these settings for my VPN provider:
    https://cryptostorm.org/viewtopic.php?f=37&t=6097

    - Tomato is in Router mode

    I have been researching this on and off in my spare time and can't for the life of me get it going.
     
  21. Ahmad Harb

    Ahmad Harb Network Newbie Member

    Hello,

    I am new to using OpenVPN on a flashed DDWRT router and I have an issue that I am using CyberGhost VPN but outgoing emails for a few mail servers I am using are not working because they block SMTP to stop spammers and they said i need to add the exceptions on the router.

    Anyone can help me on how to do this? I think I need to let traffic on port 25 bypass the VPN as far as I know.

    If anyone can help with this please give me a full detailed step by step since I am a noob when it comes to this honestly.

    Regards,
     
  22. koitsu

    koitsu Network Guru Member

    Are you sure your ISP (not talking about the VPN provider!) allows outbound packets to destination port 25? Many ISPs don't allow this (they filter it on their routers, so your packets get blocked at their routers / never make it out onto the Internet).

    Furthermore, is there some reason the mail servers you use aren't set up properly to answer on TCP port 587, which is what's commonly used for SMTP client submissions? Or better yet, whatever software you're using made to use TLS/SSL, thus using TCP port 465?
     
  23. Ahmad Harb

    Ahmad Harb Network Newbie Member

    I tried through the ISP connection directly and it sends instantly.

    I need to confirm that with the person who provided the mail server service and get back to you but can I know how to do that in general and till i see which ports need to be bypassed ?
     
  24. koitsu

    koitsu Network Guru Member

    You can use telnet or nc (a.k.a. netcat) to do so.
     
  25. Ahmad Harb

    Ahmad Harb Network Newbie Member

    I meant the script to allow the traffic on smtp.

    How it should be written and where should I add it ?

    Regards,
     
  26. MassiveCollision

    MassiveCollision Reformed Router Member

    After trying a lot of things, I found one rule that helped me a bit:

    Code:
    iptables -t nat -I PREROUTING -p tcp -d $(nvram get wan_ipaddr) --dport 5001 -j DNAT --to-destination 172.16.1.2
    With this rule in place I can access port 5001 on PC2 from the outside and the script in this thread seems to finally work, but only from the network of R1, still doesn't work from the internet. Trying to connect from the internet doesn't even show as dropped in the tomato logs, just sits there and times out.

    Can anyone tell me why that rule does what it does for me and whether this is a possible indication for anything wrong with my setup, firewall or whatever?

    Output of iptables -t nat -L -n -v --line-numbers:

    Code:
    Chain PREROUTING (policy ACCEPT 180 packets, 22376 bytes) 
    num pkts bytes target prot opt in out source destination 
    1 0 0 DNAT tcp -- * * 0.0.0.0/0 192.168.178.2 tcp dpt:5001 to:172.16.1.2 
    2 10 1841 WANPREROUTING all -- * * 0.0.0.0/0 192.168.178.2 
    3 0 0 DROP all -- vlan2 * 0.0.0.0/0 172.16.1.0/24 
     
    Chain POSTROUTING (policy ACCEPT 35 packets, 2379 bytes) 
    num pkts bytes target prot opt in out source destination 
    1 18 1152 MASQUERADE all -- * tun11 172.16.1.0/24 0.0.0.0/0 
    2 20 1280 MASQUERADE all -- * vlan2 0.0.0.0/0 0.0.0.0/0 
    3 2 388 SNAT all -- * br0 172.16.1.0/24 172.16.1.0/24 to:172.16.1.1 
     
    Chain OUTPUT (policy ACCEPT 54 packets, 3825 bytes) 
    num pkts bytes target prot opt in out source destination 
     
    Chain WANPREROUTING (1 references) 
    num pkts bytes target prot opt in out source destination 
    1 0 0 DNAT icmp -- * * 0.0.0.0/0 0.0.0.0/0 to:172.16.1.1 
    2 0 0 DNAT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:5001 to:172.16.1.2 
    
    Output of iptables -L -n -v --line-numbers:

    Code:
    Chain INPUT (policy DROP 0 packets, 0 bytes) 
    num pkts bytes target prot opt in out source destination 
    1 0 0 ACCEPT tcp -- br0 * 0.0.0.0/0 172.16.1.254 tcp dpt:80 
    2 0 0 DROP all -- br0 * 0.0.0.0/0 172.16.1.254 
    3 22 3545 ACCEPT all -- tun11 * 0.0.0.0/0 0.0.0.0/0 
    4 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID 
    5 598 198K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED 
    6 0 0 shlimit tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 state NEW 
    7 16 1174 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0 
    8 215 14597 ACCEPT all -- br0 * 0.0.0.0/0 0.0.0.0/0 
    9 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 
    10 11 1837 logdrop all -- * * 0.0.0.0/0 0.0.0.0/0 
     
    Chain FORWARD (policy DROP 0 packets, 0 bytes) 
    num pkts bytes target prot opt in out source destination 
    1 326 83232 ACCEPT all -- tun11 * 0.0.0.0/0 0.0.0.0/0 
    2 419 71207 all -- * * 0.0.0.0/0 0.0.0.0/0 account: network/netmask: 172.16.1.0/255.255.255.0 name: lan 
    3 0 0 ACCEPT all -- br0 br0 0.0.0.0/0 0.0.0.0/0 
    4 56 2864 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID 
    5 20 1280 TCPMSS tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:0x06/0x02 TCPMSS clamp to PMTU 
    6 342 66987 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED 
    7 0 0 wanin all -- vlan2 * 0.0.0.0/0 0.0.0.0/0 
    8 0 0 wanout all -- * vlan2 0.0.0.0/0 0.0.0.0/0 
    9 21 1356 ACCEPT all -- br0 * 0.0.0.0/0 0.0.0.0/0 
     
    Chain OUTPUT (policy ACCEPT 437 packets, 186K bytes) 
    num pkts bytes target prot opt in out source destination 
     
    Chain logdrop (2 references) 
    num pkts bytes target prot opt in out source destination 
    1 11 1837 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 state NEW LOG flags 39 level 4 prefix `DROP ' 
    2 11 1837 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 
     
    Chain logreject (0 references) 
    num pkts bytes target prot opt in out source destination 
    1 0 0 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 LOG flags 39 level 4 prefix `REJECT ' 
    2 0 0 REJECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 reject-with tcp-reset 
     
    Chain shlimit (1 references) 
    num pkts bytes target prot opt in out source destination 
    1 0 0 all -- * * 0.0.0.0/0 0.0.0.0/0 recent: SET name: shlimit side: source 
    2 0 0 logdrop all -- * * 0.0.0.0/0 0.0.0.0/0 recent: UPDATE seconds: 60 hit_count: 4 name: shlimit side: source 
     
    Chain wanin (1 references) 
    num pkts bytes target prot opt in out source destination 
    1 0 0 ACCEPT tcp -- * * 0.0.0.0/0 172.16.1.2 tcp dpt:5001 
     
    Chain wanout (1 references) 
    num pkts bytes target prot opt in out source destination 
    
     
  27. Hardrock

    Hardrock Serious Server Member

    Hey All!

    With my VPN running I want to be able to log into my mobile phone device and access my FTP server remotely. I've configured my mobile with a dynamic dns IP. I've added the lines (in bold) below hoping it will work... well it does until the IP address changes on the phone...

    # Allow FTP to bypass the VPN, turn OFF passive FTP in application
    iptables -t mangle -A PREROUTING -i br0 -p tcp --src xxxx.ddns.net -m multiport --dport 20,21 -j MARK --set-mark 1
    iptables -t mangle -A OUTPUT -p tcp --src xxxx.ddns.net -m multiport --sport 20,21 -j MARK --set-mark 1


    I had a go at writing a script that checks every 5 mins hoping that it will update the two iptable rules should the xxxx.ddns.net IP address change.. But it doesn't work... Any ideas /help appreciate :)

    cat <<END > /tmp/ipcheck
    iptables -t mangle -A PREROUTING -i br0 -p tcp --src xxxx.ddns.net -m multiport --dport 20,21,22 -j MARK --set-mark 1
    iptables -t mangle -A OUTPUT -p tcp --src xxxx.ddns.net -m multiport --sport 20,21,22 -j MARK --set-mark 1
    END
    chmod 755 /tmp/ipcheck
    cru a ipcheck "*/5 * * * * /tmp/ipcheck >/dev/null 2>&1"
     
    Last edited: Sep 4, 2014
  28. john9527

    john9527 Network Guru Member

    All your are doing in the script is adding additional rules, not changing the existing one....not sure, but try this (delete the existing rule first)...You'll need to start fresh to clear out the duplicate rules, so reboot also.

    iptables -D -t mangle -A PREROUTING -i br0 -p tcp --src xxxx.ddns.net -m multiport --dport 20,21,22 -j MARK --set-mark 1
    iptables -D -t mangle -A OUTPUT -p tcp --src xxxx.ddns.net -m multiport --sport 20,21,22 -j MARK --set-mark 1
    iptables -t mangle -A PREROUTING -i br0 -p tcp --src xxxx.ddns.net -m multiport --dport 20,21,22 -j MARK --set-mark 1
    iptables -t mangle -A OUTPUT -p tcp --src xxxx.ddns.net -m multiport --sport 20,21,22 -j MARK --set-mark 1
     
  29. Hardrock

    Hardrock Serious Server Member

    Hey John9527! Thanks for your reply. Will give it a shot later when I get home. In principle you think adding the

    --src xxxx.ddns.net

    line will work? I only want my mobile device using the ddns service to connect up with the open ports.
    I understanding your point ref duplicating rules, we need to delete the existing one first then apply the new one.
    I guess to be more efficient, I ought to do a test first to see if the IP changes before taking any action in deleting rules?
     
  30. Hardrock

    Hardrock Serious Server Member

    Update: It doesn't work with the -D added
    iptables -D -t mangle -A OUTPUT -p tcp --src xxxx.ddns.net -m multiport --sport 20,21,22 -j MARK --set-mark 1

    It complains "invalid rule number mangle"
     
  31. john9527

    john9527 Network Guru Member

    I'm sorry....shouldn't try and help when I'm half asleep...... The correct commands are

    iptables -t mangle -D PREROUTING -i br0 -p tcp --src xxxx.ddns.net -m multiport --dport 20,21,22 -j MARK --set-mark 1
    iptables -t mangle -D OUTPUT -p tcp --src xxxx.ddns.net -m multiport --sport 20,21,22 -j MARK --set-mark 1
     
    Last edited: Sep 6, 2014
  32. Hardrock

    Hardrock Serious Server Member

    Tried it John, it doesnt complain now so thanks for that....unfortunately though, adding the -src addition does not work for me....cant access my network remotely unless i take out the -src ddns address from the -A rules? I just want to open up these ports to the ddns address specified..seems so hard to do LOL
     
  33. john9527

    john9527 Network Guru Member

    It's generally trial and error for me too to get these things to work....anyway...

    for the the OUTPUT rule, xxxx.ddns.net is no longer the source....it's the destination. So...

    iptables -t mangle -D PREROUTING -i br0 -p tcp --src xxxx.ddns.net -m multiport --dport 20,21,22 -j MARK --set-mark 1
    iptables -t mangle -D OUTPUT -p tcp --dst xxxx.ddns.net -m multiport --sport 20,21,22 -j MARK --set-mark 1
     
    Last edited: Sep 6, 2014
  34. Hardrock

    Hardrock Serious Server Member

    John, great..with the --dst added it seems to work buddy :) Many thanks.
    Any hints or suggestions to a simple script that only runs the rules if it detects a change in ddns address?
    It would take me ages to figure this out being honest
     
  35. john9527

    john9527 Network Guru Member

    Removed post
     
    Last edited: Sep 6, 2014
  36. Ricklach

    Ricklach Network Newbie Member

    I have read through this entire thread and even tried a few of the solutions without any success, promarily because I don't have a grasp on this topic. Conceptually, I understand what is happening but that has only introduced some doubt and confusion as to what it is that I am actually trying to do. I have an Asus RT-N66U router with Shibby Tomato 1.28 (AIO) installed. To this I have added Vyprvpn, an app that modifies Tomato to give one-click vpn from any of its world-wide servers.

    In the process I lost access to my web server (on Ubuntu 14.04) because everything in my lan is now being directed through the vpn. If I turn off the vpn I can see my website from the outside but not from any of my LAN devices. So what I was simply trying to do is port forward 80 and 443 in the tomato setup. But as everyone has noted this does not solve the problem. So what I was looking for was a script that would allow all of my LAN devices to use VPN while at the same time allowing those same LAN devices to access my web site along with the rest of the world. In brief, I want everyone to be able to access my website (including me) when I have the vpn enabled. My web server is on static ip 192.168.1.3 and my other devices are on a range of static and dynamic ip addresses.

    I plead ignorance on this topic and any help would be deeply appreciated. In time I am sure I can get this to work.
     
  37. n7lnx

    n7lnx Network Newbie Member

    Hello , am new to the forum .
    I'm using a vpn with Ipvanish and my router firmware is by shibby 1.28 MIPSR2-124 K26 USB Mega-VPN .
    I tried many options but does not work for me . My network goes through the vpn , I want to access my nas from the outside (local ip address 10.0.1.120 port number 58574) , which script should I enter in the firewall settings ?
    thanks in advance
     
  38. eibgrad

    eibgrad Addicted to LI Member

    If by default all the IPs on your network are sent over the VPN, then you either have to access the NAS over the VPN, or else exempt the NAS from the VPN so you can access it via the WAN. The tricky part is that if you’re using a VPN provider, and they’re providing you with scripting, you’ll probably need to modify those scripts. But without knowing EXACTLY how your OpenVPN client is configured, it’s hard to provide specific directions for making those modifications.
     
  39. n7lnx

    n7lnx Network Newbie Member

    thanks eibgrad , the network including NAS goes through the vpn . Here is the configuration that I followed . what is the surest way to access my nas ?

    Step Two:
    In the side bar, click on "VPN Tunneling," and then "OpenVPN Client."

    Under the Basic tab, make the following changes:
    1) Put a check mark next to "Start with WAN"
    2) Change "Interface Type" to TUN
    3) Change "Protocol" to your preference (TCP or UDP)
    4) Type in your desired server ( the server list) and your desired port (choose from 443 or 1194)
    5) Change "Firewall" to Automatic
    6) Change "Authorization Mode" to TLS
    7) Put a check mark next to "Username/Password Authentication"
    8) Enter your IPVanish username and password in the appropriate fields
    9) Put a check mark next to "Username Authen. Only"
    10) Make "Extra HMAC authorization (tls-auth)" Disabled
    11) Put a check mark next to "Create NAT on tunnel"
    Step Three: Under the advanced tab, make the following changes:
    1) Put a check mark next to "Redirect Internet traffic"
    2) Change "Accept DNS configuration" to Strict
    3) Change "Encryption cipher" to AES-256-CBC
    4) Change "Compression" to Adaptive
    5) Leave "TLS Renegotiation Time" at -1
    6) Leave "Connection Retry" at 30
    7) Put a check mark next to "Verify server certificate (tls-remote). Fill in the IPVanish hostname you've selected in the "Common Name:" field (example: phx-a01.ipvanish.com). You can view our list of available hosts here.
    8) Enter the following into the "Custom Configuration":

    resolv-retry infinite
    nobind
    persist-key
    persist-tun
    persist-remote-ip
    comp-lzo
    verb 3
    auth SHA256
    keysize 256
    tls-cipher DHE-RSA-AES256-SHA
     
  40. eibgrad

    eibgrad Addicted to LI Member

    You need to add your own scripting. In the Custom Configuration field, enter the following:

    Code:
    script-security 2
    route-up /jffs/route-up.sh
    route-pre-down /jffs/route-pre-down.sh 
    These directives tell OpenVPN to call your scripts when the routes are up, and just prior to the routes being brought down. When the VPN comes up, the route-up script will create an alternate routing table whose default gateway is the WAN, and add ip rules that specify which source IPs should use that routing table. When the VPN comes down, the route-pre-down script undoes these changes.

    route-up.sh:
    Code:
    #!/bin/sh
    TID=200
    VPN_IF="$dev"  # provided by OpenVPN at runtime
    VPN_DFLT_GTWY='^0.0.0.0/1|^128.0.0.0/1'
    
    # copy main routing table to alternate routing table (ignore VPN routes)
    ip route flush table $TID > /dev/null 2>&1
    ip route show table main | grep -v "$VPN_IF" | grep -Ev $VPN_DFLT_GTWY \
      | while read route; do
            ip route add $route table $TID
        done
    ip route flush cache
    
    # specify source IP(s)/network(s) to be routed over the WAN
    ip rule add from 10.10.1.113 table $TID
    ip rule add from 10.10.2.0/24 table $TID
    route-pre-down.sh:
    Code:
    #!/bin/sh
    TID=200
    ip rule del from 10.10.1.113 table $TID
    ip rule del from 10.10.2.0/24 table $TID
    ip route flush table $TID
    ip route flush cache
    Obviously the source IP(s)/network(s) I specified above are just examples. Adjust as necessary. And make sure to mark them as executable!

    Code:
    chmod +x /jffs/route-up.sh /jffs/route-pre-down.sh
    I like to store these scripts under jffs, but you can place them anywhere that provides persistence, including creating them in the init script (under /tmp). Just make sure to adjust the Custom Configuration field appropriately.

    Now realize it’s possible to NOT use the “Redirect Internet traffic” option and do things in reverse. IOW, have nothing sent over the VPN by default, but instead build an alternate routing table that has the VPN as its default gateway, then add ip rules that direct IPs to that routing table. It just depends on what makes most sense for your needs. The way I’m suggesting it is only because you’ve already decided to use the “Redirect Internet gateway” option. But that would be kind of silly if you really only needed a handful of IPs sent over the VPN since you’d end up having more exceptions than inclusions.
     
    xulian and n7lnx like this.
  41. n7lnx

    n7lnx Network Newbie Member

    first step enable JFFS on my router
    2) I have to create two scripts via ssh mode , in my case
    route-up.sh:
    route-pre-down.sh:
    3) I have to add the script in Custom Configuration
    script.png
    4) I make the script executable
    okay, I have not received error messages
    5) my ddns is updated with the correct ip, and not that of the vpn . But I can not connect , what should I change in the settings ? TCP port is open
     
  42. eibgrad

    eibgrad Addicted to LI Member

    When the VPN comes up, check the message log (cat /var/log/messages) and make sure you didn't receive any errors, esp. wrt the execution of these scripts (if there's a problem, it will often says something to that effect, and perhaps an exit code). Also, telnet/ssh into the router and issue the following commands:

    Code:
    ip route show
    ip route show table 200
    ip rule list
    The first one dumps the default/main routing table. The second one dumps our new routing table. The third one dumps the rules database. Obviously table 200 should contain routes and the rules should show your source IP being redirected to that table. That will at least tell us if we got that far successfully.
     
  43. n7lnx

    n7lnx Network Newbie Member

    many thanks eibgrad
    everything works , I made a mistake to copy the script , i am sorry . Many user from the IpVanish forum will be happy with this :)
     
  44. eangulus

    eangulus Network Guru Member

    Hi,

    Been trying to do this for over a week now to no avail. Have tried every solution in this thread and all I ever get is ALL traffic to VPN or NO Traffic to VPN.

    What I am trying to do is the following:

    Have a PIA account for VPN.
    Running Tomato Shibby 1.24 AIO on an Asus RT-AC66U.

    I would like:
    Default ALL Traffic to be BYPASS VPN.
    Server 192.168.1.10 to default THRU VPN
    Server 192.168.1.10 port 80,443 to BYPASS VPN
    Router 192.168.1.1 port 2000 to BYPASS VPN


    Reason for this is that is that even thou I can the VPN to work, once turned on I no longer have any remote access to my network. My Server runs Apache for dev, test and demo purposes. Also, I loose remote access to my router.

    Not sure if it helps but I tried entering some of these scripts directly into the System page, and it seems to report some errors. I am not knowledgeable enough to know if they are errors or just false positives.

    All scripts I tried have been a direct copy and past and then I scan thru and edit all IP address's to suit my network. Other than that its all left alone.
     
  45. eibgrad

    eibgrad Addicted to LI Member

    Different VPN providers have different requirements in terms of how you configure your router. Some are GUI only, while others provide you with scripting (perhaps even a hybrid of both). And scripting further complicates matters since you're now forced to customize someone else's custom solution. So without having a good knowledge of how you're configured, it's hard to give good recommendations.
     
  46. eangulus

    eangulus Network Guru Member

    Well, as I already stated, the VPN setup is working. So no issues with different setups for different VPN's. I did mention I was on PIA so that means the existing VPN is setup via OpenVPN Client as per PIA instructions.

    The only exception to that is I have entered: route-nopull to the custom config and turned off Auto NAT tunneling.

    So far, the only script that has worked at all is this here: https://support.hidemyass.com/hc/en...ing-for-Tomato-firmware-Per-source-IP-address

    But my issue still stands. With the above instructions, I managed to have my Server talking to VPN and the rest of the network bypasses. What I really need, is only a specific port on the server going over VPN.

    So in saying that I have been trying all the variations on this thread and not 1 of them worked.

    In trying to debug what's going on, I did notice that in most of the scripts there is this bit of code:
    $(nvram get wan_gateway)

    But after the router is up and running, I do a NVRAM dump and the value of that is 0.0.0.0. My guess is that it needs to be the gateway to the modem (next hop). The only parts of the NVRAM that I have found to show the modem gateway is $(nvram get wan_gateway_get), have tried changing but is still not working.

    One last thing is that while reading and trying to understand the code, I notice that it creates a table and marks all of that to 1. But there doesn't seem to be any instruction on what to do with 0. In my thinking is that by default the router is not going over the VPN. remove all scripts and leave VPN on and it is still bypassing it. Adding the table with mark 1 is trying to tell it to bypass, so where is the code that tells it to use the VPN? Am I suppose to be using VPN by default by turning on Redirect all internet traffic or something?
     
  47. eibgrad

    eibgrad Addicted to LI Member

    The HMA approach used in that link is really a hack. Using the WAN Up script introduces timing issues, and which is why there's a sleep 30 at the beginning of it. And they have no way to handle the case where the VPN comes down and/or needs to be restarted.

    They tell you to use the route-nopull directive because their OpenVPN server pushes the “redirect-gateway def1” directive, which alters the client’s routing table to make the VPN the default gateway. But this has negative consequences as well. For example, any push’d DHCP options are ignored, which means you can end up w/ a DNS leak! May or may not matter to some ppl, but something to be aware of.

    OpenVPN supports its own event-driven scripting model that allows user-defined scripts be invoked at various points in the OpenVPN client’s processing. We can use that to make changes to the routing, and at just the right time, and even undo those changes when the VPN is brought down.

    We need two scripts, one that’s executed when OpenVPN comes up, the other just before OpenVPN shuts down. In the route-up.sh script, we change the default gateway in the main/default routing table back to the ISP, and create an alternate routing table w/ a default gateway that points to the VPN. We then use ip rules and mark packets to force certain traffic to use the alternate routing table, and thus the VPN. Finally, in the route-down.sh script, we undo all these changes and return the system to normal.

    route-up.sh:
    Code:
    #!/bin/sh -x
    (
    TID="200"
    MARK="0x88"
    WS="[[:space:]]"
    WAN_GTWY="$route_net_gateway"       # provided by OpenVPN at runtime
    WAN_IF="$(route | grep -Em1 ^${WAN_GTWY}${WS} | awk '{print $NF}')"
    VPN_GTWY="$route_vpn_gateway"       # provided by OpenVPN at runtime
    VPN_IF="$dev"                       # provided by OpenVPN at runtime
    REDIRECT_GTWY="$redirect_gateway"   # provided by OpenVPN at runtime
    
    # copy default/main routing table (exclude all default gateways)
    ip route flush table $TID > /dev/null 2>&1
    ip route show table main | grep -Ev "^default|^0.0.0.0/1${WS}|^128.0.0.0/1${WS}" \
      | while read route; do
            ip route add $route table $TID
        done
    # add VPN as default gateway
    ip route add default via $VPN_GTWY table $TID
    
    # add WAN back as default gateway in main/default routing table
    if [ "$REDIRECT_GTWY" == "1" ]; then
        ip route add 0.0.0.0/2   via $WAN_GTWY
        ip route add 64.0.0.0/2  via $WAN_GTWY
        ip route add 128.0.0.0/2 via $WAN_GTWY
        ip route add 192.0.0.0/2 via $WAN_GTWY
    fi
    
    # disable WAN/VPN reverse path filtering
    echo 0 > /proc/sys/net/ipv4/conf/$WAN_IF/rp_filter
    echo 0 > /proc/sys/net/ipv4/conf/$VPN_IF/rp_filter
    
    # clear the routing cache (or else it won't recognize our changes)
    ip route flush cache
    
    # route over VPN based on source IP(s)/network(s) or network interface
    ip rule add from 10.10.1.113  table $TID
    ip rule add from 10.10.2.0/24 table $TID
    ip rule add iif wl0.1 table $TID
    
    # route over VPN based on other criteria (e.g., protocol, source/destination port)
    iptables -t mangle -A PREROUTING -p tcp -s 10.10.1.42 --sport 22 -j MARK --set-mark $MARK
    iptables -t mangle -A PREROUTING -p tcp -s 10.10.1.155 --sport 80 -j MARK --set-mark $MARK
    iptables -t mangle -A PREROUTING -p tcp --dport 443 -j MARK --set-mark $MARK
    iptables -t mangle -A PREROUTING -s jims-ipod -j MARK --set-mark $MARK
    iptables -t mangle -A PREROUTING -m mac --mac-source 00:11:22:33:44:55 -j MARK --set-mark $MARK
    
    # start processing marked packets through the alternate routing table
    ip rule add fwmark $MARK table $TID
    
    ) 2>&1 | logger -t $(basename $0)[$$]
    route-down.sh:
    Code:
    #!/bin/sh -x
    (
    TID="200"
    MARK="0x88"
    WS="[[:space:]]"
    WAN_GTWY="$route_net_gateway"       # provided by OpenVPN at runtime
    WAN_IF="$(route | grep -Em1 ^${WAN_GTWY}${WS} | awk '{print $NF}')"
    VPN_GTWY="$route_vpn_gateway"       # provided by OpenVPN at runtime
    VPN_IF="$dev"                       # provided by OpenVPN at runtime
    REDIRECT_GTWY="$redirect_gateway"   # provided by OpenVPN at runtime
    
    # remove routes based on source IP(s)/network(s) or network interface
    ip rule del from 10.10.1.113  table $TID
    ip rule del from 10.10.2.0/24 table $TID
    ip rule del iif wl0.1 table $TID
    
    # remove routes based on other criteria (e.g., protocol, source/destination port)
    iptables -t mangle -D PREROUTING -p tcp -s 10.10.1.42 --sport 22 -j MARK --set-mark $MARK
    iptables -t mangle -D PREROUTING -p tcp -s 10.10.1.155 --sport 80 -j MARK --set-mark $MARK
    iptables -t mangle -D PREROUTING -p tcp --dport 443 -j MARK --set-mark $MARK
    iptables -t mangle -D PREROUTING -s jims-ipod -j MARK --set-mark $MARK
    iptables -t mangle -D PREROUTING -m mac --mac-source 00:11:22:33:44:55 -j MARK --set-mark $MARK
    
    # stop processing marked packets through the alternate routing table
    ip rule del fwmark $MARK table $TID
    
    # remove WAN as default gateway in main/default routing table
    if [ "$REDIRECT_GTWY" == "1" ]; then
        ip route del 0.0.0.0/2   via $WAN_GTWY
        ip route del 64.0.0.0/2  via $WAN_GTWY
        ip route del 128.0.0.0/2 via $WAN_GTWY
        ip route del 192.0.0.0/2 via $WAN_GTWY
    fi
    
    # re-enable WAN/VPN reverse path filtering
    echo 1 > /proc/sys/net/ipv4/conf/$WAN_IF/rp_filter
    echo 1 > /proc/sys/net/ipv4/conf/$VPN_IF/rp_filter > /dev/null 2>&1
    
    # clear the alternate routing table and routing cache
    ip route flush table $TID
    ip route flush cache
    
    ) 2>&1 | logger -t $(basename $0)[$$]
    Hopefully the examples are sufficient to get you started. Using “ip rule add …” is slightly more efficient than marking packets in prerouting, but is limited to source IP and/or network interface. Marking packets will work for ANY criteria (including source/IP and/or network interface) supported in prerouting (protocol, input network interface, source/destination port, MAC address, etc.). Just make sure you maintain symmetry between the scripts by deleting the rules in route-down.sh that you created in route-up.sh.

    You’ll need to store these files in JFFS or USB, or build them dynamically in the init script.

    You’ll also need to add the following directives to the Custom Configuration field of the OpenVPN client GUI:

    Code:
    script-security 2
    route-up /jffs/route-up.sh
    route-pre-down /jffs/route-down.sh
    Adjust the path to these files according to your needs.

    As a convenience, the scripts output to syslog. So you can always dump syslog to locate errors.

    Code:
    cat /var/log/messages
    I also find it convenient to use WallWatcher and redirect the syslog to a Windows machine, at least during setup/debugging.
     
    Last edited: Mar 3, 2015
  48. eangulus

    eangulus Network Guru Member

    Wow. Thankyou for the detailed reply. Will take a stab at that later.

    I know you said to basically make sure both are in sync, but what if I don't want to. In my situation I want my server to always use the VPN and never ever use the WAN directly. I honestly thought it would be a simpler answer than above. Scratch that, forgot to update and mention that I did get something working.

    Basically I have in my Firewall Script:
    iptables -I FORWARD -i br0 -o tun11 -j ACCEPT
    iptables -I FORWARD -i tun11 -o br0 -j ACCEPT
    iptables -I INPUT -i tun11 -j DROP
    iptables -t nat -A POSTROUTING -o tun11 -j MASQUERADE

    And then WAN Up Script:
    sleep 30

    for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do
    echo 0 > $i
    done

    ip route flush table 200
    ip route flush cache
    iptables -t mangle -F PREROUTING

    VPN_GW=`ifconfig tun11 | awk '/inet addr/ {split ($2,A,":"); print A[2]}'`
    ip route add table 200 default via $VPN_GW dev tun11

    ip rule add fwmark 1 table 200
    ip route flush cache

    iptables -t mangle -A PREROUTING -i br0 -j MARK --set-mark 0
    iptables -t mangle -A PREROUTING -i br0 -m iprange --src-range 192.168.1.10 -j MARK --set-mark 1
    iptables -t mangle -A PREROUTING -i br0 -p tcp -m multiport --dport 21,22,80,3306,5050,7000,8081,8181,8888,9091,10000 -j MARK --set-mark 0
    iptables -t mangle -A PREROUTING -i br0 -p tcp -m multiport --sport 21,22,80,3306,5050,7000,8081,8181,8888,9091,10000 -j MARK --set-mark 0
    iptables -t mangle -A PREROUTING -i br0 -p udp -m multiport --dport 21,22,80,3306,5050,7000,8081,8181,8888,9091,10000 -j MARK --set-mark 0
    iptables -t mangle -A PREROUTING -i br0 -p udp -m multiport --sport 21,22,80,3306,5050,7000,8081,8181,8888,9091,10000 -j MARK --set-mark 0




    This seems to be working very close to what I want. 192.168.1.10 is using the VPN except on the ports specified, and the rest of the network bypass's the VPN.

    My only issue with the current code now, is that it seems to fall back to WAN when VPN drops. What I would like now is for 192.168.1.10 (except the ports specified above) to never ever use WAN directly. If VPN is down so is that machine. I was thinking of trying a DROP line in the firewall or something, but have yet to get anything working in that regards.
     
  49. eibgrad

    eibgrad Addicted to LI Member

    I don't know what's going on w/ this forum lately. I just posted a response a few hours ago, and now it's GONE! And this has happened several times over the past week or so. Anyone else having similar problems? Looks like I have to post this yet again.

    I think you missed my point about "sync". All I meant was that since we’re now doing things properly, we want to make sure we don’t do something silly, like add firewall rules in route-up, have the VPN decide to restart itself (e.g., maybe the keepalive has failed), have it call our route-down script, and end up w/ duplicate firewall rules because we failed to delete the firewall rules in the route-down script.

    IOW, it’s only going to work correctly if you likewise code your scripts correctly. And that means maintaining symmetry between the route-up and route-down scripts in terms of what changes they make/unmake.

    As I said, using WAN Up is a hack. Such symmetry is not even possible since that script is only called in response to a reset of the WAN. Anything that resets the VPN happens independently of the WAN, and vice versa. And that always leaves open the possibility of things in the VPN becoming out of sync.

    At the end of the day, use whatever works for you.

    As far as blocking access to WAN, this is not a VPN issue, per se. Yes, in effect you’re doing this because you only want certain clients to have access to the VPN, but preventing access to the WAN is really the issue. And like anything else you don’t want to use the WAN, you add appropriate firewall rules in the firewall script.

    Code:
    iptables -I FORWARD -i br0 -s <ip-address> -o $(nvram get wan_iface) -j REJECT --reject-with icmp-host-prohibited
    iptables -I FORWARD -i br0 -p tcp -s <ip-address> -o $(nvram get wan_iface) -j REJECT --reject-with tcp-reset
    This is a case where you don’t want to be in-sync w/ the VPN. That’s why we add it to the firewall script. Scripting is always best when you can bind your scripts directly to the relevant processes.
     
    Last edited: Mar 3, 2015
  50. eangulus

    eangulus Network Guru Member

    Just tried out your REJECT code in the firewall, and it works, but a little too well. Even with the VPN on, it still blocks everything.
     
  51. eibgrad

    eibgrad Addicted to LI Member

    Ugh. I originally posted a reply that had an error, I corrected it, then the post disappeared for some reason, and when I reposted, I reposted the error again! LOL

    I corrected the code above, try it again.
     
  52. eangulus

    eangulus Network Guru Member

    No probs trying now.

    Although I am starting to understand iptables and such (enough to google the relevant terms) after 2 weeks at playing with all this, I still don't have a grasp on the relationship between manual iptables and the Tomato GUI.

    specifically, I need to forward a single port to and from the server ip, but only over the VPN. Does the rule in the Tomato Portforward apply to the VPN as well or do I have to make another iptable entries to accomodate?
     
  53. eangulus

    eangulus Network Guru Member

    OK tried that and it half worked.

    I know now that to do what I what, I need to do it your way using the route-up/route-down scripts.

    The above code you gave to block the server from the gateway worked. Except that once I put the VPN back on, it was still completly blocked.

    Also the ports I have open bypassing the VPN were also blocked, but I kinda expected that as your rules blocked everything.

    With your route-up/down scripts is there a way I can achieve all this? My ultimate goal is this:

    Server = 192.168.1.10
    Default Traffic to Bypass VPN
    All traffic to/from server to be over VPN
    Set a number of ports to/from server to bypass VPN.
    Stop Server from accessing WAN (except for the ports above) when VPN fails
    Resume Server over VPN (except above ports) when VPN returns.
     
  54. eibgrad

    eibgrad Addicted to LI Member

    If you’re asking whether you can port forward from the WAN of the router supporting the OpenVPN client, over to the OpenVPN server's network, the answer is yes. The port forwarding feature of the GUI doesn't care what network you specify as the target of the port forward. However, it's probably the case that only the current local network is being NAT'd over the OpenVPN client. And if that's the case, you need to add the following rule:

    Code:
    iptables -t nat -A POSTROUTING -o tun11 -j MASQUERADE
    This rules says to NAT *anything* that attempts to send something over the VPN tunnel.
     
  55. eibgrad

    eibgrad Addicted to LI Member

    If you only want to block the WAN for specific ports on the server (because those same ports should only be accessible via the VPN), then just add those ports to the iptables rules I provided. You can narrow the scope of those rules any way you see fit.

    Code:
    iptables -I FORWARD -i br0 -p udp -s 192.168.1.10 -m multiport --dports 21,22,80,3306,5050,7000,8081,8181,8888,9091,10000 -o $(nvram get wan_iface) -j REJECT --reject-with icmp-host-prohibited
    iptables -I FORWARD -i br0 -p tcp -s 192.168.1.10 -m multiport --dports 21,22,80,3306,5050,7000,8081,8181,8888,9091,10000 -o $(nvram get wan_iface) -j REJECT --reject-with tcp-reset
     
  56. eangulus

    eangulus Network Guru Member

    No I'm trying to do the other way around. I have about 6 or so ports that I need to bypass the VPN, everything else from the server to go over the vpn.

    Either way I am trying to setup the route-up/down version of your setup. Going OK I think so far, understanding most of the code thanks to the comments, but if possible can you put some more commenting into the route-down.sh script, please? I think I have it but not too sure as I am just comparing the code similarities between the 2 and using the route-up script comments to help me.

    Also just 1 particular line I am unsure about.

    ip rule add fwmark $MARK table $TID

    Is this part of the rules or not? note sure if I need it or its something I can comment out as it is very different than the iptable lines that it is grouped with.
     
  57. eibgrad

    eibgrad Addicted to LI Member

    ip rule add fwmark $MARK table $TID

    The above line is necessary in order for the marked packets in the firewall rules above it to be recognized by the routing system and have those packets processed through the alternative routing table. The ip rule is related in the sense that without it, the rules directly above it would have no effect. Likewise, if you didn’t need any of those rules, you wouldn’t need to have that ip rule either.

    So while it’s not a firewall rule like the rules above it, it goes hand in hand with them.

    I’ve updated the script w/ a few more comments. Hope it helps.
     
  58. eibgrad

    eibgrad Addicted to LI Member

    So you want to port forward in from the VPN server and over to a local server on the OpenVPN client side, and have the reply sent back out over the VPN, correct?

    First, that assumes the VPN service provider allows port forwarding (some do, some don’t). Second, the GUI based port forwarding is only looking at the WAN for these purposes, not any other network interfaces. So you’d have to create your own port forwarding rules using iptables and the firewall script. Third, it’s not clear what the source IP would be for traffic being initiated from the other side of the tunnel. IOW, is this a private network? What’s the network (10.0.0.x, 192.168.2.x, etc.)? Does it conflict w/ your own network? Or is it being NAT’d from the other side using the OpenVPN server’s IP? Without knowing the answers to these questions, I can’t say what other changes might be necessary. Because if that traffic isn’t NAT’d, then you’d end up w/ these mysterious packets appearing on the OpenVPN client side and no idea how to route them back. Finally, you’d have to make sure that those replies were always sent out the VPN using the scripts above.

    So it’s complicated, and there’s a lot of ifs. But if you have all your ducks in a row, yeah, it can be done.
     
  59. eangulus

    eangulus Network Guru Member

    First to the above, no. No VPN at all for the selected ports. Simply want all network traffic + a select number of ports to server to bypass VPN altogether (I don't have port forwarding with my VPN provider). Then all left over traffic to and from the server to go over the VPN.

    So for eg. I want port 21,22,80 to/from the server to bypass VPN. all other server traffic to go via VPN.


    Secondly, been trying all afternoon to implement your method of the up/down scripts. I have it working somewhat:

    ON BOOT (before VPN Starts):
    1. Everything has Internet including server (Need no Server traffic other than my ports)

    VPN UP:
    1. Nothing Works (Need all traffic to Bypass while Server is VPN'd)
    2. Select Ports are over VPN (its ok at this stage, haven't set those up yet)

    VPN DOWN:
    1. All Works (Need Server blocked).

    Just trying to read the script and understand more of it. Although it would be easy for me to just ask someone to do it for me, I am liking this convo, as I am learning so much about advanced networking. I am starting to see the power in knowing all this.

    PS: Not sure if it matters (in regards to the Reboot at least), I am hosting the scripts on a CIFS share which is located on the Server. Maybe VPN is starting before the CIFS connection and not loading the scripts?
     
    Last edited: Mar 3, 2015
  60. eangulus

    eangulus Network Guru Member

    I think the fact that the whole network goes over the VPN might have something to do with $redirect_gateway.

    Is there any way I can view the value it is getting?
     
  61. eangulus

    eangulus Network Guru Member

    Not sure whats going on here:
    daemon.warn openvpn[3165]: WARNING: Failed running command (--route-up): could not execute external program

    Have tried with the scripts in the CIFS and the JFFS.
    And yes, I have the security 2 line in the config.
     
  62. eibgrad

    eibgrad Addicted to LI Member

    I was getting confused because several times you referred to port forwarding. Port forwarding to me means remote access to the LAN, either from the WAN or VPN. But apparently you weren't saying anything different than you'd been saying all along about what you want to do.

    Yes. Having the scripts on external storage can cause timing issues. It's best to use JFFS or else build the scripts dynamically in the init script so that you don't run into this issue (if in fact it is the issue).

    Assuming the scripts are available at the time the VPN attempts to access them, and you have the "script-security 2" directive specified, the most likely reason you’re getting those errors is due to syntax errors. Something that’s easy to do given you're modifying those scripts.

    In my original response I explained that to make it easier to debug these scripts, they are configured to output to the syslog. And I explained how to examine the syslog. And I explained how using WallWatcher could make that easier. Please reread my original response!
     
  63. eangulus

    eangulus Network Guru Member

    I did some more testing.

    First, where do you think I got that log line about the security? I got it from cat /var/log/messages

    I have also tried running the scripts from CIFS & JFFS so far, yet to try from USB, but all are giving me the same error. There is also a line that shows up in the logs same as the route-up error but for route-down. And yes I definitly have script-security 2 in the OpenVPN config. Right above the paths to the scripts.

    Did spot this thou:

    Mar 4 09:06:18 ECS-ROUTER daemon.info dnscrypt-proxy[7147]: Done
    Mar 4 09:06:19 ECS-ROUTER daemon.warn openvpn[7067]: ERROR: Linux route add command failed: external program exited with error status: 1
    Mar 4 09:06:19 ECS-ROUTER daemon.warn openvpn[7067]: WARNING: Failed running command (--route-up): external program exited with error status: 2
    Mar 4 09:06:19 ECS-ROUTER daemon.notice openvpn[7067]: Initialization Sequence Completed

    Mentions something about an ADD command error.
     
  64. eibgrad

    eibgrad Addicted to LI Member

    Hmm, I just remembered, you also need to mark those scripts as executable:

    Code:
    chmod +x /jffs/route-up.sh /jffs/route-down.sh
     
  65. eangulus

    eangulus Network Guru Member

    I was already just trying that, still all the same. seems to be as the logs say, an add error. Have tried commenting out each one by one but doesn't seem to do anything either.
     
  66. eibgrad

    eibgrad Addicted to LI Member

    Well since you're modifying these scripts, perhaps you've made a syntax error. Let's see what you've done.
     
  67. eangulus

    eangulus Network Guru Member

    route-up.sh
    Code:
    #!/bin/sh -x
    (
    
    TID="200"
    MARK="0x88"
    WS="[[:space:]]"
    WAN_GTWY="$route_net_gateway"       # provided by OpenVPN at runtime
    WAN_IF="$(route | grep -Em1 ^${WAN_GTWY}${WS} | awk '{print $NF}')"
    VPN_GTWY="$route_vpn_gateway"       # provided by OpenVPN at runtime
    VPN_IF="$dev"                       # provided by OpenVPN at runtime
    REDIRECT_GTWY="$redirect_gateway"   # provided by OpenVPN at runtime
    
    # copy default/main routing table (exclude all default gateways)
    ip route flush table $TID > /dev/null 2>&1
    ip route show table main | grep -Ev "^default|^0.0.0.0/1${WS}|^128.0.0.0/1${WS}" \
      | while read route; do
            ip route add $route table $TID
        done
    # add VPN as default gateway
    ip route add default via $VPN_GTWY table $TID
    
    # add WAN back as default gateway in main/default routing table
    if [ "$REDIRECT_GTWY" == "1" ]; then
        ip route add 0.0.0.0/2   via $WAN_GTWY
        ip route add 64.0.0.0/2  via $WAN_GTWY
        ip route add 128.0.0.0/2 via $WAN_GTWY
        ip route add 192.0.0.0/2 via $WAN_GTWY
    fi
    
    # disable WAN/VPN reverse path filtering
    echo 0 > /proc/sys/net/ipv4/conf/$WAN_IF/rp_filter
    echo 0 > /proc/sys/net/ipv4/conf/$VPN_IF/rp_filter
    
    # clear the routing cache (or else it won't recognize our changes)
    ip route flush cache
    
    # route over VPN based on source IP(s)/network(s) or network interface
    ip rule add from 192.168.1.10  table $TID
    # ip rule add from 10.10.1.113  table $TID
    # ip rule add from 10.10.2.0/24 table $TID
    # ip rule add iif wl0.1 table $TID
    
    # route over VPN based on other criteria (e.g., protocol, source/destination port)
    # iptables -t mangle -A PREROUTING -p tcp -s 10.10.1.42 --sport 22 -j MARK --set-mark $MARK
    # iptables -t mangle -A PREROUTING -p tcp -s 10.10.1.155 --sport 80 -j MARK --set-mark $MARK
    # iptables -t mangle -A PREROUTING -p tcp --dport 443 -j MARK --set-mark $MARK
    # iptables -t mangle -A PREROUTING -s jims-ipod -j MARK --set-mark $MARK
    # iptables -t mangle -A PREROUTING -m mac --mac-source 00:11:22:33:44:55 -j MARK --set-mark $MARK
    
    # start processing marked packets through the alternate routing table
    ip rule add fwmark $MARK table $TID
    
    ) 2>&1 | logger -t $(basename $0)[$$]
    route-down.sh
    Code:
    #!/bin/sh -x
    (
    
    TID="200"
    MARK="0x88"
    WS="[[:space:]]"
    WAN_GTWY="$route_net_gateway"       # provided by OpenVPN at runtime
    WAN_IF="$(route | grep -Em1 ^${WAN_GTWY}${WS} | awk '{print $NF}')"
    VPN_GTWY="$route_vpn_gateway"       # provided by OpenVPN at runtime
    VPN_IF="$dev"                       # provided by OpenVPN at runtime
    REDIRECT_GTWY="$redirect_gateway"   # provided by OpenVPN at runtime
    
    # remove routes based on source IP(s)/network(s) or network interface
    ip rule del from 192.168.1.10  table $TID
    # ip rule del from 10.10.1.113  table $TID
    # ip rule del from 10.10.2.0/24 table $TID
    # ip rule del iif wl0.1 table $TID
    
    # remove routes based on other criteria (e.g., protocol, source/destination port)
    # iptables -t mangle -D PREROUTING -p tcp -s 10.10.1.42 --sport 22 -j MARK --set-mark $MARK
    # iptables -t mangle -D PREROUTING -p tcp -s 10.10.1.155 --sport 80 -j MARK --set-mark $MARK
    # iptables -t mangle -D PREROUTING -p tcp --dport 443 -j MARK --set-mark $MARK
    # iptables -t mangle -D PREROUTING -s jims-ipod -j MARK --set-mark $MARK
    # iptables -t mangle -D PREROUTING -m mac --mac-source 00:11:22:33:44:55 -j MARK --set-mark $MARK
    
    # stop processing marked packets through the alternate routing table
    ip rule del fwmark $MARK table $TID
    
    # remove WAN as default gateway in main/default routing table
    if [ "$REDIRECT_GTWY" == "1" ]; then
        ip route del 0.0.0.0/2   via $WAN_GTWY
        ip route del 64.0.0.0/2  via $WAN_GTWY
        ip route del 128.0.0.0/2 via $WAN_GTWY
        ip route del 192.0.0.0/2 via $WAN_GTWY
    fi
    
    # re-enable WAN/VPN reverse path filtering
    echo 1 > /proc/sys/net/ipv4/conf/$WAN_IF/rp_filter
    echo 1 > /proc/sys/net/ipv4/conf/$VPN_IF/rp_filter > /dev/null 2>&1
    
    # clear the alternate routing table and routing cache
    ip route flush table $TID
    ip route flush cache
    
    ) 2>&1 | logger -t $(basename $0)[$$]
     
  68. eibgrad

    eibgrad Addicted to LI Member

    Ok, so I took your scripts, as modified, literally cut and pasted them into my own router. I then ran them through my own OpenVPN client as a sanity check. They worked perfectly.

    Be aware, if you start making errors in these scripts, run the scripts, correct the errors, run the scripts again, it's possible that changes made from prior failed runs have not been cleaned up. For example, supposed you got as far as changing the main routing table w/ those "add route" commands and something later fails, perhaps the route-down script doesn't run for some reason. Those additions to the routing table are still there the next time you run route-up, so you get errors.

    IOW, until things are working correctly, you may need to reboot the router to make sure things are cleanup after a failure.

    Anyway, based on browsing your code change, and my actually use of the code, it should be working. So reboot, try again.
     
  69. eangulus

    eangulus Network Guru Member

    Well, I am at a loss completely.

    Have rebooted many times (mainly to make sure). Even did it again after your last comment. I have the scripts in jffs executable and the script-security 2 in the config. Only thing I have set so far (until I get it working) is turned off starting VPN on boot.

    SO in saying that, I just rebooted, waited for it all to load up. I turned on VPN Client, waited for it to settle then turn it off. Here a logs for that.

    Code:
    Mar  4 10:35:11 ECS-ROUTER daemon.notice openvpn[1769]: OpenVPN 2.3.6 mipsel-unknown-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [MH] [IPv6] built on Dec 24 2014
    Mar  4 10:35:11 ECS-ROUTER daemon.notice openvpn[1769]: library versions: OpenSSL 1.0.1j 15 Oct 2014, LZO 2.08
    Mar  4 10:35:11 ECS-ROUTER daemon.warn openvpn[1769]: WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
    Mar  4 10:35:11 ECS-ROUTER daemon.warn openvpn[1769]: NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
    Mar  4 10:35:11 ECS-ROUTER daemon.notice openvpn[1779]: UDPv4 link local: [undef]
    Mar  4 10:35:11 ECS-ROUTER daemon.notice openvpn[1779]: UDPv4 link remote: [AF_INET]108.61.96.6:1194
    Mar  4 10:35:12 ECS-ROUTER daemon.notice openvpn[1779]: [Private Internet Access] Peer Connection Initiated with [AF_INET]108.61.96.6:1194
    Mar  4 10:35:15 ECS-ROUTER daemon.err openvpn[1779]: event_wait : Interrupted system call (code=4)
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: OpenVPN STATISTICS
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: Updated,Wed Mar  4 10:35:15 2015
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: TUN/TAP read bytes,0
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: TUN/TAP write bytes,0
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: TCP/UDP read bytes,4170
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: TCP/UDP write bytes,1719
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: Auth read bytes,0
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: pre-compress bytes,0
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: post-compress bytes,0
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: TUN/TAP device tun11 opened
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: /sbin/ifconfig tun11 10.198.1.10 pointopoint 10.198.1.9 mtu 1500
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: updown.sh tun11 1500 1542 10.198.1.10 10.198.1.9 init
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1017]: exiting on receipt of SIGTERM
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: started, version 2.71 cachesize 2048
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: compile time options: IPv6 GNU-getopt no-RTC no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset Tomato-helper auth DNSSEC
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: DNSSEC validation enabled
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: DNSSEC signature timestamps not checked until first cache reload
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: asynchronous logging enabled, queue limit is 5 messages
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq-dhcp[1843]: DHCP, IP range 192.168.1.140 -- 192.168.1.169, lease time 1d
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 208.67.220.220#53
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 208.67.222.222#53
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 127.0.0.1#40
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: read /etc/hosts - 14 addresses
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: read /etc/dnsmasq/hosts/hosts - 48 addresses
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq-dhcp[1843]: read /etc/dnsmasq/dhcp/dhcp-hosts
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 208.67.220.220#53
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 208.67.222.222#53
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 127.0.0.1#40
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 203.12.160.35#53
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 203.12.160.36#53
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 209.222.18.222#53
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 209.222.18.218#53
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 208.67.222.222#53
    Mar  4 10:35:15 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 208.67.220.220#53
    Mar  4 10:35:16 ECS-ROUTER daemon.notice dnscrypt-proxy[1874]: Starting dnscrypt-proxy 1.4.1
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnscrypt-proxy[1874]: Initializing libsodium for optimal performance
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnscrypt-proxy[1874]: Generating a new key pair
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnscrypt-proxy[1874]: Done
    Mar  4 10:35:16 ECS-ROUTER daemon.notice dnscrypt-proxy[1876]: Starting dnscrypt-proxy 1.4.1
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnscrypt-proxy[1876]: Initializing libsodium for optimal performance
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnscrypt-proxy[1876]: Generating a new key pair
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: now checking DNSSEC signature timestamps
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: read /etc/hosts - 14 addresses
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: read /etc/dnsmasq/hosts/hosts - 48 addresses
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq-dhcp[1843]: read /etc/dnsmasq/dhcp/dhcp-hosts
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 208.67.220.220#53
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 208.67.222.222#53
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 127.0.0.1#40
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 203.12.160.35#53
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 203.12.160.36#53
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 209.222.18.222#53
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 209.222.18.218#53
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 208.67.222.222#53
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnsmasq[1843]: using nameserver 208.67.220.220#53
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnscrypt-proxy[1874]: Server certificate #808464433 received
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnscrypt-proxy[1874]: This certificate looks valid
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnscrypt-proxy[1874]: Chosen certificate #808464433 is valid from [2014-07-03] to [2015-07-03]
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnscrypt-proxy[1874]: Server key fingerprint is 3730:AAB4:B7FD:40F6:3C42:B12C:60DF:B615:8392:B6AF:9AA4:4CFD:C282:0BAC:E68E:2624
    Mar  4 10:35:16 ECS-ROUTER daemon.notice dnscrypt-proxy[1874]: Proxying from 127.0.0.1:40 to 113.20.8.17:443
    Mar  4 10:35:16 ECS-ROUTER daemon.info dnscrypt-proxy[1876]: Done
    Mar  4 10:35:16 ECS-ROUTER daemon.warn openvpn[1779]: WARNING: Failed running command (--route-up): external program exited with error status: 2
    Mar  4 10:35:16 ECS-ROUTER daemon.notice openvpn[1779]: Initialization Sequence Completed
    Mar  4 10:35:20 ECS-ROUTER daemon.info dnsmasq-dhcp[1843]: DHCPREQUEST(br0) 192.168.1.46 dc:85:de:7b:8f:a5
    Mar  4 10:35:20 ECS-ROUTER daemon.info dnsmasq-dhcp[1843]: DHCPACK(br0) 192.168.1.46 dc:85:de:7b:8f:a5 NB-LILY
    Mar  4 10:35:27 ECS-ROUTER cron.err crond[686]: time disparity of 23757094 minutes detected
    Mar  4 10:35:49 ECS-ROUTER daemon.info dnsmasq-dhcp[1843]: DHCPREQUEST(br0) 192.168.1.147 f0:24:75:40:03:8f
    Mar  4 10:35:49 ECS-ROUTER daemon.info dnsmasq-dhcp[1843]: DHCPACK(br0) 192.168.1.147 f0:24:75:40:03:8f Hughs-iPad
    Mar  4 10:36:18 ECS-ROUTER daemon.err openvpn[1779]: event_wait : Interrupted system call (code=4)
    Mar  4 10:36:18 ECS-ROUTER daemon.notice openvpn[1779]: /jffs/route-down.sh tun11 1500 1542 10.198.1.10 10.198.1.9 init
    Mar  4 10:36:18 ECS-ROUTER daemon.err openvpn[1779]: WARNING: Failed running command (--up/--down): external program exited with error status: 2
    Mar  4 10:36:18 ECS-ROUTER daemon.notice openvpn[1779]: Exiting due to fatal error
     
  70. eangulus

    eangulus Network Guru Member

    It looks to me that the security setting is setting OK, its warning me that may allow user-defined scripts.

    What does error status 2 mean?
     
  71. eibgrad

    eibgrad Addicted to LI Member

    So now show me the entire contents of the OpenVPN Client Custom Configuration field, all of it.
     
  72. eangulus

    eangulus Network Guru Member

    Code:
    persist-key
    persist-tun
    tls-client
    comp-lzo
    verb 1
    auth-nocache
    script-security 2
    route-up /jffs/route-up.sh
    route-pre-down /jffs/route-down.sh
     
  73. eibgrad

    eibgrad Addicted to LI Member

    Go to a telnet/ssh session on the router, and type the following:

    Code:
    ls -l /jffs
    And upload the output here.

    P.S. Update your config to verb 5 (verb 1 suppresses too much output that may be helpful).
     
  74. eangulus

    eangulus Network Guru Member

    Code:
    root@ECS-ROUTER:/tmp/home/root# ls -l /jffs
    -rwxr-xr-x    1 root     root          1877 Mar  4 10:31 route-down.sh
    -rwxr-xr-x    1 root     root          2198 Mar  4 10:06 route-up.sh
     
  75. eibgrad

    eibgrad Addicted to LI Member

    I find this line in the syslog odd:

    Code:
    Mar  4 10:35:15 ECS-ROUTER daemon.notice openvpn[1779]: updown.sh tun11 1500 1542 10.198.1.10 10.198.1.9 init
    Looks like it’s trying to run some other script(s).

    Go to a telnet/ssh session, run the following:

    Code:
    ps -w | grep [v]pnclient
    Post the output here.
     
  76. eangulus

    eangulus Network Guru Member

    Ran it, nothing happened no output.
     
  77. eibgrad

    eibgrad Addicted to LI Member

    Has to be there if the VPN is running! You're running Shibby tomato, right?
     
  78. eangulus

    eangulus Network Guru Member

    Incase you ment with the VPN running:

    Code:
    root@ECS-ROUTER:/tmp/home/root# ps -w | grep [v]pnclient
    2528 root      3440 S    /etc/openvpn/vpnclient1 --cd /etc/openvpn/client1 --config config.ovpn
    
     
  79. eangulus

    eangulus Network Guru Member

    And yes Tomato Firmware 1.28.0000 MIPSR2-124 K26AC USB AIO-64K
     
  80. eibgrad

    eibgrad Addicted to LI Member

    With the VPN running ( :) ), go to telnet/ssh and run the following, and post the results.

    Code:
    cat /tmp/etc/openvpn/client1/config.ovpn
    You can delete any sensitive data.
     
  81. eangulus

    eangulus Network Guru Member

    Code:
    root@ECS-ROUTER:/tmp/home/root# cat /tmp/etc/openvpn/client1/config.ovpn
    # Automatically generated configuration
    daemon
    client
    dev tun11
    proto udp
    remote aus.privateinternetaccess.com 1194
    resolv-retry 30
    reneg-sec 0
    nobind
    persist-key
    persist-tun
    comp-lzo adaptive
    verb 3
    script-security 2
    up updown.sh
    down updown.sh
    ca ca.crt
    auth-user-pass up
    status-version 2
    status status
    
    # Custom Configuration
    persist-key
    persist-tun
    tls-client
    comp-lzo
    verb 1
    auth-nocache
    script-security 2
    route-up /jffs/route-up.sh
    route-pre-down /jffs/route-down.sh
     
  82. eibgrad

    eibgrad Addicted to LI Member

    Ok, everything in terms of the config looks ok. The only thing I can think could be the problem is the scripts themselves.

    Exactly how did you get these files into the router? What steps did you take? What apps did you use? Etc.

    What I’m concerned about is that Windows and Linux use different newline chars, and depending on how you move/edit files, you can end up creating Windows versions of a file on the Linux system, and the Linux system can’t read them. And so it matters how you actually get these files into the router.

    Here’s what I want you to do (to make sure this isn’t the problem).

    Go to a telnet/ssh session, change to the /jffs directory, and type the following:

    Code:
    cat > route-up.sh
    The screen will show a cursor to the far left (it’s waiting for input). Now copy the contents of the route-up.sh script from the same post I did previously, and paste it into the telnet/ssh session (if you’re using putty, you can typically just hit right click). You’ll see the screen (and file) filled w/ the script. Now hit Ctrl-C to close the file. Do the same thing w/ the route-down.sh script. Finally, chmod +x each file again.

    The purpose of doing it this way is to make sure there’s no chance of Windows converting the text from Linux to Windows format before it’s entered into the scripts.

    If that doesn’t work, then I’m at a loss too.
     
    Last edited: Mar 4, 2015
  83. eangulus

    eangulus Network Guru Member

    Just tried that and no difference.

    I think I am going to try a router reset and full nvram wipe. Its the only thing I can think of.
     
  84. eangulus

    eangulus Network Guru Member

    Thats it then.... game over, cannot be done.

    Did a full NVRAM format, entered only the basics to get a connection, setup clean the OpenVPN connection, followed instructions from here and even created the files in jffs using a copy past in putty.

    Still EXACTLY the same error.
     
  85. eangulus

    eangulus Network Guru Member

    Look this is a long shot, but in google around and reading on similar errors for openvpn, I did notice some stating that the gateway address's and the like were wrong, or the tunnel name eg, tun11 or tun22.

    And in some cases were fixed when corrected.

    With the code I have there is a number of like variables at the top. Is there anyway to find out what they actually are? I want to confirm that all the paths etc are correct.
     
  86. eangulus

    eangulus Network Guru Member

    WOOHOOO!!!

    I got it reading the script. Not working as I want yet but at least now we can move on.

    Not 100% certain what it was, but I think it was a combo of the Windows/Linux file issue and creating the files in putty.

    Reason is that I first tried with the scripts already there but created with Notepad++, then second I did the copy/paste into SSH.

    But after that I opened those created files and noticed that the last line ") 2>&1 | logger -t $(basename $0)[$$]" was missing in both (even thou I know it copied and pasted, just didn't save). Anyway after adding it and saving again, started VPN and it worked. The stuff in the scripts is now showing in the logs.

    Now issue I have at the moment is that with the VPN up, Network is normal (bypassing VPN) but server has no internet at all.

    And after looking at the routing tables, there is allot of extra stuff there that doesn't make sense, do I really need all of these:

    Code:
    # remove WAN as default gateway in main/default routing table
    if [ "$REDIRECT_GTWY" == "1" ]; then
        ip route del 0.0.0.0/2   via $WAN_GTWY
        ip route del 64.0.0.0/2  via $WAN_GTWY
        ip route del 128.0.0.0/2 via $WAN_GTWY
        ip route del 192.0.0.0/2 via $WAN_GTWY
    fi
     
  87. eangulus

    eangulus Network Guru Member

    Also just tested shuting down the VPN and I am getting a bunch of errors, the script runs thou, they are errors in the script.

    Code:
    Mar  4 05:59:01 unknown daemon.notice openvpn[6580]: /jffs/route-down.sh tun11 1500 1542 10.116.1.6 10.116.1.5 init
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + TID=200
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + MARK=0x88
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + WS=[[:space:]]
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + WAN_GTWY=
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + route
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + grep -Em1 ^ [[:space:]]
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: grep: [[:space:]]: No such file or directory
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + awk {print $NF}
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: route: standard output: Broken pipe
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + WAN_IF=
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + VPN_GTWY=10.116.1.5
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + VPN_IF=tun11
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + REDIRECT_GTWY=1
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + ip rule del from 192.168.1.10 table 200
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + ip rule del fwmark 0x88 table 200
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + [ 1 == 1 ]
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + ip route del 0.0.0.0/2 via
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: Command line is not complete. Try option "help"
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + ip route del 64.0.0.0/2 via
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: Command line is not complete. Try option "help"
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + ip route del 128.0.0.0/2 via
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: Command line is not complete. Try option "help"
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + ip route del 192.0.0.0/2 via
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: Command line is not complete. Try option "help"
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: /jffs/route-down.sh: line 44: can't create /proc/sys/net/ipv4/conf//rp_filter: nonexistent directory
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + echo 1
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + echo 1
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + ip route flush table 200
    Mar  4 05:59:01 unknown user.notice route-down.sh[7472]: + ip route flush cache
     
  88. Bogey

    Bogey Network Newbie Member

    @eangulus - I'm joining you on your quest, I hope you're OK with me barging in :)

    @eibgrad - do these two scripts route-up and route-down prevent leakage and have a 'kill switch'?
    I've read in numerous places to place this in the firewall rules:
    Code:
    iptables -I FORWARD -s 192.168.1.10 -o $(nvram get wan_iface) -j DROP
    iptables -I FORWARD -p tcp -s 192.168.1.10 -o $(nvram get wan_iface) --dport 51413 -j ACCEPT
    Is this (or isn't this) necessary or can the server not access the WAN when the VPN goes accidentally down?

    The above can be achieved by adding (altering) the following rule in route-up (and vice versa in route-down) script, right?
    Code:
    # route over VPN based on other criteria (e.g., protocol, source/destination port)
    iptables -t mangle -A PREROUTING -p tcp -m multiport ! --dport 21,22,80 -j MARK --set-mark $MARK
    Cheers,
    Bogey
     
    Last edited: Mar 10, 2015
  89. eibgrad

    eibgrad Addicted to LI Member

    I need to know if the following line in the route-down.sh script is still intact, and not accidentally been corrupted or changed:

    Code:
    WAN_GTWY="$route_net_gateway"       # provided by OpenVPN at runtime
     
  90. eangulus

    eangulus Network Guru Member

    It was sorry. Not sure how or why but I changed it back and all seems to be OK.

    Still have a problem with the way its working thou, at the moment, when the VPN is up, nothing works. Have no internet on anything from anywhere.

    And with the VPN down, all network gets WAN access, including the server which shouldn't.
     
  91. eibgrad

    eibgrad Addicted to LI Member

    To be honest, we've spent so much time just getting things to run, I'm exhausted. And I'm not even sure I remember exactly what the end game was here. LOL.

    Anyway, given that so many things have gone wrong, let's take this one step at a time. And for now all I want you to do is configure the scripts so that all the ip rules and prerouting rules are commented out. Also, any firewall rules we talked about previously for preventing access to the WAN by any devices should be removed as well.

    IOW, I want a clean, known starting point. I want to know for sure that the scripts work under the simplest configuration. And that means if you configure the scripts as I’m suggesting above, ALL traffic should be using the WAN, regardless whether the VPN is up and running, or shutdown. Because w/o any rules in the scripts, the ONLY thing the script does is prevent redirection from the WAN to the VPN. The WAN should still be available.

    Once we confirm that’s the case, THEN we can start adding rules to allow specific IPs, ports, whatever, to use the VPN.

    In fact, I’ve gone so far as to repost the scripts here w/ all the rules commented out so you see exactly how it should look. Again, I don't want you using any firewall rules in the firewall script either that might be attempting to block the WAN.

    route-up.sh:
    Code:
    #!/bin/sh -x
    (
    TID="200"
    MARK="0x88"
    WS="[[:space:]]"
    WAN_GTWY="$route_net_gateway"       # provided by OpenVPN at runtime
    WAN_IF="$(route | grep -Em1 ^${WAN_GTWY}${WS} | awk '{print $NF}')"
    VPN_GTWY="$route_vpn_gateway"       # provided by OpenVPN at runtime
    VPN_IF="$dev"                       # provided by OpenVPN at runtime
    REDIRECT_GTWY="$redirect_gateway"   # provided by OpenVPN at runtime
    
    # copy default/main routing table (exclude all default gateways)
    ip route flush table $TID > /dev/null 2>&1
    ip route show table main | grep -Ev "^default|^0.0.0.0/1${WS}|^128.0.0.0/1${WS}" \
      | while read route; do
            ip route add $route table $TID
        done
    # add VPN as default gateway
    ip route add default via $VPN_GTWY table $TID
    
    # add WAN back as default gateway in main/default routing table
    if [ "$REDIRECT_GTWY" == "1" ]; then
        ip route add 0.0.0.0/2   via $WAN_GTWY
        ip route add 64.0.0.0/2  via $WAN_GTWY
        ip route add 128.0.0.0/2 via $WAN_GTWY
        ip route add 192.0.0.0/2 via $WAN_GTWY
    fi
    
    # disable WAN/VPN reverse path filtering
    echo 0 > /proc/sys/net/ipv4/conf/$WAN_IF/rp_filter
    echo 0 > /proc/sys/net/ipv4/conf/$VPN_IF/rp_filter
    
    # clear the routing cache (or else it won't recognize our changes)
    ip route flush cache
    
    # route over VPN based on source IP(s)/network(s) or network interface
    #ip rule add from 192.168.1.10  table $TID
    #ip rule add from 10.10.1.113  table $TID
    #ip rule add from 10.10.2.0/24 table $TID
    #ip rule add iif wl0.1 table $TID
    
    # route over VPN based on other criteria (e.g., protocol, source/destination port)
    #iptables -t mangle -A PREROUTING -p tcp -s 10.10.1.42 --sport 22 -j MARK --set-mark $MARK
    #iptables -t mangle -A PREROUTING -p tcp -s 10.10.1.155 --sport 80 -j MARK --set-mark $MARK
    #iptables -t mangle -A PREROUTING -p tcp --dport 443 -j MARK --set-mark $MARK
    #iptables -t mangle -A PREROUTING -s jims-ipod -j MARK --set-mark $MARK
    #iptables -t mangle -A PREROUTING -m mac --mac-source 00:11:22:33:44:55 -j MARK --set-mark $MARK
    
    # start processing marked packets through the alternate routing table
    #ip rule add fwmark $MARK table $TID
    
    ) 2>&1 | logger -t $(basename $0)[$$]
    route-down.sh:
    Code:
    #!/bin/sh -x
    (
    TID="200"
    MARK="0x88"
    WS="[[:space:]]"
    WAN_GTWY="$route_net_gateway"       # provided by OpenVPN at runtime
    WAN_IF="$(route | grep -Em1 ^${WAN_GTWY}${WS} | awk '{print $NF}')"
    VPN_GTWY="$route_vpn_gateway"       # provided by OpenVPN at runtime
    VPN_IF="$dev"                       # provided by OpenVPN at runtime
    REDIRECT_GTWY="$redirect_gateway"   # provided by OpenVPN at runtime
    
    # remove routes based on source IP(s)/network(s) or network interface
    #ip rule del from 192.168.1.10  table $TID
    #ip rule del from 10.10.1.113  table $TID
    #ip rule del from 10.10.2.0/24 table $TID
    #ip rule del iif wl0.1 table $TID
    
    # remove routes based on other criteria (e.g., protocol, source/destination port)
    #iptables -t mangle -D PREROUTING -p tcp -s 10.10.1.42 --sport 22 -j MARK --set-mark $MARK
    #iptables -t mangle -D PREROUTING -p tcp -s 10.10.1.155 --sport 80 -j MARK --set-mark $MARK
    #iptables -t mangle -D PREROUTING -p tcp --dport 443 -j MARK --set-mark $MARK
    #iptables -t mangle -D PREROUTING -s jims-ipod -j MARK --set-mark $MARK
    #iptables -t mangle -D PREROUTING -m mac --mac-source 00:11:22:33:44:55 -j MARK --set-mark $MARK
    
    # stop processing marked packets through the alternate routing table
    #ip rule del fwmark $MARK table $TID
    
    # remove WAN as default gateway in main/default routing table
    if [ "$REDIRECT_GTWY" == "1" ]; then
        ip route del 0.0.0.0/2   via $WAN_GTWY
        ip route del 64.0.0.0/2  via $WAN_GTWY
        ip route del 128.0.0.0/2 via $WAN_GTWY
        ip route del 192.0.0.0/2 via $WAN_GTWY
    fi
    
    # re-enable WAN/VPN reverse path filtering
    echo 1 > /proc/sys/net/ipv4/conf/$WAN_IF/rp_filter
    echo 1 > /proc/sys/net/ipv4/conf/$VPN_IF/rp_filter
    
    # clear the alternate routing table and routing cache
    ip route flush table $TID
    ip route flush cache
    
    ) 2>&1 | logger -t $(basename $0)[$$]
     
  92. eangulus

    eangulus Network Guru Member

    That's fair enough.

    My whole router is 100% clean other than the bare minimum to get internet and the vpn. Have no port forwarding or anything else like that, not even QoS running. Remember that I wiped the NVRAM in trying to get things working.

    Anyway here is your code above results.

    VPN Up:
    Code:
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + TID=200
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + MARK=0x88
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + WS=[[:space:]]
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + WAN_GTWY=10.20.22.117
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + route
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + grep -Em1 ^10.20.22.117[[:space:]]
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + awk {print $NF}
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + WAN_IF=ppp0
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + VPN_GTWY=10.137.1.5
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + VPN_IF=tun11
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + REDIRECT_GTWY=1
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route flush table 200
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route show table main
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + grep -Ev ^default|^0.0.0.0/1[[:space:]]|^128.0.0.0/1[[:space:]]
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + read route
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 10.137.1.5 dev tun11 proto kernel scope link src 10.137.1.6 table 200
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + read route
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 10.137.1.1 via 10.137.1.5 dev tun11 table 200
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + read route
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 10.0.0.138 dev vlan2 proto kernel scope link src 10.0.0.137 table 200
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + read route
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 108.61.96.8 via 10.20.22.117 dev ppp0 table 200
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + read route
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 10.20.22.117 dev ppp0 proto kernel scope link src 115.64.85.241 table 200
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + read route
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.1 table 200
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + read route
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 127.0.0.0/8 dev lo scope link table 200
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + read route
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add default via 10.137.1.5 table 200
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + [ 1 == 1 ]
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 0.0.0.0/2 via 10.20.22.117
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 64.0.0.0/2 via 10.20.22.117
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 128.0.0.0/2 via 10.20.22.117
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route add 192.0.0.0/2 via 10.20.22.117
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + echo 0
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + echo 0
    Mar  5 15:12:07 ECS-ROUTER user.notice route-up.sh[6407]: + ip route flush cache
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: Initialization Sequence Completed
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: OpenVPN STATISTICS
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: Updated,Thu Mar  5 15:12:07 2015
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: TUN/TAP read bytes,0
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: TUN/TAP write bytes,0
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: TCP/UDP read bytes,4484
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: TCP/UDP write bytes,1867
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: Auth read bytes,0
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: pre-compress bytes,0
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: post-compress bytes,0
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: pre-decompress bytes,0
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: post-decompress bytes,0
    Mar  5 15:12:07 ECS-ROUTER daemon.notice openvpn[6317]: END
    VPN Up Routing Table:
    Code:
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
    10.137.1.5      0.0.0.0         255.255.255.255 UH    0      0        0 tun11
    10.137.1.1      10.137.1.5      255.255.255.255 UGH   0      0        0 tun11
    10.0.0.138      0.0.0.0         255.255.255.255 UH    0      0        0 vlan2
    108.61.96.8     10.20.22.117    255.255.255.255 UGH   0      0        0 ppp0
    10.20.22.117    0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
    192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 br0
    127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
    0.0.0.0         10.20.22.117    192.0.0.0       UG    0      0        0 ppp0
    64.0.0.0        10.20.22.117    192.0.0.0       UG    0      0        0 ppp0
    128.0.0.0       10.20.22.117    192.0.0.0       UG    0      0        0 ppp0
    192.0.0.0       10.20.22.117    192.0.0.0       UG    0      0        0 ppp0
    0.0.0.0         10.137.1.5      128.0.0.0       UG    0      0        0 tun11
    128.0.0.0       10.137.1.5      128.0.0.0       UG    0      0        0 tun11
    0.0.0.0         10.20.22.117    0.0.0.0         UG    0      0        0 ppp0
    VPN Down:
    Code:
    Mar  5 15:06:00 ECS-ROUTER daemon.notice openvpn[5992]: /jffs/route-down.sh tun11 1500 1542 10.139.1.6 10.139.1.5 init
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + TID=200
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + MARK=0x88
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + WS=[[:space:]]
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + WAN_GTWY=10.20.22.117
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + route
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + grep -Em1 ^10.20.22.117[[:space:]]
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + awk {print $NF}
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + WAN_IF=ppp0
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + VPN_GTWY=10.139.1.5
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + VPN_IF=tun11
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + REDIRECT_GTWY=1
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + ip rule del from 192.168.1.10 table 200
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + ip rule del fwmark 0x88 table 200
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + [ 1 == 1 ]
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + ip route del default via 10.20.22.117
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + echo 1
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + echo 1
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + ip route flush table 200
    Mar  5 15:06:00 ECS-ROUTER user.notice route-down.sh[6171]: + ip route flush cache
    Mar  5 15:06:00 ECS-ROUTER daemon.notice openvpn[5992]: /sbin/ifconfig tun11 0.0.0.0
    Mar  5 15:06:00 ECS-ROUTER daemon.notice openvpn[5992]: updown.sh tun11 1500 1542 10.139.1.6 10.139.1.5 init
    VPN Down Routing Table:
    Code:
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
    10.0.0.138      0.0.0.0         255.255.255.255 UH    0      0        0 vlan2
    10.20.22.117    0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
    192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 br0
    127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
    0.0.0.0         10.20.22.117    0.0.0.0         UG    0      0        0 ppp0 

    Also just for good measure here is the Default Routing Table on first boot:
    Code:
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
    10.0.0.138      0.0.0.0         255.255.255.255 UH    0      0        0 vlan2
    10.20.22.117    0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
    192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 br0
    127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
    0.0.0.0         10.20.22.117    0.0.0.0         UG    0      0        0 ppp0 

    Running with the above, Internet works on whole network via WAN. Working as you said it should.
     
  93. eibgrad

    eibgrad Addicted to LI Member

    Great!

    Now, I want you to add a rule for one IP address on your network that you ALWAYS want to use the VPN, unconditionally (I’m just picking 192.168.1.10 at random, use whatever you want). And for that, we use the "ip rule" section. So add one “ip rule add…” in route-up.sh, and the corresponding “ip rule del…” in route-down.sh.

    route-up.sh
    Code:
    ip rule add from 192.168.1.10  table $TID
    route-down.sh
    Code:
    ip rule del from 192.168.1.10  table $TID
    Verify this works.

    NOTE: In all these tests, don’t get overly concerned about whether or not these rules meet your ultimate goals. What I’m trying to do here is show you HOW to use the rules, just as examples. Once you understand the process, then you can tweak these rules any way you see fit.
     
  94. eangulus

    eangulus Network Guru Member

    Did that. But, everything works over WAN, 192.168.1.10 (is the one I want), is not getting anything. No WAN or VPN.
     
  95. eibgrad

    eibgrad Addicted to LI Member

    With the VPN up and running, run these commands from telnet/ssh and post the results.

    Code:
    ip route show table main
    ip route show table 200
    ip rule list
     
  96. eangulus

    eangulus Network Guru Member

    ip route show table main:
    Code:
    108.61.96.3 via 10.20.22.117 dev ppp0 
    10.196.1.1 via 10.196.1.5 dev tun11 
    10.0.0.138 dev vlan2  proto kernel  scope link  src 10.0.0.137 
    10.196.1.5 dev tun11  proto kernel  scope link  src 10.196.1.6 
    10.20.22.117 dev ppp0  proto kernel  scope link  src 115.64.85.241 
    192.168.1.0/24 dev br0  proto kernel  scope link  src 192.168.1.1 
    127.0.0.0/8 dev lo  scope link 
    0.0.0.0/2 via 10.20.22.117 dev ppp0 
    64.0.0.0/2 via 10.20.22.117 dev ppp0 
    128.0.0.0/2 via 10.20.22.117 dev ppp0 
    192.0.0.0/2 via 10.20.22.117 dev ppp0 
    0.0.0.0/1 via 10.196.1.5 dev tun11 
    128.0.0.0/1 via 10.196.1.5 dev tun11 
    default via 10.20.22.117 dev ppp0
    ip route show table 200:
    Code:
    108.61.96.3 via 10.20.22.117 dev ppp0 
    10.196.1.1 via 10.196.1.5 dev tun11 
    10.0.0.138 dev vlan2  proto kernel  scope link  src 10.0.0.137 
    10.196.1.5 dev tun11  proto kernel  scope link  src 10.196.1.6 
    10.20.22.117 dev ppp0  proto kernel  scope link  src 115.64.85.241 
    192.168.1.0/24 dev br0  proto kernel  scope link  src 192.168.1.1 
    127.0.0.0/8 dev lo  scope link 
    default via 10.196.1.5 dev tun11
    ip rule list:
    Code:
    0:    from all lookup local 
    32765:    from 192.168.1.10 lookup 200 
    32766:    from all lookup main 
    32767:    from all lookup default
     
  97. eibgrad

    eibgrad Addicted to LI Member

    Everything in those dumps looks perfect. Doesn't make sense why 192.168.1.10 would not have access to anything. Has to be something else going on there. Can you at least ping some internet IP from 192.168.1.10, say the Google DNS server @ 8.8.8.8? Maybe you need to close all your browser windows and try again.
     
  98. eangulus

    eangulus Network Guru Member

    Done via SSH:
    Code:
    eangulus@SERVER:~$ ping 8.8.8.8
    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    ^C
    --- 8.8.8.8 ping statistics ---
    187 packets transmitted, 0 received, 100% packet loss, time 187489ms
    
    Not sure if it helps (maybe in regards to my particular setup) but here is what was working (mostly) before via Custom Config, Firewall Script & WAN Up:
    Code:
    WORKING (Mostly)
    
    # OpenVPN Custom Config
    route-nopull
    
    # Firewall Script
    iptables -I FORWARD -i br0 -o tun11 -j ACCEPT
    iptables -I FORWARD -i tun11 -o br0 -j ACCEPT
    iptables -I INPUT -i tun11 -j DROP
    iptables -t nat -A POSTROUTING -o tun11 -j MASQUERADE
    
    # WAN Up Script
    sleep 30
    
    for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do
    echo 0 > $i
    done
    
    ip route flush table 200
    ip route flush cache
    iptables -t mangle -F PREROUTING
    
    VPN_GW=`ifconfig tun11 | awk '/inet addr/ {split ($2,A,":"); print A[2]}'`
    ip route add table 200 default via $VPN_GW dev tun11
    
    ip rule add fwmark 1 table 200
    ip route flush cache
    
    iptables -t mangle -A PREROUTING -i br0 -j MARK --set-mark 0
    iptables -t mangle -A PREROUTING -i br0 -m iprange --src-range 192.168.1.10 -j MARK --set-mark 1
    iptables -t mangle -A PREROUTING -i br0 -p tcp -m multiport --dport 21,22,80,3306,5050,7000,8081,8181,8888,9091,10000 -j MARK --set-mark 0
    iptables -t mangle -A PREROUTING -i br0 -p tcp -m multiport --sport 21,22,80,3306,5050,7000,8081,8181,8888,9091,10000 -j MARK --set-mark 0
    iptables -t mangle -A PREROUTING -i br0 -p udp -m multiport --dport 21,22,80,3306,5050,7000,8081,8181,8888,9091,10000 -j MARK --set-mark 0
    iptables -t mangle -A PREROUTING -i br0 -p udp -m multiport --sport 21,22,80,3306,5050,7000,8081,8181,8888,9091,10000 -j MARK --set-mark 0
     
  99. eangulus

    eangulus Network Guru Member

    Another odd part is that there is "some" traffic on the VPN. Here is the Stats for the VPN Connection:

    Code:
    TUN/TAP read bytes    570626
    TUN/TAP write bytes    0
    TCP/UDP read bytes    12760
    TCP/UDP write bytes    721750
    Auth read bytes    2464
    pre-compress bytes    0
    post-compress bytes    0
    pre-decompress bytes    0
    post-decompress bytes    0
     
  100. eangulus

    eangulus Network Guru Member

    Oh and another thing is, I did follow all instructions regarding the PIA VPN setup for newer tomato routers, except for leaving the option "Create NAT on tunnel". PIA says to check it, I have it uncheck (for manual routing like we are doing).

    Is that still OK? Or is your script assuming that it is checked?
     

Share This Page