1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

tomato and iptables redirect problem

Discussion in 'Tomato Firmware' started by samwathegreat, Nov 27, 2012.

  1. samwathegreat

    samwathegreat Serious Server Member

    Hi,

    I have installed lighttpd to my router, and configured it properly for access. I can access the content at 192.168.1.1:8080 (lighttpd is set to listen on port 8080)

    The reason I set it to listen on 8080 is so it won't interfere with tomato's GUI running on port 80.

    According to my understanding, the following rules are necessary to first open port 80, then port 8080 (necessary for the redirect to work?), then the 3rd command to redirect port 80 to 8080 (where lighttpd is listening)

    Code:
    iptables -A INPUT -p tcp --dport 80 -j ACCEPT
    iptables -A INPUT -p tcp --dport 8080 -j ACCEPT
    iptables -A PREROUTING -t nat -p tcp --dport 80 -j REDIRECT --to-port 8080
    the first two rules work as expected - in fact after running these two lines, I'm able to connect to the server on [dynamic wan ip]:8080

    the third rule, however, will not work. when I run the command, I get the following response:
    iptables doesn't seem to accept any directives that include "REDIRECT" in them.
    Can someone point me in the right direction? Is anyone else able to use redirect with iptables?
    I'm also open to other ways of doing this. For instance: lighttpd allows listen directives that can be bound to a specific ip. I wasn't able to use this (instead binding to all addresses) because my ip is dynamic, and this configuration wouldn't survive a WAN IP change. I was thinking that if I could bind lighttpd to a specific interface instead (vlan2 - my wan interface), these trouble could be avoided, because lighttpd would not interfere with tomato's web gui running on port 80
    Then I could simply use the first iptables rule above to open port 80 and avoid the "iptables: No chain/target/match by that name" issue all together.
    I'd really appreciate any help/advice. I think it's pretty obvious what I'm trying to do: Forward wan port 80 to port 8080 where lighttpd is listening. Everything is running on the same Tomato router, which is my gateway.
    Thanks again,
    samwathegreat
     
  2. rhester72

    rhester72 Network Guru Member

    This is not needed.
    This is correct.
    This should be:

    iptables -A WANPREROUTING -t nat -p tcp --dport 8080 -j DNAT --to-destination 192.168.1.1

    It isn't strictly necessary to do this, however. By default, port 80 (the admin interface) is only listening on 192.168.1.1 - you can bind lighttpd to 127.0.0.1 (i.e. localhost) on port 80 and adjust the above (two) rules to match (i.e. change --dport and --to-destination).

    This does, of course, mean you can only access lighttpd with the WAN IP/DNS name (since the forward will 'catch' it coming in on a non-standard interface and DNAT)...if you want it reachable internally as well, do something like...

    ip addr add 192.168.1.2 dev br0

    and then bind lighttpd to that IP and adjust the IP in the above rules - then it will be reachable from WAN and inside your network (at either the WAN IP or 192.168.1.2).

    Please don't ever bind to the WAN interface for exactly the reasons you gave. The firewall rules will automatically re-establish themselves after the WAN interface resets, so the above approach will always work.

    Rodney
     
  3. samwathegreat

    samwathegreat Serious Server Member

    Rodney,

    Thanks so much for the help. I still seem to be having issues, though. Here is what I have tried:

    1. Reboot router to get stock firewall
    2.


    (this single rule alone will allow me to access webserver from [wanip]:8080 - but I need to be able access from wan port 80)

    3.
    This doesn't seem to do anything in this context, because this same box is the router and gateway and server. The first rule already establishes that traffic on port 8080 is accepted.

    Obviously, this doesn't get me any further than I've been, because I'm not able to redirect wan port 80 to port 8080 where lighttpd is listening.

    So, Following your advice, I've tried the following:

    1. Set lighttpd to listen on 127.0.0.1 port 80

    2.
    This opens port 80 on wan, but doesn't allow me access because no rule is in place to send port 80 traffic to interface 127.0.0.1 where lighttpd is listening

    3.
    I changed the code as you suggested to send port 80 traffic to 127.0.0.1

    This doesn't work. For whatever reason, using this command does not forward wan traffic to localhost?


    Any ideas? This would have been fine... The important thing is to have the webserver listening on WAN, I really have no need to be able to access the webserver from inside the LAN.

    A couple ideas: Again, everything is running on one box. My router is running TomatoUSB, is the gatweay to my network, and is also running lighttpd. Are these rules appropriate for this setup?

    Also, going through my configuration, I've noticed that my NAT is set to MASQUERADE. I'm not sure if this has any effect on what I'm doing here?


    And finally, I also tried:

    1.
    2. Set lighttpd to listen on port 80, on interface 192.168.1.2

    I didn't get any further than this, because lighttpd fails to start with the listen directive set to 192.168.1.2

    I get the following error:


    Again, I really appreciate the help. I'm just not understanding why this isn't working.



    samwathegreat
     
  4. samwathegreat

    samwathegreat Serious Server Member

    I finally got it to work - not much thanks to iptables :p

    It turned out to be MUCH MUCH simpler than I thought:

    Lighttpd actually allows you to use hostname in the listen directive.

    So finally by setting:



    and adding the following single rule to iptables:

    Everything works without a hitch.

    The only thing I'm worried about here is how often lighttpd refreshes its DNS to resolve the hostname. I wasn't able to find an answer to that, but should it ever stop working, a reboot would get it back up and running (re-resolve the hostname with DNS)

    I'm not too concerned because my WAN IP has only changed a couple of times in the past year, so hopefully it's a non-issue.

    If you have anything else to suggest or point out anything I've overlooked, I'd appreciate it.

    Thanks again!
     
  5. koitsu

    koitsu Network Guru Member

    It doesn't. Ever. It does the resolution one time when the daemon starts up and that's it. This is by design -- others (nginx, Apache, etc.) do the same thing. There are many technical reasons for this which I'd rather not go into.

    You don't need to reboot to pick up the DNS change either -- you just need to send SIGHUP to dnsmasq (to clear its DNS cache) and restart lighttpd (in that order).
     
  6. samwathegreat

    samwathegreat Serious Server Member

    Ok, perfect, thanks koitsu. If my understanding is right, I just need to add a couple of lines to the WAN up script.

    The 2nd line is pretty obvious, I'd put in:

    But I don't understand how I'm supposed to send SIGHUP to dnsmasq. I *thought* it would be as simple as:

    it seems, however, that the kill function does not recognize a process by its name - I get the following error:

    So it seems that the kill -s SIGHUP wants a PID
    Running the top command shows me that dnsmasq has PID 558
    I'm not understanding if this will ever change? Will dnsmasq's PID always be 558? Then I could use:
    in my WAN UP script.
    Is there a better way? What command do you suggest for sending SIGHUP to dnsmasq?
    Thanks again
     
  7. rhester72

    rhester72 Network Guru Member

    Remember, any rule on INPUT applies to *all* interfaces (WAN and LAN. including loopback) unless you specify an interface.

    Code:
    iptables -A WANPREROUTING -t nat -p tcp --dport 80 -j DNAT --to-destination 127.0.0.1
    
    You said this didn't work? That is _precisely_ what I'm doing with Hiawatha and it works like a charm.

    How exactly did you do the above - ssh? You didn't restart the firewall after (which would clear the rules), did you? What's the output of iptables -t nat -L -nv after you issue the command (before and after trying to hit it from the WAN)?

    Also, bear in mind that internal testing (i.e. inside your network) WILL fail unless Advanced/Firewall/NAT loopback is set to "All" (I believe the default is Forwarded Only, which will make it work outside your network but render it impossible to test inside). (NOTE: Changing this setting will reload the firewall rules and delete any manual entries! I put mine in /opt/etc/config/ipv4.fire for convenience. Make sure to #!/bin/sh the first line and set chmod u+x or it won't fire!)

    I don't know why lighttpd didn't like the alternate IP - sounds like there was a typo in the config file somewhere. It'll bind to any valid IP, as you've seen with the 127.0.0.1 example.

    I really, REALLY would advise against EVER binding ANYTHING directly to your WAN address...things can and will go completely off the reservation if you get a new IP from your upstream provider.

    Rodney
     
  8. samwathegreat

    samwathegreat Serious Server Member


    Code:
    root@stinky:/tmp/home/root# iptables -t nat -L -nv
    Chain PREROUTING (policy ACCEPT 11 packets, 721 bytes)
    pkts bytes target    prot opt in    out    source              destination
        0    0 ACCEPT    udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:443
    2546  296K WANPREROUTING  all  --  *      *      0.0.0.0/0            99.999.999.999 (wan ip masked)
        0    0 DROP      all  --  vlan2  *      0.0.0.0/0            192.168.1.0/24
    2208  276K upnp      all  --  *      *      0.0.0.0/0            99.999.999.999 (wan ip masked)
     
    Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
    pkts bytes target    prot opt in    out    source              destination
    4201  291K MASQUERADE  all  --  *      vlan2  0.0.0.0/0            0.0.0.0/0
      207 31212 SNAT      all  --  *      br0    192.168.1.0/24    192.168.1.0/24    to:192.168.1.1
     
    Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
    pkts bytes target    prot opt in    out    source              destination
     
    Chain WANPREROUTING (1 references)
    pkts bytes target    prot opt in    out    source              destination
      64  5376 DNAT      icmp --  *      *      0.0.0.0/0            0.0.0.0/0          to:192.168.1.1
      10  584 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:5050 to:192.168.1.50:5050
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:5050 to:192.168.1.50:5050
        8  480 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:6050 to:192.168.1.50:6050
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:6050 to:192.168.1.50:6050
        7  404 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:5051 to:192.168.1.51:5051
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:5051 to:192.168.1.51:5051
        5  300 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:6051 to:192.168.1.51:6051
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:6051 to:192.168.1.51:6051
     
    Chain upnp (1 references)
    pkts bytes target    prot opt in    out    source              destination
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:52059 to:192.168.1.4:52059
    root@stinky:/tmp/home/root# iptables -A WANPREROUTING -t nat -p tcp --dport 80 -j DNAT --to-destination 127.0.0.1
    root@stinky:/tmp/home/root# iptables -t nat -L -nv
    Chain PREROUTING (policy ACCEPT 7 packets, 488 bytes)
    pkts bytes target    prot opt in    out    source              destination
        0    0 ACCEPT    udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:443
    2568  298K WANPREROUTING  all  --  *      *      0.0.0.0/0            99.999.999.999 (wan ip masked)
        0    0 DROP      all  --  vlan2  *      0.0.0.0/0            192.168.1.0/24
    2230  278K upnp      all  --  *      *      0.0.0.0/0            99.999.999.999 (wan ip masked)
     
    Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
    pkts bytes target    prot opt in    out    source              destination
    4245  295K MASQUERADE  all  --  *      vlan2  0.0.0.0/0            0.0.0.0/0
      209 31630 SNAT      all  --  *      br0    192.168.1.0/24    192.168.1.0/24    to:192.168.1.1
     
    Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
    pkts bytes target    prot opt in    out    source              destination
     
    Chain WANPREROUTING (1 references)
    pkts bytes target    prot opt in    out    source              destination
      64  5376 DNAT      icmp --  *      *      0.0.0.0/0            0.0.0.0/0          to:192.168.211.1
      10  584 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:5050 to:192.168.1.50:5050
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:5050 to:192.168.1.50:5050
        8  480 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:6050 to:192.168.1.50:6050
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:6050 to:192.168.1.50:6050
        7  404 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:5051 to:192.168.1.51:5051
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:5051 to:192.168.1.51:5051
        5  300 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:6051 to:192.168.1.51:6051
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:6051 to:192.168.1.51:6051
        0    0 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:80 to:127.0.0.1
     
    Chain upnp (1 references)
    pkts bytes target    prot opt in    out    source              destination
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:52059 to:192.168.1.4:52059
    
    Rodney,

    I'm trying to get your solution, and as shown above, the rule seems to have taken effect.

    However, I'm not able to access the webserver (from inside or outside the lan) after configuring lighttpd to listen on 127.0.0.1

    Everything looks right to me. Also, referencing your previous post, my NAT loopback is set to ALL, which is the default on the shibby build I'm using.

    I would prefer to get this working as it is a more robust solution than having to add extra lines to WAN UP script to clear dns cache and restart lighttpd if the wan changes.

    Hopefully I've missed something simple. Please advise.
     
  9. samwathegreat

    samwathegreat Serious Server Member

    Code:
    root@stinky:/tmp/home/root# iptables -t nat -L -nv
    Chain PREROUTING (policy ACCEPT 2061 packets, 152K bytes)
    pkts bytes target    prot opt in    out    source              destination
        0    0 ACCEPT    udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:443
    2761  320K WANPREROUTING  all  --  *      *      0.0.0.0/0            50.129.171.147
        0    0 DROP      all  --  vlan2  *      0.0.0.0/0            192.168.211.0/24
    2393  298K upnp      all  --  *      *      0.0.0.0/0            50.129.171.147
     
    Chain POSTROUTING (policy ACCEPT 15 packets, 5020 bytes)
    pkts bytes target    prot opt in    out    source              destination
    4521  316K MASQUERADE  all  --  *      vlan2  0.0.0.0/0            0.0.0.0/0
      219 33350 SNAT      all  --  *      br0    192.168.211.0/24    192.168.211.0/24    to:192.168.211.1
     
    Chain OUTPUT (policy ACCEPT 200 packets, 21512 bytes)
    pkts bytes target    prot opt in    out    source              destination
     
    Chain WANPREROUTING (1 references)
    pkts bytes target    prot opt in    out    source              destination
      72  6048 DNAT      icmp --  *      *      0.0.0.0/0            0.0.0.0/0          to:192.168.211.1
      10  584 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:5050 to:192.168.211.50:5050
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:5050 to:192.168.211.50:5050
        8  480 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:6050 to:192.168.211.50:6050
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:6050 to:192.168.211.50:6050
        7  404 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:5051 to:192.168.211.51:5051
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:5051 to:192.168.211.51:5051
        5  300 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:6051 to:192.168.211.51:6051
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:6051 to:192.168.211.51:6051
      22  1256 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:80 to:127.0.0.1
     
    Chain upnp (1 references)
    pkts bytes target    prot opt in    out    source              destination
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:52059 to:192.168.211.4:52059
    root@stinky:/tmp/home/root#
    
    Rodney,

    After hitting it from the wan, it shows packets received. Is there a reason this isn't working? My lighttpd configuration is solid - so far I know it works fine bound to all interfaces, as well as 192.168.1.1, and it works with my hostname for wan interface.

    Am I missing another rule that is keeping the traffic from getting to lighttpd?

    Or perhaps the traffic can't make its way back FROM lighttpd?


    I'm confused...
     
  10. rhester72

    rhester72 Network Guru Member

    OK...you still have the basic:

    iptables -A INPUT -p tcp --dport 80 -j ACCEPT

    in place, correct? (iptables -t filter -L -nv should help you verify and see the trigger count)

    Traffic should get back from lighttpd just fine, since it's only a DNAT.

    Does lighttpd show up as bound to the correct interface if you issue "netstat -an | grep LISTEN"?

    Beyond those rather simple (and unlikely) possibilities, I have to admit I'm stumped as well.

    Rodney
     
  11. samwathegreat

    samwathegreat Serious Server Member

    Code:
    root@stinky:/tmp/home/root# netstat -an | grep LISTEN
    tcp        0      0 0.0.0.0:9091            0.0.0.0:*              LISTEN
    tcp        0      0 0.0.0.0:28453          0.0.0.0:*              LISTEN
    tcp        0      0 192.168.1.1:139      0.0.0.0:*              LISTEN
    tcp        0      0 127.0.0.1:80            0.0.0.0:*              LISTEN
    tcp        0      0 192.168.1.1:80        0.0.0.0:*              LISTEN
    tcp        0      0 0.0.0.0:2000            0.0.0.0:*              LISTEN
    tcp        0      0 0.0.0.0:53              0.0.0.0:*              LISTEN
    tcp        0      0 0.0.0.0:46838          0.0.0.0:*              LISTEN
    tcp        0      0 0.0.0.0:22              0.0.0.0:*              LISTEN
    tcp        0      0 0.0.0.0:51515          0.0.0.0:*              LISTEN
    tcp        0      0 192.168.1.1:445      0.0.0.0:*              LISTEN
    tcp        0      0 :::53                  :::*                    LISTEN
    tcp        0      0 :::22                  :::*                    LISTEN
    tcp        0      0 :::23                  :::*                    LISTEN
    tcp        0      0 :::51515                :::*                    LISTEN
    unix  2      [ ACC ]    STREAM    LISTENING      1588 /opt/var/run/asterisk.ctl
    unix  2      [ ACC ]    STREAM    LISTENING    420431 /tmp/php-fcgi.sock-0
    
    Rodney,

    It appears to be listening on 127.0.0.1 with the listen directive set as such.

    Yes, I have:

    in place. Without this, port 80 remains closed, and I wouldn't expect any of this to work. Am I missing a rule that routes traffic FROM localhost back out, or that allows traffic originating from localhost to go out the wan interface? So strange that everything works as expected when binding to the WAN ip...this has got to be a missing rule or something....

    I will continue to troubleshoot. If you see anything out of place, please chime in.


    samwathegreat
     
  12. koitsu

    koitsu Network Guru Member

    For sending SIGHUP to dnsmasq, what you want is this:

    Code:
    kill -HUP `cat /var/run/dnsmasq.pid`
    
    That simple. Please note those are backticks (`), as in the tilde (~) key but without shift held. They are not apostrophes.

    For details of what sending SIGHUP to dnsmasq fully does, read the docs (see NOTES section near bottom).
     
  13. rhester72

    rhester72 Network Guru Member

    Love the hostname, BTW. ;)

    I concur - that's good. Anything lighty might be doing to restrict use of 127.0.0.1? (I don't use it myself, so I don't know.)

    Concur. When you try to hit from WAN, both rules should be triggered - first WANPREROUTING, then INPUT. Are counters for both increasing during your tests?

    No, DNAT unrolls automatically within the stack.

    *scratches head* If you want to try a different one to "prove" to yourself that it works, make *sure* that you have a good, solid password set up on your Web admin interface, then change the --to-destination to 192.168.1.1 (leaving the INPUT rule alone and changing nothing else). That should expose your Web admin interface to the outside world on port 80. I'm *quite* sure this works. ;) (Just trying to rule out lighttpd here...)

    I'm reasonably certain your ISP passes through port 80 unmolested or else the WANPREROUTING rule would be never be triggered.

    Rodney
     
  14. samwathegreat

    samwathegreat Serious Server Member


    Thanks Rodney,

    I tried everything I could imagine to get lighttpd to respond on localhost, to no avail. I used putty to ssh into the router and tunnel port 80 from localhost, and I was able to pull up my webserver! I just could not figure out why it wasn't accessible from wan ip using the rules you gave me...

    I've learned a whole lot along the way, and I revisited your very first post. You suggested trying to add another interface using

    When I had tried this before, lighttpd gave me an error. I figured out that I needed quotations around the ip address:

    With the quotation marks, I no longer received an error, lighttpd started successfully, and I was greeted by my webserver at 192.168.1.2

    So I added a port forward from the web gui pointing port 80 to 192.168.1.2

    Didn't work. So I added your rule:

    And what do you know? Now I can access the webserver from both inside and outside the lan :)


    So having finally figured out what I would consider an excellent solution, I added the following to the firewall script:

    Code:
    ip addr add 192.168.1.2/24 dev $(nvram get lan_ifname)
    iptables -A INPUT -p tcp --dport 80 -j ACCEPT
    and saved the port forward in web gui forwarding port 80 to 192.168.1.2

    I rebooted to find that everything works perfectly as expected.


    I REALLY appreciate all the help and advice - again I've learned alot. Using all of this logic, I thought it would be a good idea to give my asterisk server (also running on same router) its own IP to bind to (it was previously bound to al interfaces - which of course we know won't survive a WAN ip change), so I set all of the asterisk listen directives to 192.168.1.3, and added:

    Code:
    ip addr add 192.168.1.3/24 dev $(nvram get lan_ifname)
    iptables -A INPUT -p tcp --dport (*my asterisk port*) -j ACCEPT
    and added a corresponding port forward in the web gui forwarding my asterisk port to 192.168.1.3

    Everything works better than before! I'm not sure what routing differences are going on, but now I can access the asterisk server from inside and outside my lan using hostname. Before, I could not connect using hostname from inside the lan - I had to use the ip address. Now it works both ways!

    I'm already seeing additional benefits to this. For instance, I can enable and run Asterisk's bulleting mini-http server on port 80 of the new ip (192.168.1.3) and easily pull it up without having to use a non-standard port to avoid interfering with tomato's web gui. Excellent!

    What do you think? Is this the best way to accomplish what I've done? I believe that both the asterisk server and lighttpd will both survive a wan ip change using this configuration, right?
     
  15. rhester72

    rhester72 Network Guru Member

    They will survive a WAN change. A few brief observations:

    - Don't add the IP in the firewall script, since it will be executed during ANY firewall change or WAN event, and will duplicate the add. Better to do it once in the init script (it will work the way you have it, but just cause needless harmless dupe IP failures later). I'd personally add the IPs to lo instead of br0 myself, since under certain rare circumstances the bridge itself will be rebuilt (and you will lose the custom IPs).

    - I'm shocked anything works with the single INPUT rule. Here's the way iptables works:

    First stop, WANPREROUTING after the packet enters the WAN. This is where you want to do your destination NATs to tell them where to go on the inside network, hence the --to-destination stuff. Then it enters the INPUT chain (with a new destination IP), and the rule there marks it as passable inside the network with a WAN source IP and a LAN destination IP. Without the WANPREROUTING rule, the packet enters with a source IP from the Internet and a destination IP of your WAN interface...and since there's no listener on the WAN interface, the packet should get dumped into the bit-bucket. Why this isn't happening in your case is a REAL mystery to me unless somehow you think lighttpd and Asterisk are binding to a single local interface but they are instead binding to 0.0.0.0 (i.e. all interfaces).

    I am very happy to see that you got things working, at least to a certain degree, and yes, that's precisely why I asked that the internal webserver bind only to the primary LAN interface and not 0.0.0.0 (what Tomato did long ago). It does open up a tremendous degree of flexibility. For instance, on my router, transmission is bound to its own internal IP for incoming and outgoing packets - making QoS a snap, since I can apply a single QoS rule to that dedicated IP and be assured that ALL torrent traffic (from and to the transmission daemon on the router, obviously not clients internal to the network) is managed properly.

    Please use this as a jumping-off point for more experimentation and learning with the router - that's a huge part of the joy of custom firmware! And don't forget to pay it forward...there are always new members here looking for a bit of guidance to get them to the point of being ready for self-exploration. ;)

    Rodney

    UPDATE: Hrm...I bet you're doing the WANPREROUTING without realizing it. The approach I described above is _independent_ of the Forwarding page of the GUI, since it assumes the destination is a node on your LAN that _isn't_ the router (it won't work for applications *on* the router). It will, however, build the proper WANPREROUTING rule (just not the INPUT rule). Did you, by chance, set up a forward in the GUI *and* set up the hand-built INPUT line above? If so, that would explain why things are working "without WANPREROUTING" (it's happening behind the scenes), and the optimal solution would be to remove the forwarding rule entirely from the GUI and do the WANPREROUTING in your firewall script before the INPUT rule.

    Why it can't work without WANPREROUTING:

    Say you build virtual IPs 192.168.1.2, 192.168.1.3, and 192.168.1.4. You run Apache on 1.2, lighttpd on 1.3, and Hiawatha on 1.4, all listening on port 80. A packet arrives on the WAN with a destination of port 80. Without a WANPREROUTING rule including an internal --to-destination IP, how should the router choose which of the three web servers to send the packet to? (Hint: Without the rule, it won't send it to any of them. Unless there's a port 80 listener on your WAN interface, the packet will be silently dropped.)
     
  16. samwathegreat

    samwathegreat Serious Server Member


    Rodney:

    You're exactly right. I've checked it output of

    Code:
    Chain WANPREROUTING (1 references)
    pkts bytes target    prot opt in    out    source              destination
        2  120 DNAT      icmp --  *      *      0.0.0.0/0            0.0.0.0/0          to:192.168.1.1
        3  156 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:5050 to:192.168.1.50:5050
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:5050 to:192.168.1.50:5050
        0    0 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:6050 to:192.168.1.50:6050
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:6050 to:192.168.1.50:6050
        3  156 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:5051 to:192.168.1.51:5051
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:5051 to:192.168.1.51:5051
        0    0 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:6051 to:192.168.1.51:6051
        0    0 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:6051 to:192.168.1.51:6051
      124  6872 DNAT      tcp  --  *      *      0.0.0.0/0            0.0.0.0/0          tcp dpt:80 to:192.168.1.2:80
        1    74 DNAT      udp  --  *      *      0.0.0.0/0            0.0.0.0/0          udp dpt:22222 to:192.168.1.3:22222
    
    So putting the rule in the web gui adds the appropriate wanprerouting rule to iptables. It also adds some entries to chain "wanin", which I think may be the reason that nat loopback is working now for my asterisk server.

    Code:
    Chain wanin (1 references)
    target    prot opt source              destination
    ACCEPT    tcp  --  anywhere            security_dvr1.mydomain.info tcp dpt:mmcc
    ACCEPT    udp  --  anywhere            security_dvr1.mydomain.info udp dpt:mmcc
    ACCEPT    tcp  --  anywhere            security_dvr1.mydomain.info tcp dpt:6050
    ACCEPT    udp  --  anywhere            security_dvr1.mydomain.info udp dpt:6050
    ACCEPT    tcp  --  anywhere            security_dvr2.mydomain.info tcp dpt:5051
    ACCEPT    udp  --  anywhere            security_dvr2.mydomain.info udp dpt:5051
    ACCEPT    tcp  --  anywhere            security_dvr2.mydomain.info tcp dpt:6051
    ACCEPT    udp  --  anywhere            security_dvr2.mydomain.info udp dpt:6051
    ACCEPT    tcp  --  anywhere            www.mydomain.info        tcp dpt:www
    ACCEPT    udp  --  anywhere            asterisk.mydomain.info  udp dpt:22222
    
    These entries aren't added when just using the WANPREROUTING rule, and not the web gui port forwarding. I'm not exactly sure what this is doing.

    Also, I tried adding the code:

    To the init script, but upon a reboot the ip address was not created. I tried this several times. I'm thinking there may be some sort of race condition preventing the ip address from being added at init.

    Adding the code to the firewall script works as expected. I realize that it's possible that the firewall will restart, but I'm only expecting to see something like this in my logs if that were to happen:

    This is the output I receive when I run "ip addr add 192.168.1.2/24 dev br0"

    After it's already been run.
     
  17. rhester72

    rhester72 Network Guru Member

    I _think_ those are what make "Forwarded only" work, and aren't needed for "All" (i.e. NAT loopback).

    You'll note that entries are also added to FORWARD in the filter table. Those are not needed or used for local applications, only genuine forwards. That's why I advise not using the GUI for "local" apps.

    It's possible...br0 is built somewhat late in the process. Again, that's why I suggest lo instead of br0 (in addition to the notion that br0 can be rebuilt under certain circumstances, where lo will never be).

    This surprises me, because the syntax is wrong. /24 is a subnet, /32 is the mask for a single node.

    Agreed. If you redirect 2>/dev/null at the end of the line, it will never show the error, which is fine.

    Rodney
     
  18. samwathegreat

    samwathegreat Serious Server Member


    Rodney,

    You're right again.

    I took your advice and changed the syntax to

    and moved the code from the firewall script to the init script. I think you were exactly right about br0 not being built when the init script is run. It seems that lo is already built, because it works.

    Not only do the port forwarding rules work, but the ip is still routable from inside the lan. I'm hoping this is the final solution, as it should be foolproof to both wan ip change and the possibility of br0 being rebuilt.

    About entries being added to the FORWARD chain... It doesn't look like it. Instead, it looks like the FORWARD chain points to chain "wanin"

    Code:
    Chain FORWARD (policy DROP)
    target    prot opt source              destination
    ACCEPT    all  --  anywhere            anywhere
              all  --  anywhere            anywhere            account: network/ne                                            tmask: 192.168.1.0/255.255.255.0 name: lan
    ACCEPT    all  --  anywhere            anywhere
    DROP      all  --  anywhere            anywhere            state INVALID
    TCPMSS    tcp  --  anywhere            anywhere            tcp flags:SYN,RST/S                                            YN TCPMSS clamp to PMTU
    L7in      all  --  anywhere            anywhere
    ACCEPT    all  --  anywhere            anywhere            state RELATED,ESTAB                                            LISHED
    wanin      all  --  anywhere            anywhere
    wanout    all  --  anywhere            anywhere
    ACCEPT    all  --  anywhere            anywhere
    upnp      all  --  anywhere            anywhere
    
    I'm hoping I haven't overlooked anything, and that this config is finally solid. Thanks again Rodney.
     
  19. rhester72

    rhester72 Network Guru Member

    You are right - FORWARD is used in Tomato to basically route packets from/to the LAN and WAN, but only for nodes _other than the router itself_, where INPUT and OUTPUT come into play. Thus, for anything on the router, two manual rules - one filter/INPUT, one nat/WANPREROUTING - are all you need. Everything else (i.e. to other destination nodes in your network) are better handled in the GUI.

    Sounds like you have a pretty idealized setup at this point - congratulations! I hope any knowledge you've gained serves you well in the future with Tomato...there's a _LOT_ you can do with these little things. ;)

    Rodney
     
  20. samwathegreat

    samwathegreat Serious Server Member

    Rodney,

    I don't know how I didn't figure all of this out earlier in our dialog, but I've got it all setteled.

    I deleted the rules I had made from the port forwarding section in GUI, and added the appropriate WANPREROUTING rule to firewall as you suggested. Everything still works perfectly both inside and outside the lan. Just as you said, it only required 2 rules, the INPUT rule, and corresponding WANPREROUTING rule.

    My advice to anyone wanting to do something similar (run http server, asterisk server, etc) is to add an IP for the http server to use to INIT script:

    set the listen directive on server to listen on that IP. Next, add INPUT RULE to FIREWALL script

    substuting protocol and port number, of course, then add WANPREROUTING rule to FIREWALL script

    again substuting protocol, port number, and matching IP address. Reboot, and you should be all set :) No need to put anything in port forwarding section of the web GUI.

    And as in my example, you can add multiple IPs for different servers depending on your needs.

    Looking back through this thread, I've definitely made this way more complicated that it needed to be, but the one thing that I think complicated everything is that apparently lighttpd REFUSES to answer on 127.0.0.1, but, as we've discovered, it's happy to answer on any other IP added to the lo interface.

    Thanks for all of the excellent info. Good luck everybody.
     

Share This Page