Help with VLAN problem

Discussion in 'Tomato Firmware' started by Toastman, Oct 5, 2010.

  1. Toastman

    Toastman Super Moderator Staff Member Member

    I have a new installation on an RT-N16 using two extra vlans, vlan4 and 5. Only one is in use at the moment, this is vlan4 and is on 10.0.5.xx subnet. I have two devices, a laser printer/copier, and a Belkin Skype phone on it along with 6 desktop machines. All assigned static IP's via DHCP. Firmware is Teddy Bear's latest ext. release.

    The normal LAN subnet on br0 (192.168.1.xx) functions as normal and UPnP ports open correctly, all lists and QOS functioning properly.

    When I first set this up, for a very short time all desktops on vlan4 appeared in the device list in br0 (not vlan4) and at that time the machines were correctly shown in the QOS-Detailed and QOS graph classes. However, after a while they changed in the device list to be in vlan4, and after that they no longer appear correctly in the WOL list, QOS detailed list or the QOS graph display. They seem to show as external WAN connections to the router's WAN IP address and stop there. However each machine can surf the internet just fine. It also seems that they cannot open ports with UPnP either (some of them have Skype installed and UPnP is enabled by default).

    Now, here's a clue - the Skype phone and the Laser printer have ports forwarded to their admin interfaces on port 80. I can see them remotely from the WAN and once accessed, they show in the device list correctly and in the correct QOS classes. These devices therefore seem to have no problems.

    The obvious difference here is that the desktop machines don't have their ports forwarded.

    I'm assuming QOS should work on packets crossing into the WAN interface without caring where the packets originated. Something isn't quite right here, looks like some iptables scripting is missing or the firewall script needs work. I've searched for a full day and read dozens of articles, but while a few questions have been asked, there doesn't seem to be much information on this topic.

    Can anyone out there help? Anyone know how to fix it?

    The setup here is taken from various articles on the forum


    sleep 10
    nvram set vlan1ports="2 1 8*"
    nvram set vlan3hwname=et0
    nvram set vlan3ports="4 8*"
    nvram set vlan4hwname=et0
    nvram set vlan4ports="3 8*"
    nvram set manual_boot_nv=1
    sleep 10
    ifconfig vlan3 netmask up;
    ifconfig vlan4 netmask up;


    iptables -I INPUT -i vlan3 -j ACCEPT;
    iptables -I FORWARD -i vlan3 -o vlan2 -m state --state NEW -j ACCEPT;
    iptables -I FORWARD -i vlan3 -o ppp0 -m state --state NEW -j ACCEPT;
    iptables -I FORWARD -i br0 -o vlan3 -j DROP;

    iptables -I INPUT -i vlan4 -j ACCEPT;
    iptables -I FORWARD -i vlan4 -o vlan2 -m state --state NEW -j ACCEPT;
    iptables -I FORWARD -i vlan4 -o ppp0 -m state --state NEW -j ACCEPT;
    iptables -I FORWARD -i br0 -o vlan4 -j DROP;

    iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu;

    DHCP/DNS Custom



    netstat -r

    Kernel IP routing table
    Destination Gateway Genmask Flags MSS Window irtt Iface UGH 0 0 0 ppp0 * UH 0 0 0 ppp0 UGH 0 0 0 ppp0 * U 0 0 0 vlan4 * U 0 0 0 vlan3 * U 0 0 0 br0 * U 0 0 0 lo
    default UG 0 0 0 ppp0

    Thanks for listening !
  2. Porter

    Porter LI Guru Member

    Were you at some point able to regain the initial flawless status (through reboot or restart of some services?)

    You could try to monitor the way the packets take around your different interfaces on the router and between the machines themselves and LAN vs. WAN. Tools for this would be traceroute, netstat -r (on the router) and wireshark (or some similar program for the router which hopefully is available). Or you could try printing from the clients on vlan4 and then some external clients on br0. Stuff like that.

    Is vlan4 still availabe when this happens (output of ifconfig?)

    In my experience things that "seem" to happen definitely ask for further inspection:
    Meaning that those connections are getting dropped? Where did you find clues for your interpretation?

    Take a look at how your machines are being NATed by "cat /proc/net/ip_conntrack > conntrack" and then greping for IPs.

    I think somewhere in the GUI I saw an option where DROPed packets would get logged. That might help, too.

    Just to clearify something:
    So these two devices need to be accessable from the internet?

    Hope this can help. Keep us posted!
  3. Toastman

    Toastman Super Moderator Staff Member Member

    The vlan4 in use works fine all the time as far as I am aware, nobody has complained that they have lost internet access except on one single occasion which could just have been a power brick fallen out or something (not worried about it since it hasn't happened again).

    I should have stated in my first post that I am rather limited because this is a business setup many miles from here, and I have no easy access to it other than remotely.

    The 2 devices I mention don't NEED to be accessed from the internet, I can just set port forwards to them remotely to use as tools, to try to see what is going on. But these machines show correctly and the only difference is that they are port forwarded. i.e. it appears that the QOS ***CAN*** work correctly on vlan4 if only the routing/scripts or (what?) were correctly done.

    I can't say I have been able to regain the original state where the vlan machines showed up as br0. I don't know why they did that - actually I don't know what it is supposed to show, but vlan4 seems correct. However, if QOS processes br0 >WAN then I assume that somehow vlans 3 and 4 need to be attached to br0 in some way. Remember I don't understand iptables scripts much, today I will be trying to learn a little more though.

    The packets I see on the detailed list that are from outside internet addresses and appear to stop at the WAN IP, I believe are actually in use on vlan4. But I'm not 100% sure. Sometimes I see connections that also stop at the WAN IP address that have been classified as VOIP - and I know some of the workstations use Skype. Again, they might be incoming connections - could that be another clue? It's a lot of guesswork really and I am out of my depth here - that's why I need help! I was hoping the regular vlan users might have an answer already.

    I am currently looking at all examples I can find in the forums of people's scripts to see if any of them work, there are a few variations. But it would help if I knew what I was doing :biggrin:

    I added the netstat -r output to the above post.
    cat /proc/net/ip_conntrack > conntrack doesn't produce any output.
  4. Porter

    Porter LI Guru Member

    You could probably try to use gain access to the six clients via remotedesktop. As far as I remember the only hurdle would be, that it's not activated by default, so you would need to guide someone through the process of activiating it for you. Then one client is similar to the two devices.

    This command:
    cat /proc/net/ip_conntrack > conntrack

    actually produces output. That is in a file called conntrack. Open it on the console with "less conntrack". Quit viewing by hitting "q". The Output will be difficult to read, so to only get the important stuff you should use grep, i.e. "grep conntrack" which should print out all the lines in which this IP occurs. With that you could find out how traffic is being processed (aka solve the mystery, why the clients don''t seem to have connection problems).

    By the way: are theses clients always on or do they get shut down?

    My course of action would be to first make sure that the vlan-config is working correctly (therefore monitoring NAT to see how the packets are getting processed) and after that worrying about iptables+tc.

    If you really need to go heavy on knowing what's happening on your network you could use tcpdump. Someone seems to have compiled it already:

    But be warned: depending on how big the traffic flowing through the specified interface is, the faster this protocol can get really large. Also it's a breach of privacy, since you will know erverything the users are doing, except encrypted stuff...
  5. Toastman

    Toastman Super Moderator Staff Member Member

    vlans and QOS

    I think I've read everything ever posted about vlans on the net, my eyeballs are toast. Interesting that most posters airily mention using QOS but carefully evaded the subject when someone queried it. Hmmm.

    Now, after watching for a long time, I think I see that QOS actually correctly classifies connections from the 2 vlans, so perhaps the QOS is in fact working, but it's impossible to be 100% certain, or see what machine it is because only the router's WAN IP shows in the lists. By using the new webmon facility it's possible to see that websites were in fact being accessed though. Here are some examples.


    Anyway, all articles are basically correct in what they say. They all successfully set up vlans that work but the full integration with Tomato QOS displays has never been addressed, as far as I can see. Possibly it might be done by the use of a bridge ... there are hints about this but no practical examples, so I am still grasping at straws. The fact that the display is correct in every way when a port is forwarded to a machine and accessed from outside must be the best clue.

    What I am doing is slowly working through variations by estimating when traffic is low (lunchtime etc) when I am able to change the router config, and then try to see the results.

    Your idea of the remote desktop is a good one, but probably needs someone in attendance on that machine. TCPdump running on their router is probably also not going to be allowed. Privacy and all ..

    Oh well. No comments yet from other vlan users? vlan machines showing correctly in ARP /WOL and QOS-detailed lists?

  6. Toastman

    Toastman Super Moderator Staff Member Member

    Porter, I've appropriated a spare RT-N16 and set up an identical network here. My connections are now CORRECTLY classified on my pie chart. All of them terminate at the router's WAN ip address in the QOS-connections list though. So it's back to how it was on the business setup. Reboot and it's the same, no wait period. Maybe this happened after a reboot - I don't know. Never mind! But it's stable. QOS appears to work just fine on the vlans.
  7. Toastman

    Toastman Super Moderator Staff Member Member

    I wondered if it were possible to get MiniUPnPd working on the vlans. Looking at the config, it seemed to be a matter of setting the listening_ip to the IP addresses of the vlan ports in router/miniupnpd.conf file and recompiling it, but seems like the router builds a config file of its own at runtime. Anyone know how to add and to the list, so I can try it to see if any ports get forwarded? I tried adding to the router's /etc/miniupnpd.conf file, and restarting the service, but it's overwritten again when the service starts.

    EDIT - followed suggestion of Teddy Bear's from this thread and established that ports can be opened from the vlans to the outside world, though the config isn't surviving a reboot at the moment. *** I solved this by adding the vlan as a upnp "listening port" by loading a small configuration file from JFFS usb stick. The compile that does this is posted on my websites.

    So QOS and UPnP / NAT-PMP works correctly on all vlans.

    The last remaining issue is that of the QOS - details display - which isn't so important anyway. BUMP .... :biggrin:
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice