1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Using QOS - Tutorial and discussion

Discussion in 'Tomato Firmware' started by Toastman, Dec 24, 2008.

  1. Toastman

    Toastman Super Moderator Staff Member Member

    Hi Mishe

    Sorry, I don't understand the second question...

    The first - these scripts were designed to be pasted into the firewall script box. So when executed, the -I means that 1st (DROP) line is moved to the top of the iptables list. I = INSERT AT TOP. Next executed is the ACCEPT line, which is again moved to the top, BEFORE the drop line. So you are partly right in your assumption!

    A = APPEND (add to the bottom).
  2. michse

    michse Networkin' Nut Member

    ahh, so if I use -A it would be attached at the end? in which file can I see all rules on my router?

    ok, second question I wrote sometimes and I receive no answer. I try...

    wrt54gl with victek tomato 1.23
    in init script is:
    Code:
    sleep 5
    ip addr add 192.168.1.3/24 dev vlan1 brd +
    ifconfig vlan2 192.168.3.1 netmask 255.255.255.0
    ifconfig vlan2 up
    where I get a way to touch my modem and vlan2 gets an IP and Netmask

    In firewallscript is:
    Code:
    iptables -I INPUT -i vlan2 -j ACCEPT
    iptables -I FORWARD -i vlan2 -o vlan1 -m state --state NEW -j ACCEPT
    iptables -I FORWARD -i vlan2 -o ppp0 -m state --state NEW -j ACCEPT
    iptables -I FORWARD -i vlan2 -o br0 -j ACCEPT
    iptables -I POSTROUTING -t nat -o vlan1 -d 192.168.1.0/24 -j MASQUERADE
    I should insert some rules to deny wlan clients get my vlan2 net with my computers.


    nvram variables are left:

    Code:
    lan_dhcp=0
    lan_domain=
    lan_gateway=0.0.0.0
    lan_hwaddr=00:16:xx:xx:xx:xx
    lan_hwnames=
    lan_ifname=br0
    lan_ifnames=vlan0 eth1 eth2 eth3
    lan_ipaddr=10.0.0.254
    lan_lease=86400
    lan_netmask=255.255.255.0
    lan_proto=static
    and changed:

    Code:
    vlan0hwname=et0
    vlan0ports=5* 3
    vlan1hwname=et0
    vlan1ports=4 5
    vlan2hwname=et0
    vlan2ports=5* 0 1 2
    thats it I think. I use pppoe.

    Qos details shows:

    HTML:
    Proto	Source	     S Port	Destination	D Port	Class
    TCP	212.227.17.162	993	84.183.121.xxx	51947	Highest
    TCP	213.244.185.41	80	84.183.121.xxx	1528	High
    TCP	74.125.43.102	80	84.183.121.xxx	51944	High
    TCP	209.85.135.138	80	84.183.121.xxx	2086	High
    TCP	217.188.32.97	80	84.183.121.xxx	1549	High
    ICMP	193.99.144.85		84.183.121.xxx		Lowest
    TCP	192.168.3.1	80	192.168.3.100	51978	Unclassified
    where 84.183.121.xxx is my external IP address, *3.100 is my PC, *3.1 my router. Class is not working correct, maybe happens on this problem.

    So I hope you can turn on the lamp above me - and sorry for my english. you are welcome to correct me ;-)

    michse
  3. Toastman

    Toastman Super Moderator Staff Member Member

    Hi again

    Telnet or ssh to your router, obtain a list of iptables commands with "iptables -help" or abbreviate it to "iptables -h"

    iptables -L
    iptables -t nat -L

    The other questions are to do with vlan and routing. I'm not an expert at this sort of thing, so hopefully someone else will answer these questions!
  4. michse

    michse Networkin' Nut Member

    so that makes no sense after iptables -L?

    target prot opt source destination
    ACCEPT 0 -- anywhere anywhere
    DROP 0 -- anywhere p54B77808.dip.t-dialin.net

    first accept all, then drop something special. ok, maybe other thread...

    thank you
  5. fookxixi

    fookxixi Networkin' Nut Member

    Hi Toastman,
    I cannot find a good way to set a port rule for Skype, since the outgoing call requires any port above 1024. It always fall to the default class, which is used for P2P. How to differentiate it from P2P? Thanks.
  6. fookxixi

    fookxixi Networkin' Nut Member

    Hi Toastman,
    I cannot find a good way to set a port rule for Skype, since the outgoing call requires any port above 1024. It always fall to the default class, which is used for P2P. How to differentiate it from P2P? Thanks.
  7. karogyoker

    karogyoker Networkin' Nut Member

    Try Layer 7 skypetoskype (and skypeout)
  8. Toastman

    Toastman Super Moderator Staff Member Member

    Flashing routers over the web

    I had a few mails recently asking about flashing routers remotely.

    I was forced a while back to upgrade 24 AP's and 2 routers over the web, some 200kM from here. I did this half expecting to have to drive to the site to recover busted routers. But I made the discovery that actually flashing over the web was quite reliable. The secret is to WAIT and not panic if the router does not accept a flash in what you might think is a reasonable amount of time. It is quite normal for a remote flash to take up to 10-15 minutes and sometimes longer. If no flash after 15 minutes, I would wait an hour or two before I gave up. Once the connection is closed, then you have a big problem!

    TIP: You can check the remote router's GUI in another browser window to see if it still responds - if it does, then the router is not accepting the flash and it is safe to disconnect. If it doesn't respond, either a) it's accepting the flash b) it has rebooted and DDNS not yet updated, or c) it's dead Jim ....

    I have now flashed remote sites many, many hundreds of times with no failures.

    I have never bothered to do it, but before uploading anything onto a busy router it might be a good idea to prevent anyone using it, which would allow best access speed and also free up RAM. Changing it's LAN IP no. is one method.
  9. benny.shen

    benny.shen Addicted to LI Member

    cool thread, thanks a lot
  10. Toastman

    Toastman Super Moderator Staff Member Member

    #121, #122 Concerning Layer 7 skypetoskype and skypeout.

    I've found that a significant amount of skype traffic is not classified by using the L7 skype filters. Searching the web, I found several articles also saying the same thing. Unfortunately, there seems to be no easy way round this.

    EDIT - It's now 1 year later and the L7 filters appear to work better - always use the latest version of firmware, and you'll stand a better chance of success. The skypetoskype filter seems good, but the sykepout one allows a lot of P2P through into that class. I decided against using it.
  11. phuque99

    phuque99 Networkin' Nut Member

    I'm guessing that L7 library developers need to keep up with ever-changing protocol pattern matching changes.
  12. jochen

    jochen LI Guru Member

    Hi Toastman,
    thank you for that wonderful QoS explanation!
    I have just setup your settings from example #19. Seems ok so far, but I'm wondering why internet radio streams (mp3, http port 80) are not classified as class C (dst port 80, transferred 256KB+), but as normal www connections (class lowest, dst port 80, transferred 0-256KB)? :confused:

    Addition:
    after listening a mp3 stream at 128kbps for approx. 7 minutes, the class switched from lowest to class C.
    The traffic was: 128kbps/8*60sec*7min = 6720 kB

    My rule was to switch after 256kB. Why is the switch after 6720kB?
  13. Azuse

    Azuse Networkin' Nut Member

    Although I haven't touched this, I'll bet it simply uses a range of ports from which it picks them at random. Essentially it make qos impossible, but alot of newer programs e.g. msn list thousands of ports as it's way of handling nat and ensuring it functions on larger networks. Total pain in the ass. If you can't catch them with L7 you probably never will.

    In the end I set all my classes to max and used the mac limiter (raf) leaving it to keep data moving and class rule more for the visual graphs plus lag reduction i.e. they aren't used to control bandwidth, just priority. Probably impractical for some, and a nuisance for other, but it's the only solution I've found after a month of tinkering :mad:
  14. Toastman

    Toastman Super Moderator Staff Member Member

    Classifying SHOUTCAST AV

    Anyway, QOS itself is probably working correctly, remember the outgoing traffic is what is being counted for the >256KB rule. If the incoming traffic is UDP there are no ACKS, so there is very little outgoing traffic at all, perhaps just a small amount on what I will call the "control" channel. If TCP, then the ACKS are mostly what is being counted - so it takes a long time to cross over to class C.

    If the L7 filters worked, and the "incoming L7" box is checked in the Conntrack page, then one should be able to classify it.

    ADDITIONAL


    I am using Winamp for the Shoutcast streams, and a few other bits of odd software for icecast and others. Spider Player is a great little application for both Shoutcast and Icecast.

    The latest level 7 filters usually work for video but were not quite so good with MP3 streams. New version L7 filters will appear from time to time, so keep trying.

    So far most of these connections seem to be TCP, (but I do see odd UDP ports opening and these do seem to be related to Winamp as they disappear when the application is closed).

    The only reliable way to use QOS has been to prioritize ports for the complete range used by the MP3 and Video servers listed. Setting both TCP and UDP, no size filters. Without going through every possible server, there seems to be no information as to what range of ports are most common - but so far ports 7000-12100 seem to cover the video servers I've tested. All streams show in the correct classes. Video streams generally open one stream on a good solid link, but on poorer links will often open a good many connections - probably only the most recent one is actually the one in use and the others very quickly time out. This means that any limits set on numbers of connections and speed of opening may also impact the performance. Conntrack timeout settings need to be aggressive.

    One problem remains - P2P using any ports in that range will also be placed in this class. This is something that one may just have to put up with, so far it doesn't seem to be a major problem. The method is the one used in the examples on this site, which has been tested for about 18 months - P2P generally holds in check OK.

    EDIT: Lately, I have been finding the L7 filters have been improved. There is a new post below with example QOS, using L7 filter for Shoutcast. It seems to work OK and is not processor intensive. Give it a shot.
  15. jochen

    jochen LI Guru Member

    Ahhh, ok. I missed that. Maybe I should reduce this value?
  16. Toastman

    Toastman Super Moderator Staff Member Member

    It won't help - because it's just counting control traffic or whatever, the actual download is what we need to find and try to classify. Going lower will most probably just end up putting all WWW into class C.

    Try to see how it is arriving, what ports, protocol, into what class is it going (probably the default).

    EDIT: See later example QOS for use of L7 filter
    .
  17. jochen

    jochen LI Guru Member

    Today I installed Victeks Mod with your Mod (proper class labels). Really great work! Thanks a lot to all that made this nice piece of firmware.
    The Radio Stream I was listening is a normal HTTP Stream with Dst Port 80. I don't know how to distinguish this from normal www traffic. But I think this is ok. Radio has a constant limited bandwidth and it should be guaranteed that the stream becomes not disrupted.
    The last 6 months I used a Draytek Vigor 2710n. My family often complaint about our internet. I found that this Draytek is full of bugs. The DNS forwarder often returns no answer, making surfing the web a pain. The internet radio (Freecom Musicpal) often had short dropouts. So I replaced it this weekend with my old WRT54GL, installed your QoS rules, and now all are happy. :)
  18. CandyBoy

    CandyBoy Networkin' Nut Member

    Toastman, i want to ask you ,are these values in screenshot tcp limit and udp limit not too low? I want to do this for a few clients . I was readed all your posts about optimising these things , but i want to control all this from a gui.

    http://i50.tinypic.com/suu2av.jpg
  19. CandyBoy

    CandyBoy Networkin' Nut Member

    Toastman, i set Maximum Connections to 2048, isn't too low? Or should i set it to 4096?
  20. Toastman

    Toastman Super Moderator Staff Member Member

    QOS example for Tomato - compatible with v1.26+

    In recent tomato releases there is a restriction of 10 port entries per QOS rule. Earlier versions of QOS with many entries per rule will probably load and run fine, but after any edit they will be rejected by the GUI checker, and you won't be able to save the rule. This is an improvement, as it was previously possible to corrupt your configuration by entering too many ports in a rule.

    Here is the latest QOS setup used here for everyone, from home users right up to 400 room residential blocks. Some rules have been split into sections and a general tidy up has been made. It seems to run faster. Use it as a base for your own setup. If you copy all the settings exactly as given, this should work reasonably well for you, needing just a few changes for your own setup, or if you wish, disable and later delete things that are not appropriate.

    NB - Please note that the use of Class E for "P2P uploads" (seeds) did not work as well as I had hoped, which is why there is no rule for P2P Uploads. Use this class as a "crawl" class to slow down or dump unwanted stuff in.

    [​IMG]

    [​IMG]

    [​IMG]

    Most popular chat services are covered including QQ, as is most streaming audio and video, either in the appropriate class or by use of HTTP ports by the application. Shoutcast, Icecast, TV and MP3 streams are now classified correctly by an L7 FILTER (most of the time). Occasionally a P2P connection can be identified as Shoutcast, but it will usually not cause any trouble. I have added an L7 filter to cover Flash/Youtube videos.

    This setup runs on 1Mbps/16Mbps link. Since the values are given as percentages, just adjusting the maximum limits for your own link speeds should get things working without too many other changes needed. Remember you ***must*** set the maximum outgoing limit to, say, 85% of the minimum speed measured on the line. In fact, to begin with, set it lower to 70%. Once you know things are working you can up it later.

    With the greater bandwidth available, allowing a higher level of P2P has been found much safer. However, I want to point out that in my opinion, a WRT54GL router, being clocked at 200MHz and with a small amount of memory, is severely limited in what it can do. I have found that a 16Mbps line can push it to the limit - if you run it at or close to SUSTAINED full throughput. The CPU Load can and probably will exceed 0.5 at times, maybe even higher. At these levels, the router becomes sluggish. The web GUI responds more slowly. While the above QOS does a very good job even on a little GL, some users may find this sluggishness to be annoying (as I do).

    The ASUS RT-N16 is the answer to this, the faster router takes care of most of the slowdown problems.

    Most users may not experience this problem, remember I usually have around 80 to 100 users active on my networks, so it's hammered quite hard. I would imagine those with even greater bandwidth via cable will have more noticeable slowdown. As in all things, your mileage may vary!

    ADDIT: uTorrent has a new protocol from version 2.0, based upon UDP. One really *needs* harsh conntrack timeouts for this protocol. Without it, these UDP connections rise quickly and take up the router's resources. Clicking "drop idle" usually deletes around 90% of them. Setting UDP timeouts of 10 seconds for both unreplied and assured actually increased P2P throughput and freed up the router - in this instance reducing number of my own P2P connections from 2037 to 194 ! [This setting so far has not resulted in any complaints from anyone over the last few months]. In fact, dumping UDP packets altogether seems to improve downloads too. Strange.

    Refer to posts 165 and later below.

    VOIP users be careful with the UDP timeout settings. Use 10 and 25. Some users may need to increase the asured timeout figure towards 300 to avoid disconnection. Use the smallest number that is reliable for your own VOIP system.

    The versions of Tomato with labelled classes are available here:

    ftp://toastman.dyndns.org
    http://toastman.dyndns.org

    As a bonus I've added the wireless connection rate display from Fedor's USBmod. These compiles seem to work here but should be considered betas.

    There's now a version for the RT-N16 also - this will become the router of choice in future.

    The classes are in the exact same order Highest down to E, if you use normal version.

    i.e.

    Highest---Service
    High------Game/VOIP
    Medium--Media
    Low-------Remote
    Lowest---WWW
    A---------Mail
    B---------Messenger
    C---------Download
    D---------P2P/Bulk
    E---------Crawl

    Firewall scripts:

    You may find the addition of one or more of the following scripts, to the firewall section in ADMIN/SCRIPTS, will place some limits on the total number of connections allowed per client. Please note - I have found these scripts to work one day and not the next, depending on what version of Tomato is in use, what religion you belong to, or maybe on the prevailing wind. Your mileage may vary :biggrin:

    In theory (!):

    The FORWARD chain defines the limit on what is sent to the WAN (the internet). This therefore places a limit on the connections to the outside from each client on your network.

    The INPUT chain limits what comes in from the internet to each client. Without this limit, the router can still be overloaded by incoiming P2P etc. Often it is necessary to use both chains.

    #Limit TCP connections per user FORWARD=to WAN INPUT=from WAN
    iptables -I PREROUTING -p tcp --syn -m iprange --src-range 192.168.1.50-192.168.1.250 -m connlimit --connlimit-above 80 -j DROP
    iptables -I INPUT -p tcp --syn -m iprange --src-range 192.168.1.50-192.168.1.250 -m connlimit --connlimit-above 100 -j DROP

    #Limit all *other* connections per user including UDP
    iptables -I PREROUTING -m iprange --src-range 192.168.1.50-192.168.1.250 -p ! tcp -m connlimit --connlimit-above 20 -j DROP
    iptables -I INPUT -m iprange --src-range 192.168.1.50-192.168.1.250 -p ! tcp -m connlimit --connlimit-above 50 -j DROP

    #Limit outgoing SMTP simultaneous connections
    iptables -I PREROUTING -p tcp --dport 25 -m connlimit --connlimit-above 10 -j DROP

    If you test the above scripts with a limit of say 5 connections in the line, you will often see that it doesn't appear to be working, you will have many more connections than your limit, maybe 30-100, that you can't explain. Some of these may be old connections that have not yet timed out, and waiting for a while will fix it. Be aware that often these may be TEREDO or other connections associated with IPv6 (windows Vista, and 7) which is enabled by default. You should disable it on your PC by command line:

    netsh
    interface
    teredo
    set state disabled

    Associated post: http://www.linksysinfo.org/forums/showpost.php?p=359084&postcount=152

    Accessing modem via the router:

    Give your modem an IP in a different subnet to your router. Normally it's easy to use 192.168.0.1 which is probably the one in most common useage.

    Enter the following scripts into these sections of ADMIN/SCRIPTS page:

    init: ip addr add 192.168.0.13/24 dev $(nvram get wan_ifname) brd +
    firewall: iptables -I POSTROUTING -t nat -o $(nvram get wan_ifname) -d 192.168.0.0/24 -j MASQUERADE

    The first allocates an IP in a different subnet to the appropriate vlan interface for your router.
    The second sets a route for that subnet via that vlan interface to the modem.

    The scripts will discover the correct vlan for your router from NVRAM.

    Now you should be able to access your modem by typing its IP into the browser. Not all modems seem to respond to this, though, even if they work just fine when connected directly to a PC.
  21. jochen

    jochen LI Guru Member

    Hi Toastman,
    really good work this thread! Thank you very much.
    Do you have an updated version of your compile with your own class labels?
    The last I'm aware of was Victeks 1.23
  22. Toastman

    Toastman Super Moderator Staff Member Member

    No, I did not bother because the 1.23 version was so stable it hasn't been worth doing. I have been waiting for 1.27 etc. to become stable but still seems to have problems. Perhaps the fastest and most stable tomato so far has been RAF 1.25 v8515.2, but in my case it seemed pretty similar.

    When Vic releases a new version I'll probably do something. I have also been thinking about changing the class names a little, esp. P2P upload to become Crawl/Choke/Dump/Discard or something - but not decided yet. To be honest, now that Teddy Bear has released a (pretty damned stable!) beta version of Tomato that will run on the RT-N16 and uses the newer linux kernel, I will probably begin to replace the main routers, I think we will see a huge improvement in stability. I don't at the moment see much use in large blocks like these for "N" AP's - so we'll probably still use "g" wireless access for some time.
  23. Mahanakorn

    Mahanakorn Serious Server Member

    T-man, thanks for write this thread. I am still reading but wanted to say I used these last setting and it is working perfectly for me and my friends at my dorm.
  24. Toastman

    Toastman Super Moderator Staff Member Member

    Updated QOS example.

    I finally got higher speed 16Mbps ADSL connection here.

    I've updated the example QOS setup a few posts back to reflect these new ADSL lines (1Mbps/16Mbps), but reducing the max bandwidth figures for lower speeds should be OK. Also, some rules are now split into sections due to the restriction of 10 ports per rule in recent Tomato versions, which is a good thing to do as we now know where we stand on that point. Shoutcast TV and MP3 streams are now (mostly) correctly identified by an L7 filter.

    I note that the higher speed lines allow more P2P to be used without totally screwing up the other users. Try putting the limits high and see what happens to you! It is possible for me to almost max out the link with P2P and still have very fast browsing with this QOS setup. Your mileage may vary :tongue:
  25. karogyoker

    karogyoker Networkin' Nut Member

    I have set up my QoS and it worked fine.
    I had 10Mbit down/0.5Mbit up
    Now I have 5Mbit/0.5Mbit

    When me or somebody else downloaded something the ping times stayed OK.
    Now, with less bandwith, they are go higher.

    I solved this by setting max inbound speed to 4800kbit and all classes to 100% Limit, except the Highest class where (my games are), which is None.
    Now ping is alway stays low, even if others are downloading. Or myself.

    I hope this will help for somebody.

    In the case of upload, there was not any problem. The key is to set the max upload speed less then the real max upload speed by 70-80%.
  26. nyonya

    nyonya Networkin' Nut Member

    Maybe I'm missing something, but where in these setting do you specify in which cases P2P/Uploads should be used?
  27. QSxx

    QSxx Addicted to LI Member

    Yup, it seems to be missing from that screenshot...

    Also, could you make it a leeeeeeetle bit bigger... kinda hard to read it out. If there's some restriction on it I hereby BEG admin to up it just a feeeeeeeeeeew dozen px... it WILL help...

    If it's BW question, i'm volonteering my debian server... (it had 400 day uptime, then cpu upgrade, now 10 and counting)
  28. Toastman

    Toastman Super Moderator Staff Member Member

    I've updated the latest sample QOS post above http://www.linksysinfo.org/forums/showpost.php?p=357556&postcount=135 with bigger pics.

    You can use the P2P uploads directory for whatever you like. Originally I had tried to use it to separate P2P into two distinct categories. If you are on a relatively low speed ADSL line with 512k or less upload bandwidth, allowing a torrent client use that bandwidth to upload (seed) files will actually prevent you from downloading torrents quickly. In uTorrent, you can set the upload bandwidth to minimum (1) to get around this. However, in an apartment block where you have no control at all over users, it is useful to just be able to prevent their P2P uploads - or they will take over your network.

    I tried to separate uploads by setting source ports 1024-65535 - data over say 256k to go into the upload class. However, it causes too much hassle and did not do what I expected. It's not an individual counter, but adds everybody's use into the rule.

    I use this class now to dump things into which I want to slow down, and allow little or no outgoing bandwidth. At the moment, there's either no entry or I've used it to dump unwanted UDP and uTP traffic from uTorrent. Use it for whatever you like - remember it has the lowest priority of all the classes. In any future compile with class labels I will change the name to CRAWL class or something similar.

    All this is covered in earlier posts, so if you find something unclear, do read the whole thread. Somewhere in all this stuff, the answer you need is probably lurking - all you have to do is find it :grin:
  29. Toastman

    Toastman Super Moderator Staff Member Member

    ADSL Modem reliability

    Since moving up to higher speed 16Mbps ADSL line on the WRT54GL's, I have noticed a problem I have not seen before. This speed requires ADSL2+ connections. A lot of modems I've tried will not provide 16Mbps sustained throughput without crashing - locking up or rebooting. Even in bridge mode! This seems to apply especially to the crappy modems / wireless routers provided "free" as part of promotional packages from many ISP's.

    Next:

    When operating under sustained 16Mbps down and 1Mbps upload speeds, I found that my modems were sometimes dropping the WAN connection and then they do not reconnect! The ADSL connection LED either remains unlit or flashes, trying to reconnect but failing to sync.

    Rebooting the modems usually caused them to re-acquire an ADSL2+ connection. However, when Tomato (using PPPOE) then logged onto the ISP via the newly booted modem, although it picked up an IP and all other details from the ISP, there was no actual access to the web. Rebooting the router then returned things to normal.

    This means there are (or seem to be) two distinct problems, one with the modem, and the other may be firmware related. [ EDIT - yes, this was so - we've fixed the firmware, now the modems remain the big headache... ]

    I have searched all junkboxes belonging to myself and my friends, and found a total of 7 different ADSL router/modems capable of ADSL2+. I am shocked to find that around 50% of them are not actually capable of 16Mbps. Those are usually the "freebies" supplied by the ISP's with a new connection. Others run at full speed but occasionally drop the connection. The issue may be heat related, especially since most modems I have seem to have almost no cooling slots in the cases. I am investigating this now.

    I have determined that the cause of the modem disconnections is most definitely NOT heat related (supercooled with fans, placed in fridge - same problems).

    In the past, there have been many posts on the various forums about instability in Tomato and other firmwares causing WAN disconnection. I now suspect that many of these complaints are in fact due to the modem in use, and may be nothing to do with the firmware. Those who use their system relatively lightly may never have a problem.

    Just a tip - always suspect your ADSL modem if your WAN drops connection - the ADSL sync is lost, and the modem's ADSL SYNC light just flashes - forever...

    Another point - if you use your 16Mbps + line in earnest on a WRT54 and keep it close to maximum throughput, you will find the processor useage rising towards 1 and over. The router will become sluggish - it does not really have enough power for these sustained speeds along with the overhead of all the other processing it is required to do. The ASUS RT-N16 is far better.

    I had better explain further. Under normal sustained high use with 5Mbps line, under all circumstances the ping can remain low and the GUI responds quickly. As processor use rises to around 0.5 the whole router begins to slow down and response times get longer. Of course, the higher your connection speed, the more likely this is to occur. There are many posters who seem to overlook the fact that although the router might work with high processor loading - it's a lot slower in response and page loading is sluggish.

    Unless your QOS limits things a little, you may find this lag to be unacceptable - it annoys me a lot. Some people may not even notice because they are used to slow response times, so I just wanted to mention this in passing.
  30. hjf288

    hjf288 Addicted to LI Member

    DG834GT and DG834N can sustain these speeds, they just suffer from terrible firmware - Slow NAT Loopback, broken UPNP, dumb user interface..

    I'm running a Dg834N in bridge mode and using a WRT54GL and can max my 24Mbps ADSL2+ :)

    Tested downloading over 30GB constantly (Non stop)
  31. QSxx

    QSxx Addicted to LI Member

    Well i have no such problems ATM :( ... I mean i WISH i had... only thing we can get in this god forsaken place is t-com 4 or 10 mbit line down with 256 or 512 up respectively... that's slow for the price we pay for it .. but well... (4/256 in my case being shared among 6 computers)

    I would ask our dear Toastman another question: I assume what you gave us is an example of QoS for very large residential wifi setup. Are there any tweaks that can be introduced for cases of smaller networks. I assume that there's a lot of cases like me on LI.org forum.

    DISCLAIMER: By "assuming" author did not intend to make ass of anyone, let alone u or me :) :) :)
  32. Toastman

    Toastman Super Moderator Staff Member Member

    Qsxx no worries. Until a few weeks ago, we had only 5Mbps maximum, although that was quite good. Then suddenly a new promotion promising 16 Mbps actually came to fruition as a new cable was snaked down the concrete poles in the street to a local distribution box. So it is only now I am beginning to see any problems with higher speeds. It is absolutely unacceptable for ADSL2+ modem to lose sync and then be unable to pick up again.

    Regarding the QOS, these rules are not - per se, for a large number of users. It has to cover a large number of applications that those users might be running and which need to be controlled. I have to try to cover anything that a resident might want to use, because if he can't use it, he may move out and we lose money. At the same time, any application that can hog bandwidth has to be controlled or it will affect everybody, and then they all start shouting at me. For this reason, these rules should be an ideal base for people to start with. Almost certainly you can improve things though, everyone's needs are different.

    Of course, if you are a single, standalone user, you don't even need QOS at all.
  33. michse

    michse Networkin' Nut Member

    Hi,

    I have a 16/1 MBps ADSL+ line from t-com (Speedport200). You have to differentiate that also exists a 16/1MBps via vdsl variant which works direct with ip and without ATM.

    so connection loss is not only a thing of the modem. the sync can still be good and stable but router loses connection as well. It could be too the dslam. So t-com hotline discussed a while that my connection is pretty good because no sync problems. but pppoe drops with increased regularity so they had to repair the outdoor dslam! Tell someone from a hotline something they don't have in their computers...

    Resume, it could be the modem, but don't have to be.

    I got down and upspeed at fullspeed with wrt54gl tomato victek 1.23.
    Download at 1,5MB/s:
    HTML:
    Mem: 13768K used, 760K free, 0K shrd, 1836K buff, 4592K cached
    CPU:   0% usr  71% sys   0% nice  27% idle   0% io   0% irq   0% softirq
    Load average: 0.25 0.10 0.02
    and Router shows no delay on the gui. I have some bad made rules in QOS, maybe one do this :)
    What I don't understand, top shows 70% of usage, but the sum of all programs show a lot less. And the sum of ram usage is around 150% - works virtuell?

    michse
  34. QSxx

    QSxx Addicted to LI Member

    Well, me again...

    After whole day of intensive testing by all 6 (wow) clients, I have the following problem.

    While all video streaming services seem to have been "accelerated" (having their part of a bandwidth pie properly), YouTube streaming seems to be suffering.

    Symptoms are intermittent buffering (it starts, then stops, restarts... in regular intervals) mostly.

    I tried creating class for youtube and giving it a leeeetle bit more but I don't seem to be able to do it. All connections that i managed to identify go to port 80 (thus being identified as browsing). Also, i noticed that youtube buffering opens up dozen or more connections to same server. That normal? Anyone knows more?

    Thx...

    Also, when you have the time, would you make quick howto for renaming classes?
  35. Toastman

    Toastman Super Moderator Staff Member Member

    uTube used to bereally quite difficult to do much about. But things are a bit better now. There are supposedly two protocols used, normal HTTP and RTSP. RTSP can be covered but HTTP is not so easy. TBH I think that there are many YouTube videos which seem not to even come from YouTube servers, and I think several protocols may actually be in use. So - belt and braces time:

    We will use the FLASH and HTTPVIDEO L7 filters which seem to work most of the time. To test them I used this link to a quite high quality video (480p)

    This video is 10 minutes long, and just so I could see if the classifications worked, I placed them in HIGH (Games) class above WWW, where it streamed happily without a single hiccup. After confirming that it worked reliably on several other streams, I moved it back to the MEDIUM (Media) class.

    I am currently streaming a shoutcast TV from Germany, another HD shoutcast from USA, 8 local IPTV sessions, while downloading uTorrent at 7Mbps, and I have just opened a 2 way web TV session with my brother. There are also nine other people online, everything is still working, and I've been streaming the same shoutcast station from Germany all day now. So I guess it's Bingo!

    I will add these new rule to the last QOS example setup in the QOS thread http://www.linksysinfo.org/forums/showpost.php?p=357556&postcount=135 . If you try them, please let me know how well it works for your YouTube viewing.

    The fact that Youtube delivers their files over HTTP is precisely why the service (and online video in general) were able to become so popular. While streaming protocols have (and still do) fail over small, unstable or walled connections, a simple download always succeeds. No special cases, no proprietary protocols; just the same old HTTP packets any internet-related hardware and software is optimized for. But - we do need to identify and give it a priority if we run a busy router.

    I think that, in common with shoutcast streaming over HTTP, after a series of dropped packets the client opens a new connection to the server. Thus, after a while, there are many old connections waiting to time out. So you need to be very harsh when setting conntrack timeouts. FAST Conntrack timeouts are very important to getting the router's QOS working well. And I do mean FAST. Pare it right down to the bone.

    In fact, streaming video represents 36% of all HTTP traffic. And of that 36%, YouTube represents 20% – equating to 10% of all internet traffic. And there's very little anyone can do about it.

    TIP - A big improvement in uTube video (accelerators) can often be obtained by opening several streams for the same video and recombining them at the receiving end - for example, take a look at this:

    http://www.videoaccelerator.com/

    Useful information site on Video providers: http://en.wikipedia.org/wiki/Comparison_of_video_services

    Oh. The class names are hard coded in the firmware. You can't easily change them. I keep hoping that one day someone will add the choice, it would be very easy to do, but I don't know enough about programming to do it. ( *** EDIT - so I learned - and added this feature myself - look later in thread).

    EDIT 2012 - A new youtube filter has been designed by Porter, and is worth as try... look for latest version of Toastm,an Tomato.
  36. QSxx

    QSxx Addicted to LI Member

    You are my hero... Serously..

    Implementing that rule as we speak - i'll report back in a day with results :)

    EDIT1:

    Router upgraded to WRT54G-TM :) :) :)
    Speed upgraded to 10/512

    For the moment, it's not catching buffering connections properly... only control ones end up in Medium (Video) class. Since youtube keeps most of it's stuff on *.1e100.net servers - i'll try adding it's ranges to one of the rules...

    Google has whole forest of IP's assigned http://ws.arin.net/whois/?queryinput=N . GOOGLE
    Interesting range (at least for SE Europe) is this one 209.85.128.0 - 209.85.255.255

    Youtube can be found there too

    I'll try to catch some of it by placing connections to port 80 in Video class but above flash rule...

    P.S. Toastman: Since my clients (now i sound important) won't use chat services other than WLM and don't play online (xcept maybe when Star Trek Online finally hits the stores) - i removed corresponding rules to make QoS life just a LEEEEEETle bit easier :)

    Will post back results...
  37. Toastman

    Toastman Super Moderator Staff Member Member

    Howto: Limiting numbers of TCP and UDP connections

    A few of my local university friends just mailed me to asked me how to set up the limits for TCP and UDP connections. They can't get it right, and wondered why they can't seem to see the limits working.

    FIRST - A WARNING

    I have never been totally sure about these scripts. When setting up the router, the following procedures can be followed and all appears to work. After putting the router online, after a while it can be seen that the connection tracking table is still filling up rapidly. Unfortunately Tomato does not give us enough information to properly establish what is happening. Thus - in many instances, the scripts will not do what you intended, and you may find them not worth bothering with.

    It may be that the scripts are flawed, or that the position of them in the iptables list is incorrect. If anyone can asist with scripts that are guaranteed to work in every instance, please post them! NB - Robson's script generator has also given me the same doubts ...

    EDIT - make sure you read the footnote about the router instability when using these rules, and the FIX!!

    ***

    OK, let's see if we can figure out how to set these rules up and how to verify what is going on.

    But first, why do we want to limit the number of connections anyway? Essentially it is to prevent large numbers of connections from filling up the router's memory and causing it to crash or reboot. This is most often caused by P2P applications. It's quite necessary on the older generation of routers with 16MB of memory, but I am finding the ASUS RT-N16, a faster router with more memory, has so far never rebooted or crashed under stress. Nevertheless it can still benefit from some limits.

    So, first, we will look at how the TCP connections can be limited with an iptables rule, and how to decide what limit to the number of connections we should set.

    The first rule works in the PREROUTING chain, and limits the number of connections to the WAN (internet) from each computer in the IP range stated.

    iptables -I PREROUTING -p tcp --syn -m iprange --src-range 192.168.1.50-192.168.1.250 -m connlimit --connlimit-above 5 -j DROP

    If you start off with this rule and no others, set the limit to 5, and watch how the router performs. You'll find it slow to open web pages. The more junk on the page, the slower it seems. A normal web page these days actually uses a large number of connections to open several parts of the page simultaneously.

    Increase the limit slowly from 5 to the point where it seems OK, you can reboot the router each time to be sure these rules are really in use. (You can see the rules by starting a telnet session and issuing "iptables --list" - but they may be hard to spot and understand - so don't fret about it).

    When the web pages are responding quickly, stop increasing the limit.

    OK, so now put the limit back to 5. Now also add this second rule, which limits the number of connections FROM the WAN (internet) to each computer in the IP range stated.

    iptables -I INPUT -p tcp --syn -m iprange --src-range 192.168.1.50-192.168.1.250 -m connlimit --connlimit-above 5 -j DROP

    Once again, we set a small limit of 5 for the incoming TCP. Again see what effect this has on web pages and increase it to see the effect it has. If you have both rules set to 5, and open some web pages, you should see the limit is now taking effect at 10 total.

    The total connection count in the rules below is 250 per client for both TCP and UDP (and any other stuff which might creep in). You may decide to use smaller limits for safety, and accept slower opening of web pages. You may also decide not to place a limit on any incoming connections at all. It's up to you. It's a trade-off between stability and speed, and only you can decide what is best for your own setup.

    When you have finished, and quite likely entered quite high figures such as my examples, it will be extremely hard to see what is going on. But nevertheless, by now you should feel comfortable in knowing that iptables is actually doing what you asked even if you cannot see it easily.

    You can adjust UDP limits similarly. It's more difficult because you may not have an application that uses a lot of UDP. Until you have a problem, just leave it alone.

    ]If you test the above scripts with a limit of say 5 connections in the line, you will often see that it doesn't appear to be working, you will have many more connections than your limit, maybe 30-100, that you can't explain. Some of these may be old connections that have not yet timed out, and waiting for a while will fix it. Be aware that often these may be TEREDO or other connections associated with IPv6 (windows Vista, and 7) which is enabled by default. You should disable it on your PC by command line:

    netsh
    interface
    teredo
    set state disabled

    --------------

    FIREWALL WARNING

    Traditionally the firewalls in Linux operate on the FORWARD and INPUT chains. At the bottom of the page I have added a new note - a "MUST READ" - you will see that this raises the question about firewalls being of much use in the event of a connection storm or DOS attack. On this and other forums you will see many well intentioned firewall scripts written by well meaning individuals who imagine hat their script is going to protect the routers from all kinds of malicious attacks. They give the rules fancy names to make them seem important. Now, Tomato has a built-in set of rules that works for most things already. Adding your rules will often screw it up. Next, the possibility of most of these problema being experienced on a domestic router is almost nil. Who is gonna waste their time on you when there are very obvious targets which are more fruitful? Lastly, the sad truth is that the vast majority of those scripts just don't work. But they can by themselves screw up normal operation of your router. So unless you know what you are doing, don't screw around. I would particularly advise "newbies" to desist from trying every piece of crap they find on a forum somewhere.

    Also, any extra rules operating in the tomato forward chain are likely to be useless. See the note at the bottom of this page to see why.

    Chains

    The FORWARD chain defines the limit on what is sent to the WAN (the internet). This therefore places a limit on the connections to the outside from each client on your network.

    The INPUT chain limits what comes in from the internet to each client. Without this limit, the router can still be overloaded by incoming P2P etc. Often it is necessary to use both chains.

    So traditionally, the following rules were used and can be found in the forums:

    #Limit TCP connections per user FORWARD=to WAN INPUT=from WAN
    iptables -I FORWARD -p tcp --syn -m iprange --src-range 192.168.1.50-192.168.1.250 -m connlimit --connlimit-above 80 -j DROP
    iptables -I INPUT -p tcp --syn -m iprange --src-range 192.168.1.50-192.168.1.250 -m connlimit --connlimit-above 100 -j DROP

    #Limit all *other* connections per user including UDP
    iptables -I FORWARD -m iprange --src-range 192.168.1.50-192.168.1.250 -p ! tcp -m connlimit --connlimit-above 20 -j DROP
    iptables -I INPUT -m iprange --src-range 192.168.1.50-192.168.1.250 -p ! tcp -m connlimit --connlimit-above 50 -j DROP

    #Limit outgoing SMTP simultaneous connections
    iptables -I FORWARD -p tcp --dport 25 -m connlimit --connlimit-above 10 -j DROP

    The next script is to prevent a machine with a virus from opening thousands of connections too quickly and taking up our bandwidth.

    #Limit UDP packet opens from all users - UDP to Router
    iptables -I INPUT -p udp -m limit --limit 10/s --limit-burst 20 -j ACCEPT

    #Limit UDP packet opens from all users - UDP out to WAN
    iptables -I FORWARD -p udp -m limit --limit 10/s --limit-burst 20 -j ACCEPT

    ***

    There may be times when you want to limit the speed at which connections can be opened, this could happen if somebody tried a DOS attack, for example. These scripts *may* be useful - choose the one that seems to work for you! Be careful that you allow enough DNS connections for clients to be able to use DNS services, or everything will suddenly turn into a nightmare! These include uTorrent v2.0 and VOIP users.

    #Limit UDP opening speed from WAN to clients
    iptables -I INPUT -p udp -m limit --limit 10/s --limit-burst 20 -j ACCEPT

    #Limit UDP opening speed from all users to WAN
    iptables -I FORWARD -p udp -m limit --limit 10/s --limit-burst 20 -j ACCEPT


    ADDITIONAL OBSERVATIONS DECEMBER 2010 - PLEASE READ THIS CAREFULLY

    While trying to get to the bottom of router reboots caused by connection storms, I have been fortunate to catch a nice machine infected with a mass SMTP mailer worm. This crashed the router almost immediately, but I was lucky enough to see some connections show up in the "details" page before the router crashed. After a reboot it would immediately crash again until the user turned off his machine.

    I discovered that although the above scripts work when tested on normal traffic, in the event of a REAL connection storm or DOS flood, they simply can't process the incoming stream fast enough and the router dies. It shouldn't do this. And this was on an RT-N16 which has more processing power than the older routers.

    THE FIX

    Following suggestions from phuque99 in the QOS thread, the existing limit rules were switched to the prerouting chain and run for about a week. The rebooting issue was pretty much gone. Returning the rules to the forward chain made the reboots return again. So it looks pretty conclusive - the connection limit rules in the normal firewall chains INPUT and FORWARD seem to invoke some mechanism which rebooted the router almost instantly in the event of a "real" DOS attack or connection storm. We don't know why exactly but we do know now what to avoid.

    The new rules in the firewall script box were:

    iptables -t nat -I PREROUTING -p tcp --syn -m iprange --src-range 192.168.1.50-192.168.1.250 -m connlimit --connlimit-above 150 -j DROP

    iptables -t nat -I PREROUTING -p ! tcp -m iprange --src-range 192.168.1.50-192.168.1.250 -m connlimit --connlimit-above 100 -j DROP

    iptables -t nat -I PREROUTING -p tcp --dport 25 -m connlimit --connlimit-above 5 -j DROP

    .
  38. onehomelist

    onehomelist Serious Server Member

    Code:
    #Limit TCP connections per user
    iptables -I PREROUTING -p tcp --syn -m iprange --src-range 192.168.1.50-192.168.1.250 -m connlimit --connlimit-above 80 -j DROP
    iptables -I INPUT -p tcp --syn -m iprange --src-range 192.168.1.50-192.168.1.250 -m connlimit --connlimit-above 100 -j DROP
     
    #Limit all *other* connections per user
    iptables -I PREROUTING -m iprange --src-range 192.168.1.50-192.168.1.250 -p ! tcp -m connlimit --connlimit-above 20 -j DROP
    iptables -I INPUT -m iprange --src-range 192.168.1.50-192.168.1.250 -p ! tcp -m connlimit --connlimit-above 50 -j DROP
    I am using 255.255.0.0 subnet. Is there any way I can modify the above quoted code to make it applicable to all the users instead of using the iprange?
  39. Toastman

    Toastman Super Moderator Staff Member Member

    Well, you just have to specify what the rule applies to, so it doesn't really matter how you do it, range or mask.

    Change the subnet and mask to suit your setup. If you trawl through the netfilter stuff you'll find more ideas & inspiration. Here's a few. Just be sure that the netfilter in tomato supports the commands you use.

    iptables -I PREROUTING -p tcp --tcp-flags SYN,RST,ACK SYN -s 192.168.1.0/24 -m connlimit --connlimit-above 100 -j DROP

    iptables -I PREROUTING - s 192.168.1.0/24 ! tcp -m connlimit --connlimit-above 100 -j DROP
  40. bmx888

    bmx888 Networkin' Nut Member

    BT hog on my router

    Toastman, I am having problem with my QoS.
    I have Asus WL-520GU behind motorola SBG900.
    My router is updated with Tomato Firmware v1.27.8744 ND USB Ext.
    The 1.5TG USB drive works great.
    My isp is comcast, avg 10Mbps/1.5Mbps.

    When I start bitspirit 3.6.0 with dht disabled and upload speed at 30kB/s.
    The real download speed is 40KB/s D 20KB/s U

    but then the router get really slow. a http request to google will time out.
    I set qos as you described and disabled acks.

    But still no luck. I used to have download to 1MB/s upload 30KB/s with asus own firmware.
    Can you check what is wrong?
    Qos1 QOS2 is the setting. QOS3 is when bt is closed. QOS4 is when bt is opened and download at 40KB/s.


    Here is all photo is detail
    http://s275.photobucket.com/albums/jj281/baomx888/Router/

    Thanks a lot!!

    Attached Files:

  41. Toastman

    Toastman Super Moderator Staff Member Member

    My good friend, you have not set QOS as I described, not even vaguely similar. So I am unable to comment.
  42. onehomelist

    onehomelist Serious Server Member

    Few days ago I had posted the following on a thread in this forum

    I was able to track the problem. It was the following firewall script

    Code:
    #Limit UDP opens from all users - UDP to Router
    iptables -I INPUT -p udp -j DROP
    iptables -I INPUT -p udp -m limit --limit 10/s --limit-burst 20 -j ACCEPT
     
    #Limit UDP opens from all users - UDP out to WAN
    iptables -I PREROUTING -p udp -j DROP
    iptables -I PREROUTING -p udp -m limit --limit 10/s --limit-burst 20 -j ACCEPT
    I came to know about it when I used the above script on Asus RT-N16. I could not open any web page, eventhough I was the only user connected to router [might be because of the newer kernel]. Only when I removed the script I was able to browse normally. I think pc's open lot of UDP connections which are not related to what users are doing. So when users start doing something, becuase pc's have used up all available UDP connections for internal activity, their browsing speed gets affected.

    I even wrote that the issue is too severe on fedora. It looks like it needs nore UDP connections than windows xp or vista.
  43. Toastman

    Toastman Super Moderator Staff Member Member

    Strange. If you can't open any web pages, which use TCP connections, then there's something very wrong. I just tried the scripts (individually) on my RT-N16 and I am able to connect with any website, no effect. Try removing the "DROP" lines - they are already in the router's chains as the main policy, and aren't necessary - this may indeed be the problem.

    My guess is that your machines are being prevented from accessing the DNS server. So each item in the web page that needs another DNS lookup will again delay the page. That being the case, something on your client may be opening a large number of UDP connections, perhaps without your knowledge. The scripts would then do what they are designed to do, and restrict the numbers. Firstly, find which of the two scripts is responsible. Then see if you can check what is happening.

    Perhaps you don't really need the scripts, and can just dispense with them. They do seem to be somewhat hit and miss - agressive connection timeouts are really all that is necessary in most instances.
  44. onehomelist

    onehomelist Serious Server Member

    I use interrupt DNS port option, so my users are forced to use my routers ip as the DNS server ip. Most of my users are heavy bittorrent users, and they use all sorts of file sharing clients. So, as you said, all users open large number of UDP connections.

    My aim is to provide stable and consistent access to legit users, those who browse websites. I use QOS, with bulk ports having lowest priority, and IPP2P filter. But still, when there are lots of users, the speed comes down considerably.

    I will try and find out which of the script is responsible, and will post the results.
  45. Toastman

    Toastman Super Moderator Staff Member Member

    Firstly, IPP2P filters don't work well. They will only trap a percentage of your P2P. The L7 filters are a little better but still let a considerable amount of connections through into other classes. If you have a lot of P2P, that leakage is enough to completely wreck the QOS. This is why most home routers with the "game booster" and "P2P control" buttons, which usually use L7 filters, don't work very well - but it's great marketing.

    If you are not doing so already, set the default class to E and let all P2P bypass your other rules and drop into that class. That is the only way to trap P2P. Some of it will still bleed through into other classes, but is usually not significant.

    Secondly, these huge numbers of UDP connections don't usually succeed in generating any downloads, but do take up a lot of bandwidth. Many users have DHT and DNA running without any knowledge of what they are really doing. This is why we usually have problems on WRT routers. However, it isn't so much of a problem on the RT - so that's an advantage. DNA in particular can hog all of your bandwidth, and has no advantage at all to the user.

    EDIT: This has all changed with the issue of uTorrent v2.0. This version uses a new protocol which is carried over UDP. So from now on, expect more P2P UDP traffic than TCP. As far as I can see so far, however, no change in QOS setup is necessary. See posts below.

    When you say that the speed comes down considerably when you have lots if users online, do you mean normal web browsing? That should not happen if the QOS is working effectively. No matter how many users come online here, web browsing is still snappy.

    You need to allow more UDP for machines to access DNS so you can't use the last two limit scripts. You can try to use the TCP and UDP limit scripts for total numbers of connections per client. That limits the damage to one client, and that becomes his problem to deal with. You may find they don't work well for you.

    Keep the numbers of UDP etc. ports down by expiring all unused connections very quickly with the Conntrack timers. For the 2 UDP settings I have found 10 is OK. By doing this, even with 50-60 users online I rarely see more than 1000 or so connections open. (VOIP users may have trouble with settings of 10).


    Good luck!
  46. bmx888

    bmx888 Networkin' Nut Member

    I just follow the instruction on the first page, set to different class.
    http://www.linksysinfo.org/forums/attachment.php?attachmentid=921&d=1241725214
    http://www.linksysinfo.org/forums/attachment.php?attachmentid=922&d=1241725214

    I also adjust some setting according to my isp. Now fully download at 300-400KBs without any problem and my web browsing and other application are super fast too.
    Thanks
  47. Toastman

    Toastman Super Moderator Staff Member Member

    MTU and Serialization delay on low speed phone / ADSL links

    Serialization delay is the amount of time it takes a router to send a packet onto a WAN link. The slower the link, the longer it takes to send a packet. It can be seen that a very slow link such as the old 56kbps (nominal) telephone modem takes a few hundred milliseconds even to send one fullsize packet! From this it's obvious that it is very hard to send reliable VOIP or games data over such a slow link - unless perhaps the MTU is reduced in size. That would make it somewhat faster.

    Serialization delay with the default Ethernet maximum transmission unit (MTU) of 1518 bytes only becomes a big issue if the WAN link is around 768kbps or slower. Certainly our older telephone modems were very slow, but not many people realize that the MTU can still be a problem on the lower-speed ADSL links many people across the globe are still using. 128kbps upload speed is still commonplace in many countries! Low link speeds and large MTU sizes lead to increased levels of jitter, which in turn can lead to packet discard and degradation in voice quality. Therefore, people with such links might find it useful to change their MTU settings to something lower.

    The chart below shows the variation of serialization delay (time taken to send a packet) with MTU size and link speed - jitter would generally occur as some multiple of this serialization delay. The chart relates to a 1Mbps connection.

    [​IMG]

    The following shows the serialization delay of an interface when transmitting full 1500 byte MTU size frames (Ethernet default):

    768kbps = 15ms
    512kbps = 23ms
    256kbps = 46ms
    128kbps = 93ms
    64kbps = 187ms
    56kbps = 214ms

    Think about this post from a user a few years ago.

    And of course, if a couple of packets were already in the queue waiting to be sent, we would have around 300mS delay. See how this compares with my 1Mbps/16Mbps link, where I have reliable ping response from my ISP's gateway of 17mS.

    A well set up QOS can maintain this ping time to between 17mS - 25 mS without too much jitter.

    There is a currently a guy calling attention to what he calls "buffer bloat" with all sorts of wizard schemes which he reckons will suddenly improve your connection. Now our dear friend Mr. Gibson is turning this into a big issue and scaring the crap out of everyone. Now, don't you think that thousands of developers have already done this a million time already? Some buffers are made large for a reason. Go read about it. While you may care to experiment, most of it sounds good but is actually mumbo jumbo. Leave your windows settings, router txqueue and other settings alone. All this has been tried on Tomato and it made no significant difference at all.

    That is the whole point of QOS. It places high priority traffic into a small queue!
  48. kardzzz

    kardzzz Networkin' Nut Member

    Toastman, I was hoping to see a labeled Victek v1.25.8515 .2RAF ND (that thing is a masterpiece!). most stable build i've ever used.
    Nevermind,,, maybe next time.
  49. Azuse

    Azuse Networkin' Nut Member

    I'm curious, not that utorrent 2.0 is finally out of beta, what effect uTP has had on your networks.

    My qos rules allowed 80% up and 100% down coupled with the mac limiter (RAF mod) and always meant a lag free lan. ok, so it's alot small than yours, but it does 13 down and uTP is choking it at 130k despite torrents having zero impact on the lan at 13meg. time to tweak our rules? :S
  50. Toastman

    Toastman Super Moderator Staff Member Member

    Kardzzz - there's one now if you look on my servers.

    Azuse - I installed uTorrent with uTP, set up with around 40 mixed file downloads, and spent some time with it on (a) a dedicated ADSL2+ 16Mbps line and (b) a similar line presently shared between 80 users, with what looks like about 30 users online. About 5 of these seem to be using P2P.

    On the exclusive line I got 8Mbps download within minutes. Most of them are using TCP connections, not uTP. Quickly changing the same uTorrent setup to the shared router and re-starting it, only 1.5Mbps - and it has not picked up over half an hour. It certainly wouldn't seem to be aggressive.

    Previously, uTorrent didn't use many UDP packets, with v2.0 there are many, and more than half the peers are shown as uTP. It has made no difference to the QOS operation - as far as I can see at the moment, as it is all classified under class D (my P2P/Bulk class) and appears to be subject to the normal limits of that class. If I limit outgoing P2P class D to 35kbps, it throttles all the outgoing uTorrent traffic and with it, the downloads. If I instead limit incoming bandwidth for class D - it limits all uTorrent downloads as before.

    So, doesn't seem to have made any difference to the network, as far as I can see at this moment. The ratio of UDP to TCP will go up, I suppose.

    EDIT 2: Gone back to the dedicated line to experiment. The TCP and UDP limit scripts do not appear to be working. Hmmm.

    I just placed Dst UDP ports 1024-65535 into Class E (Crawl) to separate it from other P2P and gave that class the same bandwidth as P2P class D. The ratio of UDP to TCP connections 5:1 and outgoing traffic 40:1 - indicating most of *MY* P2P traffic is now UDP using uTorrent 2.0. One big problem though - I dfon't actually have any worthwhile downloads occurring, yet a lot of bandwidth is being used for [what???].

    My rules at least, probably don't need to be changed at the moment.

    One thing that is very clear is that using the new versions with uTP results in much slower torrent downloads and more aggravation.
  51. Azuse

    Azuse Networkin' Nut Member

    I'll just put my experience down to a client bug then :) It is nice having it throttle itself however, and I'm sure every isp is breathing a sigh of tentative relief :biggrin:

    If, over time, it's as effective as they claim we'll probably see more udp floating around.
  52. Toastman

    Toastman Super Moderator Staff Member Member

    I am totally not sure if it's throttling itself yet. I am going to do some further investigation to see what happens in the event of QOS being turned off :biggrin: . I am also very suspicious that under the surface, this protocol may be being used to support DNA, which means using our bandwidth for commercial purposes. I'm like that ... :sheesh:

    EDIT: One really *needs* harsh conntrack timeouts for this protocol. Without it, these UDP connections rise quickly and take up the router's resources. Clicking "drop idle" deletes around 90% of them. Setting UDP timeouts of 10 seconds for both unreplied and assured actually increased P2P throughput and freed up the router - in this instance reducing number of my own P2P connections from 2037 to 194. This setting so far has not resulted in any complaints from anyone over the last few months. However, the end result of using v2.0 is thousands of connections and much slower downloads. It's a really CRAP thing to do on a low bandwidth consumer ADSL line!

    Be careful with VOIP if you experiment with conntrack timeout settings. UDP timeout settings of 10 and assured of up 300 seem to work for most VOIP providers. Use the smallest assured value you can get away with.
  53. Toastman

    Toastman Super Moderator Staff Member Member

    After 2 days of using uTorrent 2.0, it seems to me that the speed of acquiring downloaded files is rather worse than with previous versions. This is on a dedicated 16Mbps line with 50 quite well seeded files. Instead of maintaining an average of around 11Mbps it is hovering around 1.5-2Mbps! It is a resource hog, but doesn't produce downloads. I am not very impressed :frown
  54. Azuse

    Azuse Networkin' Nut Member

    Tbh it's almost certainly a bug. I've reinstalled both 2.0 and the beta several times. Initially it was throttled to 10%, with or without uTP, but it's working fine now. Well it seems to drop speed momentarily but constantly. Both those and downloads getting stuck seem to be issues a lot of people are having. I continually get the disk overloaded warning myself, again software bug, but it throttles too (quite badly) :(. When it does work however it seems to flow better, at least the peaks and troughs are gone at my end. I haven't been able to make it throttle itself though, which really is a pity.

    That said I've dropped my udp from 15/30 to 10/10 and have an even split between tcp and udp with DHT*. Of those the tcp is mostly established connections so if everyone was using uTP then most of those connections wouldn't need to be held in the router for nearly as long but the cpu is peaking at .55 now, as apposed to .7 before.

    Now all that's needed is a. bug fixes and b. everyone to adopt it.

    *For load testing only :D

    Edit: The 2.0.1 beta fixes the tup/uTP simultaneous traffic and uTP packet size. With ~500, only 30 of which were tcp, the router was cpu was hovering just under 0.4. Once new peers with older clients connected the tcps rose to ~200 the cpu hit 0.6/7. I can also stream flash videos to the pc running the torrents without lag. A good improvement from my end at least, if the trackers & other users have similar performance increases hopefully we'll see things switching to uTP quite quickly.
  55. Toastman

    Toastman Super Moderator Staff Member Member

    I'm testing uTorrent v2.01 here build 18284 - graphs are nice...

    EDIT: I have gone back to using an old version of uTorrent.

    v2.0/2.01 are too bandwidth intensive, and over a 3 day period I have only downloaded a quarter as much as usual, although it's certainly been very busy doing something. A very big thumbs down from me, I'm afraid :thumbdown:
  56. Azuse

    Azuse Networkin' Nut Member

    Yes it's quite buggy and very hit-and-miss sadly. Seems to overload the disk and choke itself every few min. uDP also has problems, although scaling the packet size depending on the swarm is handy, by preferring clients running uDP over everything else i.e. utorrent > the rest, the whole swarm slows. In time things should improve.

    Although I wouldn't know where to start, tomato does seem to be doing something odd. Most of the time I start downloading the router (qos is correctly setup) chokes them at ~150k (line is 1.3MB). Stopping then restarting qos fixes it, but the odd thing is it's chocking the pc at under 200k while the router cpu is 0.01 instead of 0.5. Shut down uTorrent and 5 seconds later it's back to normal, really strange :S
  57. tbjerret

    tbjerret LI Guru Member

    Build 18408 seems much faster.
  58. Toastman

    Toastman Super Moderator Staff Member Member

    Yes, it does seem better!

    For anyone who's interested, I have made a compile for the ASUS RT-N16 etc. labelled classes, and some of Victek's RAF features from his older code. At the moment, he's very busy, so that's the best we can do until he is able to make his next release.

    I also made a version of Victek's RAF 1.25.8515.2 with labelled classes and as a bonus, I've backported wireless connection rate information on the status and devices pages. Since this version was the fastest around, this should be a good choice as long as anything I added didn't break it!

    http://firmware.mooo.com/Toastman Builds/MIPS32R2 Kernel 2.6 (RT-N16 etc) Builds/

    The ASUS RT-N16 is proving to be very stable. The marketing hype says it can handle many thousands of connections while providing sustained throughput at high speeds. http://www.linksysinfo.org/forums/showpost.php?p=359894&postcount=731

    I would, however, suggest that you don't set the numbers of connections too high, 4096 is fine. Make sure your Conntrack timeouts are fast, and you will see faster throughput. It is very rare for more than a few thousand connections to be seen, even with several dozen users online.

    Patches to upgrade code with Victek's RAF features are located on the git repository at http://repo.or.cz/w/tomato.git/shortlog/refs/heads/tomato-RAF
  59. alien3456

    alien3456 Networkin' Nut Member

    I need to get an RT-N16. The newer utorrent on just one machine using my WRT45GL will bog down everything; the only remedy is to severely limit the number of connections. When everyone is just browsing or playing games, everything is golden. Even when there's a FTP/HTTP download, QoS handles it fine. But start up the torrents and everything goes slow-mo.

    I don't like taking an axe to particular connections, as everyone deserves a fair cut of the bandwidth (splitting the bill). Time to upgrade my little packet parsing machine!
  60. Lost_Animal

    Lost_Animal Serious Server Member

    Thanks Toastman,
    I will try to see how it goes.
    I believe ASUS RT-N16 can handle much more than 40000 Connections.:biggrin:
  61. Toastman

    Toastman Super Moderator Staff Member Member

    Asus' advertising says up to 300,000 and the GUI has now been adjusted in Fedor's version to allow the setting of high numbers. That may be just a prediction by Asus based on memory constraints. But - when I did my experiment, the router actually failed due to lack of memory after around 75,000 connections. You can find that experiment on this forum somewhere if you look for it.

    These were, however, not active connections, probably only 1000 or so were actually active, the rest were waiting for timeout. It would be highly unlikely that anyone would ever reach this figure with sensible conntrack timeout values. To actually allow this number of connections would be incredibly stupid. Or - to put it another way, it's Total Marketing Bull****. If this number of connections were in fact active - then the router would have died long before.

    Bearing in mind that although this router runs at 480MHz, that is not a "quantum leap" in performance, it is just double that of an overclocked WRT54GL. As a maximum limit, I recommended 4096 as a more sensible limit than 300,000 (which is wildly optimistic and ridiculous) but conntrack timeout settings should actually be set with low values, so that even this limit will never be reached. I have well over a hundred users and I always set 4096 or less. Allowing the list to be populated by old connections doesn't help anything at all, but does take up resources. As a guide, since I got this thing set up, I am seeing around 2500 absolute maximum under some really unusual load conditions, but usually I get only 700-1000 average with around 170 busy users. Yes, it's smaller than you'd think or expect from your previous experiences. That's the whole point!

    You will see faster response times as you free the router from unnecessarily tracking closed connections.

    If people are going to run applications on the router, such as in your optware project, then this is doubly important. However, I would strongly recommend that you remember a router is supposed to route, and if you add things like torrent clients to it, it will no longer be so effective in routing.
  62. weiyu99

    weiyu99 Serious Server Member

  63. Azuse

    Azuse Networkin' Nut Member

    The links are all in his sig, you'd need to be more specific though, 99% of the qos post on this forums are his :0

    Also disregard my negative 2.0 remarks. The eating the hard drive problem is only present on vista, windows 7 is silky smooth. Secondly Tomato can easily handle my 550-600 connections at the full 1.3/4MB, it seems my speed drops were (are) isp related. Although they don't throttle and there's no congestion it's more likely a physical fault which isn't surprising. My council has a habit of "accidental" cutting the wrong cables when doing, well, anything.

    P.S. When things are working 2.0.1 is keeping my line maxed out where 1.8.5 would have been going up & down, but the 54gl is more than up to it (@ 250mhz).
  64. weiyu99

    weiyu99 Serious Server Member

    Hi Azuse,

    Thanks for the reply. Here's what I see in Toastman's post:

    QOS example setups
    http://www.linksysinfo.org/forums/sh...&postcount=135 (LATEST AND BEST SO FAR)
    http://www.linksysinfo.org/forums/sh...0&postcount=76
    http://www.linksysinfo.org/forums/sh...9&postcount=85
    http://www.linksysinfo.org/forums/sh...1&postcount=19

    and when I go to the "Latest and Best so far" QoS example, I can find the actual example in that post, I knew it was there because I read it few weeks back (before his 2/27 update) but now the actual example setup is gone...

    Hi Toastman,

    Can you check your "LATEST AND BEST SO FAR" QoS example in http://www.linksysinfo.org/forums/sh...&postcount=135? I think you delete the actual example in the post.

    Thanks,
    Dennis
  65. Toastman

    Toastman Super Moderator Staff Member Member

    weiyu99 - I just clicked on the link in your post 178 and it went to the page OK - try it again. If it doesn't work just page back until you find it. Checked the other links you just posted and none of them worked, but when I went to the "common topics" page the links worked for me. So I have no idea what is going on :)

    Azuse
    I see some problems on all my networks in the last week or so - high outgoing bandwidth, pegged on my limit of 70%, but very little downloads. Of course, I have no way to tell if this is due to users swapping over to 2.0 - but it does seem likely.
  66. Azuse

    Azuse Networkin' Nut Member

    Fair enough but It really was a faulty line, snr was jumping up and down :)

    Anyways, its sorted and my speed problem seem to have come from this pc having TC tag 10 in the mac limiter, switched it to 16 and no slowdown. So it was tomato in the end :frown:

    So on the topic of qos, is there any easy way to classify teredo traffic? I'm getting a flood of IPV6 traffic with the utorrent beta. Someone finally using it.
  67. amitava82

    amitava82 Serious Server Member

  68. Toastman

    Toastman Super Moderator Staff Member Member

    It will be back up later with more stuff on it.

    And I don't know about Teredo. Will have to check it out.
  69. Porter

    Porter Addicted to LI Member

    There is no Overall Inbound Limit in Tomato 1.27

    Hi everyone,

    I'm a bit confused after what I think I found out today. But at the same time I still think that I'm probably wrong, so I'd like to ask if anyone has experienced this, too.

    I'm still new to Tomato, which means I bought a WRT54GL v1.1 a few days ago, put the officiall Tomato 1.27 on it and just started experimenting. Soon I realized that inbound QoS didn't work as well as I wished it would. I have an 4500/600 ADSL-Connection and there are three to four active users. P2P and HTTP are the most common services, so I was trying to give P2P as much bandwidth as possible without ruining the http responsiveness. While experimenting I found out that http- and P2p-Traffic ends up in the right Classes - inbound and outbound. Http goes in High, p2p in Lowest.

    My next experiment was to find out, how well http would get priority over p2p-traffic. For that I started two very well seeded torrents and surfed the web quite heavily, which meant I opened several big websites at a time, which wouldn't end up in the http-download-class. What I expected to see, was a significantly drop of p2p inbound traffic while surfing the web. This didn't happen at all. P2p-traffic stayed at its given limit. What I noticed on the other hand was something I didn't expect: Although my overall inbound limit was set to 3000 I could easily generate enough traffic with p2p and http, that I would reach my physical line speed of 4500, which meant that for some reason the overall inbound limit wasn't working.

    My basic setup:
    [​IMG]

    After that I wanted to know, which tc commands Tomato used to shape my traffic, so I connected via ssh and searched for the script. It's in /etc/qos and the iptables-commands are in /etc/iptables, although this doesn't seem to be a standalone script. Inside the qos-script I found these commads for inbound shaping:

    I=ppp0
    TQA="tc qdisc add dev $I"
    TFA="tc filter add dev $I"

    $TQA handle ffff: ingress
    # ingress 1: 80%
    $TFA parent ffff: prio 2 protocol ip handle 2 fw police rate 2400kbit burst 96kbit drop flowid ffff:2
    # ingress 2: 70%
    $TFA parent ffff: prio 3 protocol ip handle 3 fw police rate 2100kbit burst 84kbit drop flowid ffff:3
    # ingress 3: 95%
    $TFA parent ffff: prio 4 protocol ip handle 4 fw police rate 2850kbit burst 114kbit drop flowid ffff:4
    # ingress 4: 99%
    $TFA parent ffff: prio 5 protocol ip handle 5 fw police rate 2970kbit burst 118kbit drop flowid ffff:5


    And now I searched for the number 3000, since I expected my overall rate to be in there somewhere. Notice something? It isn't! What we find there are independently working filters for shaping inbound traffic, but no overall inbound limit. That's why the combined traffic of p2p and http could fill up my connection to its physical limit and not as intended just to around 3000.
    2970 + 2400 = 5370!

    Or to restate my thesis: there is no Overall Inbound Limit.

    The number we can enter on the settings page under "Max Bandwidth" is used to calculate the limits for the classes on the website, but nothing else.

    I hope that I didn't overlook something or that this behaviour is already known. I only found this post by Toastman, who did seem to suffer from the same problem: http://www.linksysinfo.org/forums/showpost.php?p=340797&postcount=56

    If my findings are true, then there needs to be another solution to good inbound traffic shaping. Probably it's using the IMQ-Device. I haven't read up on that enough, yet. But this seems promising.


    Porter
  70. Azuse

    Azuse Networkin' Nut Member

    Dont use none for your highest inbound. None means 0-100.
  71. Porter

    Porter Addicted to LI Member

    Azuse:

    Sorry, but that's wrong. None means, that the traffic from an inbound Class isn't limitted. This can also be derived from the tc-commands I posted, because there are only four filters which limit incoming traffic in Lowest, Low, Medium and High, but there is no fifth filter for Highest.
  72. gingernut

    gingernut Addicted to LI Member

    Here's a good extract from here that maybe explains how SFQ, default used by Tomato, works in some detail.

    SFQ
    Stochastic Fairness Queuing (SFQ) cannot limit traffic at all. Its main idea is to equalize traffic flows (TCP sessions or UDP streams) when your link is completely full.

    The fairness of SFQ is ensured by hashing and round-robin algorithms. Hashing algorithm divides the session traffic over a limited number of subqueues. After sfq-perturb seconds the hashing algorithm changes and divides the session traffic to other subqueues. The round-robin algorithm dequeues pcq-allot bytes from each subqueue in a turn.


    I saw that the author of the Gargoyle firmware has just swapped from using SFQ over to using Red Queues to try and improve on it's QOS implemetation. Not sure if would be possible to try out on Tomato but does seem a good alternative.

    RED
    Random Early Detection is a queuing mechanism which tries to avoid network congestion by controlling the average queue size. When the average queue size reaches red-min-threshold, RED randomly chooses which arriving packet to drop. The probability how many packets will be dropped increases when the average queue size becomes larger. If the average queue size reaches red-max-threshold, the packets are dropped. However, there may be cases when the real queue size (not average) is much greater than red-max-threshold, then all packets which exceed red-limit are dropped.
  73. Azuse

    Azuse Networkin' Nut Member

    Logically none would mean no limit is imposed. However tomatos inbound doesn't work lie that.

    Ignore your commands for a minute. The percentage in the inbound field is the bandwidth that class is guaranteed not the shaping applied i.e. 80+70+95+99=344%. None doesn't mean that the traffic from an inbound Class isn't limited, it means that the traffic from the class is guaranteed 0 bandwidth. However it behaves as if 0-100.

    That is unless something has changed.
  74. Toastman

    Toastman Super Moderator Staff Member Member

    I believe Porter is right. And this should be fixed if possible.

    I'd always known that the maximum limit was used to calculate the percentages for the individual classes and that setting NONE would ignore the limit of 100% (of that set maximum). But - that each individual class limit can add up to exceed the maximum setting is the explanation for my post. Well spotted. I wonder if it was always like this? It probably was!

    If anyone can work out the method to add up all incoming classes and apply a true maximum limit, it would solve one of the few remaining problems. Ideas, anyone?

    Am busy for a few days but will get back to this.
  75. Porter

    Porter Addicted to LI Member

    Azuse:

    I know what you are referring to, when you talk about "rate", because often you need to use the "ceil" parameter to shape to a specific datarate. But I haven't quite figured out, what these inbound rules actually mean. I just noticed that they somehow limit the right traffic to a certain amount of Kbit/s, but not the whole link.

    Speaking of knowledge or lack thereof I would like to post some links, so that everybody who's interested can join. These links are just some I came across in the last days:

    Maybe a good starting point:
    http://www.opalsoft.net/qos/DS-21.htm

    Linux Documentation:
    http://www.tldp.org/HOWTO/pdf/Traffic-Control-HOWTO.pdf
    http://www.tldp.org/HOWTO/pdf/Adv-Routing-HOWTO.pdf

    This still doesn't solve our problem of a missing inbound shaping possibility, but maybe this does: http://www.linuximq.net/index.html

    I've noticed that Tomato already has this compiled as a module, so hopefully, we can use it right away. Building a script without an interface is really difficult. A few years ago I had a real PC running linux with a traffic shaper called Mastershaper. A demo can be found there: http://www.mastershaper.org/index.php/Demo

    I still got this thing running in a VMware, so I can easily make scripts for myself. In the next few days I will try to experiment with putting it on Tomato and running it there. Don't get me wrong though: I won't port Mastershaper to Tomato, I will just use the script Mastershaper compiles and use it with some bash-scripting on Tomato. I already looked at Mastershaper's php-Files and came to the conclusion, that porting it from php to whatever Tomato uses might be a very long job.

    Have fun reading up, hopefully we will find a solution!
  76. rhester72

    rhester72 Network Guru Member

    The problem with ingress shaping is that you only control the link from the router to the end stations.

    Let's say, for instance, you are torrenting a large Linux ISO file that can completely fill your inbound bandwidth pipe and then you attempt to do something latency-sensitive like a VoIP call.

    Both the torrent senders and the VoIP senders are going to try to get the packets to you as quickly as possible - likely far more quickly than your inbound pipe can hold, so packets begin queueing at your ISP. As those packets enter your router, ingress shaping can basically use a router-local queue to change the order of delivery, such that VoIP packets (when queued) are delivered before the torrent packets, but therein lies the problem...you can't control when the *router* will get the VoIP packets, because you don't control the ISP's ingress queue. Thus, the _best_ you can do is literally delay or drop packets of lower priority...which doesn't help a thing, because it doesn't get the higher-priority packets into your router any faster - the packets still have to show up at the WAN edge of the router before the iptables decision tree even comes into play (so by the time you can decide to prioritize the packet, it's already too late - you've got it, so you may as well just deliver it...there's no point delaying it because it won't make others come faster, and there's no good reason to drop it because the receiving station is just going to ask for it again).

    Long story short - really, really, no matter what you may read on the forbidden corners of the 'net, consumer ingress shaping simply doesn't work because it *can't* work, no matter how clever the algorithm. This will remain the case until end-to-end QoS is possible...and I expect that right after IPv6 (I'm sure you can detect the note of sarcasm there ;).

    Rodney
  77. Porter

    Porter Addicted to LI Member

    I don't think we are entirely helpless when it comes to ingress shaping. Sure, ISPs have queues that we don't have control over directly, but ingress shaping tries to take control in an indirect way. By shaping our inbound interface we can prevent large queues building up along the way, so that when there actually is important traffic, i.e. time-sensitive traffic like VoIP, chances are better that this service won't suffer as much and people can still use it. We sacrifice a part of our overall bandwidth to ensure stability of the link.

    Today I have managed to get a QoS-script running, that finally did what I wanted it to do: prioritize.

    [​IMG]

    In this image you can see the utorrent traffic graph. Inbound bittorrent traffic dropped immensly when I started surfing the web, so finally http traffic gets priority over p2p. I wasn't able to achieve this with just the QoS-settings the Tomato UI provides.

    Well, that's all the progress for now, maybe tomorrow there will be some new stuff.
  78. rhester72

    rhester72 Network Guru Member

    That's not where I'm coming from at all. All you've shown is the traffic rate of packets arriving at uTorrent. That says nothing for what was happening on the link.

    For a simplistic example, let's say packets arrive at the following order on the router (T=torrent, V=VoIP):

    Time=1: T
    Time=2: T
    Time=3: T
    Time=4: T
    Time=5: V

    If your desire is to prioritize VoIP traffic over torrent traffic, there is *nothing* you can do with traffic shaping to make V come any earlier than time=5. You can attempt to artificially degrade the bitrate of the torrent, but your options for doing so are a) to do so at the application level, which does work or b) to do so at the router level, which *doesn't* work.

    Why?

    For each packet, which comes into the router in an indeterminate and unalterable order, the router _must_ either a) deliver the packet, b) queue the packet, or c) drop the packet. a) is the default, b) and c) are the attempts people have used to "shape" ingress traffic. The speed penalty incurred by choosing a) is almost immeasurably small as opposed to b) and c), so there's no particular reason to even _attempt_ shaping. b) will simply drive up latency of torrent packet delivery to the end station *without improving latency or bandwith of the VoIP packets* (and at the cost of more memory utilization _and_ reprocessing time), and c) is just utterly without value - all that will happen is you will force the end station to request the packet again, which it will happily do, and now you've potentially incurred a penalty of throwing the data away *twice*.

    There is simply no such thing as "shaping an inbound interface" in any meaningful way that proves to be anything other than snake oil when analyzed by a sniffer or tap between the DSL/cable modem and the router. I would be most happy to be proven wrong, but the math is pretty solid (and simple). There's no free lunch.

    From what I can see from your above graph, your rules chose path c). This is actually the worst-case scenario.

    Rodney
  79. Azuse

    Azuse Networkin' Nut Member

    No.1 your just stating the obvious, everyone knows qos can only truly be done by the sender. No one is trying to prioritize they want to reduce the latency when latency sensitive apps use the line but have downloads run at full speed the rest of the time.

    No.2 Reducing the inbound traffic through dropped packets/delayed responses forces the sender to reduce transmission speed reducing the queue at the isp end.

    No.3 While not "true" qos by preventing the inbound choking you drastically reduce pings for everyone. My uk pings are 28-30 for game servers, by limiting inbound bandwidth they rise to 38-40 if a download/torrent starts making them quite playable. No inbound limits and the 4-600 pings make it completely unplayable.

    Tomato isn't exactly new, but line have changed alot in the past few years. It used to be enough to simply control the uplink to control the down but most lines now have a much higher uplink than then need to simply download. Personally my line has ~4x more up speed than needed for downloading therefore controlling the uplink, while important, has no effect on inbound traffic which if uninfluenced negatively impacts every user. Precisely what qos is ment to avoid.

    It only takes 5 minute playing with it to realise that the inbound qos functions as 10 independent rules meaning if they exceed 100% they become useless, and if they don't then most of the bandwidth is wasted most of the time. I understand why it was never built to function like the outbound i.e. throttle back a lower class if a higher one begins to use the inbound but we've long since passed the point where we need to influence the inbound to prevent things being ruined for everyone.

    Until tomato can actually do this (and if Porter does then he'll make alot of friends :biggrin:) most people who want this will just continue ti use the mac filter in the raf mod to choke downloads instead. It's cumbersome, require the input of ips and limits where most of us would just rather use the inbound qos boxes, but it most certainly works. Even if it is just a workaround.

    Also don't way snake oil :) This pc does a fair bit of heavy downloading, but the other devices on the lan continue to brows, stream video and game no matter how many torrents are pushed through this.
  80. rhester72

    rhester72 Network Guru Member

    Regarding No. 2:

    This is only true of TCP connections, where sliding windows can be utilized to force sender delays. It does not apply at all to UDP, which is the realm torrents live in. I maintain my stance that there is no ingress shaping solution to this problem. If you feel otherwise, enjoy it. =)

    Rodney
  81. Azuse

    Azuse Networkin' Nut Member

    Oh that's true, fortunately there's a fair amount of tcp going around, to which it does work. My torrent client still have a large amount of tc connections despite the switch to udp, tc will never really go away :) Also quite afew p2p programs use tcp heavily, I assume because of drm concerns but things like spotify seem udp allergic.

    Torrents only recently moved onto udp en-mass, and it's true the tcp tactics wont work, but it's also true the uTP used will reduce it's own speed if it detects any congestion (I believe the target delay is around 100ms) so creating your own with the router will effect the clients themselves.

    Until my isp offers qos there will never be a perfect solution however my partial mac limiting fix certainly has positive results, even if it isn't what I'd like. We might not be able to control what is sent, but we can influence the rate at which it is done. If porter can create a script which only influence half the data it's far better than influencing none of if =D
  82. Toastman

    Toastman Super Moderator Staff Member Member

    Under normal circumstances using dropped TCP packets to slow down incoming data is the real key to making router "QOS" work. I guess, if I add things up, I have something like 2000 users in these apartment blocks who could not use the internet properly before Tomato QOS made it possible. If the QOS is switched off, even ONE P2P user can screw up the internet for everyone in the entire block. And that is why a great many people are using Tomato.

    But here is an issue that would be nice if corrected. And it would seem to be a relatively small issue, because I think that the router has actually got the information needed to implement an overall limit, which would make the QOS work as we believed it was intended.

    Is there anyone who can look at this with a view to adding it to the source code? Porter, can you share your script and describe what it does, it may be useful for others?

    Rhester is quite right in that it is UDP that poses a big problem. But let's try to deal with one thing at a time. The TCP overall limit issue is a very relevant one. That is something I commented on before, as Porter found, and there have been numerous other posts about this in one guise or another over the past 3 years. There *should* be an overall limit that works, or the traffic shaping we do have can be defeated. As probably the biggest Torrent user in these blocks, I am particularly concerned that we have control over it. I can handle the UDP just by dumping it if necessary - torrents will still continue to work with TCP. And much better, I might add.

    In the past, torrents were mostly carried by TCP. While there was some UDP traffic, DHT and the like, it never generated much in the way of downloads (I have never seen any) but created a lot of completely useless overhead traffic.

    By switching off DHT and dumping UDP the download speeds increased. Now things are changing with this new scheme, uTP, oddly enough created by the company that is trying to use OUR bandwidth for commercial purposes (DNA). I think we need to be very suspicious about that. Every user in my apartments who has realized this has now dumped the latest versions of uTorrent after initially blaming things on the ISP.

    Looking at the peers for a good seed also shows that most experienced "power" uTorrent seeders have deliberately switched off uTP (The P flag is not showing). While uTP does produce some downloads, they are not very significant, and the traffic created is too high. Globally, ISP's are very concerned about this uTP crap and are far from being pleased !

    If it ever does catch on - we need to look at what is happening with the TCP carried over UDP as with uTorrent (??), and how it works. There may be a way to control it. Perhaps L7 ? But in my apartments the problem is actually going away as people realize it is not doing anything for their downloads. People aren't stupid.

    And Rhester - I agree - I am not expecting to see IPv6 during my lifetime. If I do - the shock will probably kill me anyway :grin:

    EDIT - August 2010 - uTorrent has been using this stupid new UDP protocol for several months now. However, downloads still increase in speed with it switched off and only TCP delivery enabled. Typically, using TCP only results in 4 times the download speed.
  83. Azuse

    Azuse Networkin' Nut Member

    Make it go away please :tongue:

    [​IMG]
  84. Porter

    Porter Addicted to LI Member

    Just a few words, before I get to the interesting part. Yes, I was suprised when I found out about Tomato's inbound QoS-capabilities, so I wanted to do something about it. For me, there were two options: 1. try my old script and see what it was capable of under Tomato or 2. Robsonn's Generator. I've donwloaded the generator and played with it a bit, but wanted to try the old script first, becaus this seemed easier. On the other hand I still couldn't find out how well Robsonn's generator works. The only thing I noticed without explicitly searching for it was, that nobody actually seemed to use the generator. So my question is: is there anybody who has some experience with it?

    There's another point I would like to comment about. I think Linux-QoS is really antihuman, because the whole topic is a bit complex, the documentation often lacks good qualitiy, although I think it improved, and last but not least the commands used to do QoS are highly cryptic and just scare people off.

    That being said I'm happy to help, but I'm not an expert myself, so everything I do could be buggy and most of the time I need to read the documentation, because I just don't know.

    Let's talk about the script then.

    [​IMG]

    This picture is taken from the Mastershaper UI and at the top you can see the Arcor-Service Level which is the root Queue and you can also see, what its limits are. This root queue contains the limits I want my physical line to shape to. Every other service level ist somehow attached to the root queue. The In- and Out-Limits you see is the guaranteed bandwidth each level gets (except for the root class, which the way mastershaper is organized accounts for). What you can't see in here is how much maximum bandwidth each service level can get. This wolud have been to much work to extract, but you can find it in the script that I'm providing now.
    The script itself for some reason has at some point many duplicate commands, which don't hurt, only the script gets bigger.

    View attachment mshaper1.zip

    Inside the zip-file you will find two scripts, one for starting the shaping und one for stopping it.
    I still don't know how to make files survive a reboot, so please keep in mind that once you reboot, you have to follow the instructions once again. On the other hand this probably is a bit safer, because you don't actually change something.

    Now the instructions:

    1. Install putty (other clients might be fine, too, but that's what I used).
    2. Connect to your Tomato via ssh (as root).
    3. Now a few commads you will need to enter:
    4. cd /root
    5. cat > mshaper
    6. open the file mshaper.txt in Windows and copy everything to your clpboard; i.e. ctrl+a, ctrl+c
    7. get the putty window active again and just press the right mouse button somewhere over the black background - the text should appear in there
    8. In the putty window hit ctrl+d. This should bring you back to the command prompt showing this: # (if it doesn't, try to hit enter once and then ctrl+d)
    9. chmod 744 mshaper (this will make our script executable)
    10. cat > mshaper-stop
    11. repeat steps 7 and 8 for mshaper-stop.txt
    12. chmod 744 mshaper-stop

    Don't start anything just now, first Tomato's QoS function needs to be turned off, so go to the QoS section, deactivate the box and save.

    Back in the putty-window the script can now be executed:
    type on the command prompt: ./mshaper [enter]
    On my Box this took about 12 seconds, so just wait. If there are error messages, please post them here.
    Now you can experiment with your traffic.

    To turn of the script use: ./mshaper-stop [enter]
    I don't know if this stop-script really sets back everything to the previous state, but I hope so...
    After all you can always reboot.

    You will have to reexecute the script after each reconnect.

    If you want to know how the script works just open it in an editor and read the comments. There you can also see how much bandwidth each service actually gets. Just to give you one figure I did test (I haven't tested any others): Bittorrent downloads get about 320KB.

    The next step would be to test Robsonn's Generator, because then erverybody can save their configuration as a project and this project-file can be easily adapted, because everyone just needs to adjust their bandwidth-figures, which shouldn't be to difficult. That way, fewer people have to deal with the difficult commands or knowledge.

    Well, that's it for now. Good luck testing the script!


    UPDATE:

    I'm sorry, I forgot something. In the script there are the standard bittorrent-ports and my personal port, which I use for portforwarding. It's port 55202, so if you want to make sure your traffic gets matched you should edit the script to whichever port you use. Please keep in mind, that this script does not use L7-Matching, so everything that doesn't get machted will end up in the low triffic class, called "fallback".
  85. Porter

    Porter Addicted to LI Member

    Test results

    Since nobody else has been posting his experiences with the script I will now post mine. I'm working with a WRT54-GL.

    I had the script running for a few days and the most important finding is that it works quite well, although not perfect. What it does well is preventing congestion, that means that ping times are rather stable. It also does prioritize http-traffic well, so no change to my initial observation. What it doesn't achieve so well is to limit bittorrent-traffic, which isn't necessarily a flaw in the script. The script is rather stupid, because the tc-commands can only limit certain ports. This means that if a connection decides to switch ports, which is possible with TCP (correct me if I'm wrong), then tc won't track that change. Every connection that doesn't use one of the specified ports in the tc-commands will end up in the default Class. So I think that sometimes there are bittorrent connections who are not filtered by tc because they use different ports and therefore end up in the default Class, which then leads to higher download rates over bittorrent. Not exactly what I meant to achieve, but in this setup I don't think this can be prevented. One last thing I would like to talk about is the amount of cpu power needed to run the script. On my connection (4500/600) this script is really a piece of cake. I had idle times of 60-70%, but didn't remember to look at the load, sorry. But I think it's save to say that only using tc to shape your connection is your best shot when dealing with high connection speeds.

    There may be a solution to the problem described above, that there are probably not alle bittorrent connections getting matched and filling up our default queue. The solution would be to use iptables' connection tracking capability, which notices when TCP-connections change there ports. I did not yet test this thoroughly, but my first test was a bit disillusioning, because there was considerably more cpu time needed. With a full link I had idle times of about 40%, which would mean, that this script needed 30% more processing power. And this script didn't even use L7-filters. Please keep in mind, that this only were preliminary tests, I will conduct some more.

    As I've mentioned before I don't think I will be able to make the changes to Tomato's UI and code myself, because this would take up much more time then I have. On the other hand I don't think that there needs to be done quite a lot. The requirements are already in Tomato, for instance the kernel already has all the needed modules and I even think that the QoS UI just needs to be upgraded and not rewritten. So I will compile a mail for the author of Tomato with all the information that has been gathered here and hopefully he will be able to introduce this feature in one of the following releases.

    Well, let me know that you think.
  86. Azuse

    Azuse Networkin' Nut Member

    I think trying to construct any rule around torrents using a specific port(s) is a waste of time, Period. I use a specific port, but there are plenty p2p based apps over which I have no control and even if I did some other user would inevitably come along and use different ports anyway. Torrents use any port, hence the impossible task of blocking them, so no tracking of any ports will work. The only reasonable solution is to have the default class the lowest with everything you actually want prioritised above that.

    In other words, or my view at least, all that's actually needed is for it to give priority in order of class (like the outbound). Essentially that's all anyone really wants and basic what people expect when the see the inbound qos. Build a script, or modify tomato, that works based on class rather than ports, and your there. Slowing torrents isn't an issue for the script, it comes down to your classifications as always.

    Sadly I myself have zero coding skills. My talent lies in prodding others work until I manage to break it :)
  87. Toastman

    Toastman Super Moderator Staff Member Member

    I have been using QOS successfully to control multiple users in several very large buildings for some years now. By and large, the existing method of allowing any stuff you don't explicitly address with a QOS rule to fall into the default class works extremely well. Control of outgoing data is achieved with prioritization, and limits. I think that incoming stuff does not particularly need any "prioritisation" with regard to delivery to PC's on the LAN, but rather the prioritisation is really used to decide which packets to drop first to control incoming data. To prevent it arriving or slow it down, we can use the incoming class bandwidth limits to force the remote server to back off by dropping packets - using the retransmission timer's backoff implemented in the TCP protocol itself. The thing that occasionally screws it up, as discovered by Porter, is this lack of an overall limit.

    I've been busy and not had any time to look at this, but likewise, my coding skills are very close to zero. I'm sure it would not be so hard to add this, but I am frustrated I can't do it.

    The other thing that is missing is incoming class bandwidth data - another pie chart similar to the 2nd, outgoing one. That would make network admin so much easier! I think this would also be possible to implement. Anyone? EDIT - now added in later versions of Toastman Tomato...
  88. Porter

    Porter Addicted to LI Member

    Azuse:
    You could use L7-Filtering to detect traffic which uses different ports. But that's futile when the traffic gets encrypted like bittorrent traffic does. So I have to agree with you, that the best way to deal with it is to put it in our default Class. Nonetheless I do match bittorrent traffic, because I don't use encryption (my ISP doesn't care) and I like the idea that unidentified traffic gets in my default Low Class and not Lowest, because sometimes there is traffic that I can't identify and which would then have to compete with my bittorrent traffic. This happened to some video streams before which then weren't watchable. But that's my setup because my network is rather small and I can intervene easily if there's a problem. With big networks you need a best effort script.
    I think there's a theoretical misunderstanding when it comes to how Tomato works right now. The way Tomato's QoS works right now it implies, that there are inbound classes like there are outbound classes. That's actually an inaccuracy in the UI.

    To clarify here's a quote from the Linux Traffic-Control Howto:

    Using Classes to shape your traffic doesn't mean that you won't use filtering (which means classifying) by ports. Right now Tomato's inbound QoS works without classes just as described in the Howto. What my script does is to make use of the intermediate queueing device (IMQ) which circumvents the restrictions of the ingress part of each interface by making every traffic look like outbound traffic, which then makes it possible to make use of every shaping utility (whicht means real classes of traffic that allow limiting).


    Toastman:
    I still don't understand exactly why nobody seems to like inbound prioritization. The way Tomato works right now there is no bandwidth borrowing between the different types of traffic. For instance there are only two types of traffic that you configured in inbound QoS, that's bittorrent and http. Each gets 50% of the whole bandwidth. So when there is data on both flows then http will never get more then 50% of the bandwidth, just like bittorrent, they are equals, no prioritisation occurs. Another problem is that with this setup you always have free bandwidth that's wasted, because nobody can get more then 50%. That's why prioritisation is such a cool thing although you have to drop packets that are already there. But that's just the nature of your internet connection. Your bandwidth is never infinitely high, so at some point there will me packetloss. Inbound QoS is just an artificial way of installing a physical limit for TCP to notice and act accordingly. To make use of your connection in an effective way, which means not to waste bandwidth, you need to do prioritisation.

    One last thing concerning the pie charts: I don't like them alot, because they just look nice, but don't provide enough information. I would like to know how my traffic flows over time, so I would like to have a real graph with a colour for each traffic class.
  89. Azuse

    Azuse Networkin' Nut Member

    That's actually quite clever even if it is beyond me.

    There's some stigma against inbound qos mostly because the outbound is the more effective. Unfortunately, and I can only speak about Europe, we passed the point where isps provided minimal upload speeds five years ago. I have roughly 3/4 times the upload speed I need to download at full speed as have alot of people (excluding those stuck on certain cable networks*). The odd thing is, considering the number of people who say it's not worth the time, is there are a lot of people using the raf specifically for the mac limiter so that they can get the most out of their line at all times. I wonder what it does with all those packets it doesn't let through :)

    Don't worry about dropped/delayed packets. The only difference between what you're doing and what a long phone line does is you're controlling the loss to your benefit rather than being lost in transmission.

    *Virgin at least is planning a large increase in uploads by the end of the year, so as far as the UK goes, the outbound qos is almost past its inbound effectiveness.
  90. Toastman

    Toastman Super Moderator Staff Member Member

    Understood. For me, the graphs are ideal. I have absolutely no interest in who downloads what over a long period of time. But I need to know at a particular moment in time how QOS rules are actually affecting users. The pie charts are easy to read and show up quickly any class suddenly increasing to take up a large segment of the pie, and is easily visible from a distance. That's why I like the pie charts. There have been a large number of posts from people who wanted this feature over the years. I think it would be easier to use the existing GUI to add another pie, and later another page for time/bandwidth charts.

    BTW - I believe Gargoyle has inbound prioritization, maybe you might have time to check out the source code, if it's available, and see if there's any overall limit in use? It also has a pie chart for inbound classes.

    So - there is no inbound QOS as such. It's just a class bandwidth limiter. It is more helpful to think of this as part of an "overall" strategy for controlling flow, which has somehow or other become known as "QOS" on SOHO routers.

    EDIT - Inbound IMQ based QOS is now incorporated in Toastman builds... 2012
  91. Porter

    Porter Addicted to LI Member

    Hey Toastman, thank you for the hint about Gargoyle! It indeed seems to have a more elaborate QoS-System, although the UI does lack some serious style... From what I've seen they are adding features quite regularly. Unfortunately I will have to wait for a more silent time at home to try out this firmware. Looking at the source code isn't my first choice because it's just really difficult. Installing it is the better choice and more fun, because you get to play with something new.

    What I didn't know, but what Gargoyle points out is that Tomato isn't completely open source, which surprised me a bit: http://www.gargoyle-router.com/wiki/doku.php?id=faq#what_about_tomato_and_dd-wrt_they_certainly_have_very_nice_user_interfaces_aren_t_they_open_source_projects

    But since there are so many mods of Tomato which probably work with a different UI this doesn't seem to big real concern.
  92. Toastman

    Toastman Super Moderator Staff Member Member

    Porter, Jon put a restriction on the use of the Tomato GUI that limits it to Tomato unless permission is given.

    I believe that Tomato is characterised by the GUI, the Pie Charts and graphs, and the QOS system.

    There are moves afoot by new and impressionable some code fiddlers to change the GUI, and I personally am opposed to that. Changing a perfectly functional and clear GUI to something else is somewhat of an insult to Jon Zarate, it doesn't make the router any better, and it isn't Tomato. For those twiddlers who want to take someone elses' work and wreck it to suit your own personal opinion of what constitutes an improvement, I ask you please to have some respect for Jon Zarate and leave it alone. Write your own firmware and call it "Beetroot" or whatever. Pay due homage to Jon in your credits. If people add any new features to the GUI, I hope they will respect the look and feel of the firmware and not start adding stupid graphics and pictures of half-naked girls. (Send them to me instead).

    The rest of the code is quite open. Looks like there is also a restriction of some sort on the Gargoyle GUI, but I am afraid I didn't understand the woffle.

    BTW - may I suggest, rather than just compose a mail with your suggestions and just forward to Jon, post it here also. There is a much greater chance of someone on this forum coming up with it than the original author.
  93. Porter

    Porter Addicted to LI Member

    Ok then, here is a summary of what seems to be the problem and a request for help. This is the email Jon got, with a few things added or corrected.



    I have been trying to use Tomato’s QoS capabilities in the last week and came across what I would call a design flaw. The problem concerns the way Tomato is shaping inbound traffic. I’ve already joined a discussion here: http://www.linksysinfo.org/forums/showpost.php?p=360754&postcount=185 explaining the problem. My thesis was, that there is no Overall Inbound Limit. I will now try to sum up the discussion for you, so that all the information is in one place.

    Taken from the forum, but with a correction here and there:

    “I'm still new to Tomato, which means I bought a WRT54GL v1.1 a few days ago, put the official Tomato 1.27 on it and just started experimenting. Soon I realized that inbound QoS didn't work as well as I wished it would. I have an 4500/600 ADSL-Connection and there are three to four active users. P2P and HTTP are the most common services, so I was trying to give P2P as much bandwidth as possible without ruining the http responsiveness. While experimenting I found out that http- and P2p-Traffic end up in the right Classes - inbound and outbound. Http goes in High, p2p in Lowest.

    My next experiment was to find out how well http would get priority over p2p-traffic. For that I started two very well seeded torrents and surfed the web quite heavily, which meant I opened several big websites at a time, which wouldn't end up in the http-download-class [correction: I now know that the filter rules are for http-uploads, why aren’t there any for http-downloads?]. What I expected to see, was a significantly drop of p2p inbound traffic while surfing the web. This didn't happen at all. P2p-traffic stayed at its given limit. What I noticed on the other hand was something I didn't expect: Although my overall inbound limit was set to 3000 I could easily generate enough traffic with p2p and http, that I would reach my physical line speed of 4500, which meant that for some reason the overall inbound limit wasn't working.

    My basic setup:
    [​IMG]

    [​IMG]


    After that I wanted to know, which tc commands Tomato used to shape my traffic, so I connected via ssh and searched for the script. It's in /etc/qos and the iptables-commands are in /etc/iptables, although this doesn't seem to be a standalone script. Inside the qos-script I found these commands for inbound shaping:

    I=ppp0
    TQA="tc qdisc add dev $I"
    TFA="tc filter add dev $I"

    $TQA handle ffff: ingress
    # ingress 1: 80%
    $TFA parent ffff: prio 2 protocol ip handle 2 fw police rate 2400kbit burst 96kbit drop flowid ffff:2
    # ingress 2: 70%
    $TFA parent ffff: prio 3 protocol ip handle 3 fw police rate 2100kbit burst 84kbit drop flowid ffff:3
    # ingress 3: 95%
    $TFA parent ffff: prio 4 protocol ip handle 4 fw police rate 2850kbit burst 114kbit drop flowid ffff:4
    # ingress 4: 99%
    $TFA parent ffff: prio 5 protocol ip handle 5 fw police rate 2970kbit burst 118kbit drop flowid ffff:5


    And now I searched for the number 3000, since I expected my overall rate to be in there somewhere. Notice something? It isn't! What we find there are independently working filters for shaping inbound traffic, but no overall inbound limit. That's why the combined traffic of p2p and http could fill up my connection to its physical limit and not as intended just to around 3000.
    2970 + 2400 = 5370!

    The number we can enter on the settings page under "Max Bandwidth" is used to calculate the limits for the classes on the website, but nothing else.

    I hope that I didn't overlook something or that this behaviour is already known. I only found this post by Toastman, who did seem to suffer from the same problem: http://www.linksysinfo.org/forums/showpost.php?p=340797&postcount=56

    If my findings are true, then there needs to be another solution to good inbound traffic shaping. Probably it's using the IMQ-Device. I haven't read up on that enough, yet. But this seems promising.“

    As I found out later Tomato already has everything available to do good inbound shaping, because the modules for the IMQ-Devices are already there.

    There was a discussion about the usefulness of inbound shaping. Probably because Tomato doesn’t shape correctly and therefore people think it’s useless. I don’t know if you have given up on inbound shaping. I’m still of the opinion that it is needed for good overall throughput:

    “I still don't understand exactly why nobody seems to like inbound prioritization. The way Tomato works right now there is no bandwidth borrowing between the different types of traffic. For instance there are only two types of traffic that you configured in inbound QoS, that's bittorrent and http. Each gets 50% of the whole bandwidth. So when there is data on both flows then http will never get more then 50% of the bandwidth, just like bittorrent. They are equals, no prioritization occurs. Another problem is that with this setup you always have free bandwidth that's wasted, because nobody can get more then 50%. That's why prioritization is such a cool thing although you have to drop packets that are already there. But that's just the nature of your internet connection. Your bandwidth is never infinitely high, so at some point there will me packetloss. Inbound QoS is just an artificial way of installing a physical limit for TCP to notice and act accordingly. To make use of your connection in an effective way, which means not to waste bandwidth, you need to do prioritization.â€

    Ok, this concludes the first part where I wanted to show you what the problem is and why it is beneficial for everybody who needs to take care of their connection to have good inbound traffic shaping.

    In the next part I would like to talk about what I would like to be able to do with Tomato and QoS.

    Let me first tell you that I’m very inexperienced when it comes to dealing with HTML or ASP, which makes it difficult for me to help with developing the UI needed for the changes to the QoS-System. I do have some knowledge about QoS in Linux though, because I used this project for some time: http://www.mastershaper.org/index.php/Main_Page
    Actually I built my tomato script for QoS with Mastershaper. This script Is included as an attachment. It believe it worked almost from the start, because Tomato already has everything that’s needed for good shaping, only it’s not being used by the UI, yet.

    View attachment 1208

    “I think there's a theoretical misunderstanding when it comes to how Tomato works right now. The way Tomato's QoS-UI works right now it implies, that there are inbound classes like there are outbound classes.[To be more specific: it implies that outbound classes have the same functionality as inbound classes which they have not.]

    To clarify here's a quote from the Linux Traffic-Control Howto:

    […]Right now Tomato's inbound QoS works without classes just as described in the Howto. What my script does is to make use of the intermediate queueing device (IMQ) which circumvents the restrictions of the ingress part of each interface by making every traffic look like outbound traffic, which then makes it possible to make use of every shaping utility (which means real classes of traffic that allow limiting and an overall inbound limit).â€

    This means that Tomato has to be taught to use IMQ-Devices – again: the modules for this are already there. You can see how the mshaper-script initializes, loads the modules and adds some things with iptables. After this the tc-rules are loaded. I’m not entirely sure how many side effects this script can have or that especially the iptables-rules really cause no harm. On my Tomato everything seems to work fine, although you need to reload the script every time the WAN-Interface comes up.

    A real important thing is performance. From my experience a traffic shaping script with only tc rules used for matching and filtering is rather fast. At the same time it’s dumb, because it can’t realize when TCP decides to change to another port, which leads for the connection to end up in the default class, which usually is Low. Iptables on the other hand can use connection tracking and is able to classify connections more accurately. This has a price. In my experience iptables-matches use considerably more cpu-power than just tc, even without using L7-matching. Therefore I think the UI would need to have to present a choice between the fast but inaccurate tc-rules and the slower (although newer more powerful routers a being sold now) but more accurate iptables-matches. This is a challenge for the UI, because it would be nice not to change the filters under QoS/classification when you change from tc-rules to iptables-matches and vice versa. Mastershaper has this ability, but unfortunately it’s written in PHP. I didn’t like to use Robsonn’s Generator because it doesn’t support multiport-matches, so for every port you need to make a new rule and for each new protocol it generates a new class instead of just applying a new filter to an already existing class. (This might not be easy to understand right now, but just to be thorough I thought I would give you my reason.)

    The pie charts already help a lot with figuring out what’s going on in the QoS-System, but I’m not content, yet. When there actually will be real inbound shaping it will be easy to make pie charts for the inbound QoS. This would be nice. But something more I would like to have is to be able to view QoS-traffic over time. Like the real-time bandwidth graph, only with coloured curves for each traffic class, inbound and outbound.

    Something that bothered me alot actually are missing explanations throughout the UI. That's not restricted to the QoS section of Tomato. Very little information is given on how to configure Tomato. You have to go on the internet to find your information there and that's something that makes setting up difficult. Although I really like the reduced and silent look of Tomato, there should be advice on how to configure it. I don't know what's possible with javascript etc. The information doesn't need to be always visible, but you need to make it available in the UI!

    One last problem I came across is the ICMP prioritization. Somehow my pings ended up in my default Low Class, but I still have no explanation for this. I will be looking into it at a later time. Perhaps anybody else noticed already.

    Well, I hope I’ve provided you with enough information so that this can be implemented. If you think I could help or somebody else could help please contact me or perhaps send a message to the Linksysinfo-Forum.

    Good Luck,
    Porter
  94. alien3456

    alien3456 Networkin' Nut Member

    I happen to be pretty proficient with asp/css/javascript, and am also constantly fighting Tomato's QoS to prioritize the things I tell it to. I've wanted to check what QoS is doing to inbound as well, so maybe I'll see if it's possible for me to code that.

    I took a quick peak at the source, and it would take me a bit of time to figure out how it's all connected. A bit daunting. Maybe I'll start with a complete CSS overhaul; I could easily make Tomato look better, and in the process learn how it handles things like the pie graph.
  95. Porter

    Porter Addicted to LI Member

    I have opened a new thread concerning the development of Tomato's QoS-system, because I thought this thread here was more about using the QoS-system rather than changing it.

    That's the new thread: http://www.linksysinfo.org/forums/showthread.php?p=361270

    If you think you might be able to help (that's probably the case if you can write C) please feel free to join!
  96. rhester72

    rhester72 Network Guru Member

    I'm going to take it as read that you already have experience with using IMQ for inbound via OpenWRT or similar - because those folks have literally had it for years (I ran that way for about 3 years on my original 54G), and they'll tell you right away that a) it looks very good on paper and b) it doesn't help much/at all in practice. I feel like one of us is beating a dead horse here, but I encourage you to continue to pursue it as a learning experience if nothing else.

    I'm more than happy to compile the kernel and userland libraries for IMQ if you want to play around with it via modprobe.

    Rodney
  97. Porter

    Porter Addicted to LI Member

    I don't really get why you are still in this discussion. You neither read my posts carefully, because then you would have known that I never ran another firmware and that Tomato already does have IMQ-modules, nor do you think that it's even possible to be successful.

    So what's the point?

    Like everyone else you will have to proof your thesis.
  98. Toastman

    Toastman Super Moderator Staff Member Member

    Sorry, my friend. That's just a home server which runs on my PC, and it's been in use testing dozens of different Linux distros ( most of which aren't even up to the level of Windows 95 yet).

    Take a look now. I've placed some new compiles there which hopefully should work. There's a version of Victek's most stable RAF 1.25 v8515.2-ND with labelled classes added, and also something new - wireless connection speeds, ported from USBmod by Tedddy Bear.

    http://firmware.mooo.com/Toastman Builds/MIPS32R2 Kernel 2.6 (RT-N16 etc) Builds/

    Please note - the RT-N16 stuff is very much BETA and should be treated as such at this time. I will update it regularly.
  99. fun.k

    fun.k Networkin' Nut Member

    ah! sweet

    I'm trying Victek's mod on my WRT54GS v1 today (been on stock versions ever since I discovered tomato) but this per IP/MAC QoS looks dandy

    I'm guessing that I don't have to do a thorough NVRAM erase + can feed my current (1.25.8515) Victek settings

    Thanks heaps Toastman :)
  100. Azuse

    Azuse Networkin' Nut Member

    Probably because just adding the flash l7 filter seems to flag the initial request (i.e. port 80) as flash but no the stream. Add 1935 to your media ports and you'll catch more flash videos. It's been so long I don't even remember when adobe changed the port to 1935 but it still reverts to 80 if the link become saturated, the 10.1 beta seems slightly less likely to do this. Httpvideo has always caught everything for me bar 443 streams though :)

    Two things.

    First, I want to choke afew things by limiting their uplink speed. What's the ratio of upload to download? That is, for every 100KB something downloads how much will it upload?

    Second, Is the crawl class doing any good now there's so much uTP going around or is it still a redundant class :)

Share This Page