Using QOS - Tutorial and discussion

Discussion in 'Tomato Firmware' started by Toastman, Dec 24, 2008.

  1. Marcel Tunks

    Marcel Tunks Networkin' Nut Member

    Telnet/SSH into the router when downloading a large file with QoS enabled, enter the command top, look at CPU load (it will probably be at or above 1). QoS is CPU-intensive, with rising load as the bandwidth increases. Disabling all non-essential processes may be helpful, such as PnP, USB, ethernet port states, ip traffic and bandwidth monitor, web monitor, etc. Not sure if you can overclock an E2000. You can also compile Tomato with CPU optimization flags (-march=kwhatever -o2)
    As for QoS settings, disable (don't delete yet) any L7 filters.
    If your CPU load is less than 1, let us know and we'll rethink the situation.
  2. yasavvy

    yasavvy Reformed Router Member

    Thanks for replying quickly. I made a quick video showing the CPU usage while running and quickly fly through pertinent router settings. Keep in mind I just tossed this together for you and wasn't meant to be thorough or professional. :)

    Edit: I disabled all of the L7 entries in Classification and it suddenly clicked that disabled means that rule is disabled. For some reason my brain was telling me that if I enter my IP address and select disabled as the rule, that device would be "unlocked" and go full speed both ways. Wrong.

    But even if I set my IP to rule 1, 4, 6, or 8 which are all set to "No Limit" I still can't get more than 37~Mbps. It must be one of those settings, though.
    Last edited: May 19, 2014
  3. yasavvy

    yasavvy Reformed Router Member

    So after another 2 hours of testing, the only way I could get more than 35~Mbps was by setting rule 7-default from 5% / 60% to No Limit / No Limit. Even changing all classifications that mention "7-default" to "5-WWW" didn't change anything. So somehow is using rule 7-default yet it's not in the list or it's unspecified. But I thought used one of the ports and protocols already in my rules: 80, 443, 8080? I have no idea how to figure out what port is using so I can set a rule specifically for it.

    I also noticed that there's some mysterious rule #255 in QoS -> View Details. I don't have a rule #225. It only goes up to #45 in my classification list.

    When I sort bytes in / bytes out, I don't even SEE the bandwidth was/is using.

    Now I'm all messed up and none of makes sense.

    Edit: I searched YouTube for help with QoS and tomato tutorials to see if I can set mine up quickly. Here's what I found."qos" "tomato" -round -round6 -round5 -round4

    Despite the thousands of views, none of those videos explain the setup and configuration, so they were of no help. The majority that I've read about QoS has also been useless and didn't get me any closer to getting QoS working the way a network should in a typical home. Once I figure that out--whenever that will be--nobody with Tomato wanting to set up QoS will have to earn a doctorate just to get up and going ever again.
    Last edited: May 19, 2014
  4. Porter

    Porter LI Guru Member

    As you videos proofs, you have a clear issue with your cpu usage. The idle percentage goes to 0% percent while downloading = 100% cpu usage.

    1. I strongly advise you to revert back to the default settings. They have been tested and probably won't introduce more problems.

    2. I haven't read all your posts in detail, but do you really need QoS? You have a lot of bandwidth. Maybe the easiest solution is to just not use QoS. If you need QoS, that's totally fine, but you might have to get a more powerful router, although I can't give any recommondations right now.

    3. Meanwhile, you can disable the L7-filters. Especially the Flash-filter because that's a slow one and, at least in my network, it's pretty useless.

    4. You could just limit your bandwidth to what your CPU can sustain so that you still get real QoS. Maybe limit it so your CPU maxes out at 90% when you download, just to have some safety margin.
  5. cloneman

    cloneman LI Guru Member

    It seems to be the sirq% matters rather than CPU usage, although no one has ever explained to me in a way I can understand why sirq% can be at 100% and load can be below 1.00.

    Unfortunately this happens for "fast" connections. It would be nice to be able to disable downstream QoS completely for situations like this, while still benefiting from upstream QoS.

    Which router do you have? I think the best one available the supports Tomato QoS at the moment is the Asus RTN-66u. With that router, you should be able to handle your 57mbps connection - I think. My Sirq% hits about 40% when I use it on my 30mbps connection.

    EDIT: Just re-read your post. E2000 topping out where it does it probably correct. It's not a very fast router. (~350Mhz)

    Visualware VoIP is a good way to test if QoS is working. Like porter said, for troubleshooting purposes, you can set your max global bandwidth to 30Mbps which will of course severely throttle you, but at least QoS will work without overwhelming your router and you will be able to play with all the settings to see what effect they have.
    Last edited: May 20, 2014
  6. yasavvy

    yasavvy Reformed Router Member

    I'm not sure what I did exactly, but it seems to be working all of a sudden. Getting about 53Mbps downstream with QoS enabled and 10.53Mbps up which is pretty close to what I want. Despite SIRQ% shooting up to 99% again during a speedtest just now, everything was behaving normally. I fixed up and cleared a lot of rules, but I don't think it was the amount of rules -- I think it had to do with port overlapping or something. I'm feeling much more confident today than I was yesterday about QoS, but I still have lingering questions.

    These are the only two questions I have right now. If you can answer these, oh boy I'll be in your debt. Thanks.

    Edit: Also, with my current configuration, Visualware passes the ulaw (g.711) VoIP test both when doing a download and a youtube video upload. I get 1.5ms jitter when not doing anything intensive, and 6.4ms when under stress, which is still in the green according to Visualware.
    Last edited: May 20, 2014
  7. cloneman

    cloneman LI Guru Member

    Sometimes stuff gets unclassified, I don't know why.

    Rule 255 I guess just means default (no rule matched, so class 7)

    I believe "unclassified" traffic also hits rule 7, same thing as far as the router is concerned. I don't know why it didn't pick it up. I wouldn't be too worried about it. Do you have "Reset class when changing settings" checked? When playing with settings for hours on end things get messed up and it just might need a reboot.
  8. Touche

    Touche Reformed Router Member

    Hi, guys! Could you review my QoS settings if there are some changes I should make?

    I'm using:
    Asus RT-N66U with Shibby 1.28 MIPSR2-114 K26 USB VPN-64K
    55.5/33.5 Mbps VDSL connection as measured on local servers

    Out of L7 rules active, I have these:


    youtube 2012


    Could I relax max bandwidth limits or is 15% minimum no matter the speeds?

    Thank you!
    Last edited: Jun 8, 2014
    Moogle Stiltzkin likes this.
  9. yasavvy

    yasavvy Reformed Router Member

    Thanks for all the info, guys. I think I have enough now to make a video tutorial.

    However, you were right about the router performance with QoS enabled. My 54/12Mbps speed is too fast to handle QoS for downloading to my Linksys E2000, but it's fast enough to handle the uploading. I'm constantly finding myself enabling and disabling QoS depending on whether I'm uploading or download. It's pretty inconvenient.

    I almost think it might be better to somehow just do QoS over my server computer in the living room. Isn't that better? It's way more powerful than my router (E8400 Duo Core). I really don't feel like buying a new router just now. If I had a new one, I'd be able to make a more useful tutorial by being able to show what a difference performance makes with QoS.

    Oh well.
  10. Toastman

    Toastman Super Moderator Staff Member Member

    When you have something that a short time ago would have been an enterprise - level connection speed, obviously using a cheap consumer grade router which is limited in processing power is not really going to be the answer. Yes, there are newer routers that offer more speed and power, but as far as I am concerned they are far too expensive at the moment for general use by most members of the unwashed public. And some of those achieve their high advertised throughput by use of CTF so at the moment QOS isn't an option with those. Using a real PC which has massively higher processing power is obviously a nice solution if you don't mind the expense of the extra power and extra hardware. But then, you have to learn how to use a different QOS system, which may or may not be what you wanted.

    Do you really need QOS at home? If all of your computers are under your control, then it is unlikely that any one of them will hog all the bandwidth without your being able to control it. But if one of your family members breaks the rules and will not listen to you, then QOS can be of help.

    QOS is not necessarily there to help you get the maximum speed. It can, however, help you get a stable and working system under heavy and unpredictable load. Even though your little plastic router may not be able to cope with the maximum speed on your line, is that actually what you really need? Perhaps setting around half of your speed as the maximum might give you better overall throughput. I can tell you that if QOS is turned off in any of my installations (apartment blocks with hundreds of simultaneous users) then within seconds it all becomes totally unusable - no downloads are possible, no mail, websites don't open, youtube stops working, and so on.

    One should ask - which would actually be better ... a 100mbps download speed which is totally unusable, due to one machine running uTorrent to download 50 HD movies and a few ISO's of LINUX ... or around 50mbps that is working properly and well controlled, with snappy http response and orderly use of bandwidth? International gateway speeds are such that most likely you won't notice anyway.

    At the moment, the faster ARM based consumer grade routers that we need for these higher speeds can't run proper QOS due to the fact that essential parts of the firmware (IMQ) aren't working properly in this build of Linux. Shibby and Victek are doing a great job so far, and I'm sure that eventually everything will get addressed. It just takes time and effort... be patient!

    If there is anyone out there that can assist with getting IMQ to work, we'd welcome your help!

    There have been many posts where people make the statement "I have 100Mbps and at these speeds, we don't need QOS." Why not? If you have a pc on your network that is e.g. downloading torrents with uTorrent and no download limit, it is just as likely to take all of your bandwidth as when you had a lower speed. If you needed QOS before, most likely you will still need it.
  11. Touche

    Touche Reformed Router Member

  12. Porter

    Porter LI Guru Member

    I just came across a rather curious use case: I've been watching Google I/O live and youtube sends the stream over port 443 BUT it's UDP! So this traffic won't be matched by any filter and therefore ends up in the default class, probably giving you bad performance. For this reason I'm thinking about modfying the two HTTP filters from TCP only to TCP/UDP. I've done this for a test and it works. Right now I'n not entirely sure if there might be adverse effects. So keep this in mind if you try this.
  13. HunterZ

    HunterZ Network Guru Member

    Several of the random well-known port lists I'm looking at via random Google searches mention that both TCP and UDP 443 are used for HTTPS (aka HTTP over TLS/SSL).
  14. Porter

    Porter LI Guru Member

    Oh, I didn't know that! All the more reason to incorporate this into the default QoS rules. What do you think, @Toastman ?
  15. Toastman

    Toastman Super Moderator Staff Member Member

    Yes, agreed. I just woke up and blinked at this, will look later ! Modding the L7 rules seems to be a good idea.

    Added a rule on one large condo to check if anyone is using UDP on port 443 - so far, not seen any.
  16. Toastman

    Toastman Super Moderator Staff Member Member

    Still not seen a single UDP port 443

    But I do see several downloads hitting the L7 Flash filter, granted, not many though.
  17. Porter

    Porter LI Guru Member

    Seems to be pretty rare. Hadn't seen this before watching the youtube stream.

    Which L7-filters did you want to change? If any, I would recommend scrapping the flash-filter because that doesn't match a lot of traffic (at least in my network) and it's one of the slowest filters.
  18. skoub

    skoub Network Guru Member

    hi everyone

    i have a problem with the QOS on my router. When i connect to my job via vpn (Cisco VPN Client), i only get ~45kb/s in transfert rate and when i take my remote desktop, it's really slow. I disabled the QOS on my router and then i get 1000kb/sec and the remote desktop is really fast.

    So what should i change in my qos to get a good speed when i connect to my job?

    I'm using Tomato Shibby Firmware 1.28.0000 MIPSR2-116 K26AC USB AIO-64K and i haven't changed the default QOS settings.

    thank you for the help!
  19. Porter

    Porter LI Guru Member

    Seems like your traffic ends up in the default class and starves there.

    You might need to add a filter mentioning these ports for UDP:

    As class choose Remote. Although I'm not sure how much bandwidth you would get there. Since this is you work, you might want to give it 100% upstream and 90% downstream.

    Add this filter and put it somewhere higher up at rule #10 or something.
  20. skoub

    skoub Network Guru Member

    great! thank you very much for the help. I'm now at full speed :)
  21. Porter

    Porter LI Guru Member

    Glad I could help! ;)
  22. Derbo

    Derbo Serious Server Member

    I recently updated my Chrome to 37 Beta on OSX and I noticed my youtube videos were getting slower. I looked at the graphs and noticed the HTML5 player is using UDP 443. I believe there is no rules for this UDP 443, so the traffic is set to crawl. Youtube is using https as well. Does the latest firmware have any updates related to this particular scenario?

    I am using :

    Dark Knight
    Tomato Firmware v1.28.0505 MIPSR2Toastman-RT-N K26 USB VLAN-VPN
  23. koitsu

    koitsu Network Guru Member

    Just some information regarding the whole UDP port 443 thing:

    HTTP is a TCP-based protocol. I have never in my entire life seen someone implement HTTP over UDP. Ever. Port number has no bearing on that.

    UDP is not used for HTTPS as we know it today (web browsers/transport protocol). I cannot stress this enough. There are many others who say the same thing. (And I'm not going to get into a discussion about SCTP)

    I do see that something called "QUIC" is involved here, and that is an unofficial (i.e. not IANA-approved) port assignment, used solely by the Chromium browser:

    And as far as I remember, QUIC is essentially pointless as Google has advocated use of SPDY instead (and that is simply protocol changes, going across HTTPS normally -- e.g. TCP port 443, or a custom TCP port if someone chooses to run an HTTPS server on a different port).

    As for official IANA assignments (a.k.a. /etc/services on UNIX machines): IANA has always, historically, assigned both protocols to a port when it's registered with them, even if it's not true/valid/correct. That's just how they've done it for the past 30 years. Proof is here and here and below (from my FreeBSD box):

    https           443/sctp
    https           443/tcp
    https           443/udp
    So until someone shows me an actual packet capture on their WAN interface involving UDP port 443, including all payload (ex. tcpdump -p -i vlan2 -l -n -w -s 0 /tmp/capture.pcap "udp and port 443") so it can be investigated, I'm declaring shenanigans. If anything, re: QoS rules, a separate rule for UDP port 443 (with a description of "QUIC for Chromium") could be added, but I've never in my entire life seen this used.
    Toastman likes this.
  24. Porter

    Porter LI Guru Member

  25. Derbo

    Derbo Serious Server Member

    Classifying the HTTP filters to use TCP/UDP instead of TCP fixed the bandwidth problem. Youtube videos aren't being captured to be the media L7 youtube2012 filter though. It is just classified as File Xfers. Does this happen to you Porter? Thanks for the help @Porter.
  26. Porter

    Porter LI Guru Member

    The youtube-filter only works on unencrypted traffic. Nowadays youtube is mostly encryptet which makes this youtube-filter more and more useless.
    HitheLightz likes this.
  27. Derbo

    Derbo Serious Server Member

    Thanks for the info @Porter. I'm glad its okay as far as bandwidth goes.
  28. Touche

    Touche Reformed Router Member

    Does "No Ingress QOS for UDP" have any effect on online gaming latency?

    Also, since I'd upgraded my ADSL to VDSL connection, I've found that QoS actually has a negative effect on performance.

    I'm running:

    VDSL 60/30 Mbps
    Asus RT-N66U
    Tomato Firmware 1.28.0000 MIPSR2-114 K26 USB VPN-64K

    I've disabled all the L7 rules in QoS and use only port based rules. The rest of the settings are similar to the ones I've posted several posts up. When I max out my download with torrent traffic, router CPU goes to >1.5 usage and my pings increase. If I disable QoS, CPU usage is <0.3, my pings barely increase and web pages and Youtube still load without visible slowdowns. I guess I don't need QoS at all now, but I thought that the N66U was powerful enough for 60/30. I'm not able to test what would happen if my upload got taxed, so I was still hoping to be able to use QoS without issues.
    Last edited: Sep 15, 2014
  29. cloneman

    cloneman LI Guru Member

    I've had weird issues where my CPU usage is much higher than it should be while moving packets (0.5 load for 25mbit). Starting fresh resolved the issue for me. I would advise clearing NVRAM and trying a different firmware (latest Shibby perhaps). Although, In my situation, the cpu usage was high regardless of QoS being on.

    It's not impossible 60mbps is too much for your router to handle with QoS (if you torrent via WiFi, for example). Although I've been told this router can handle it, my sirq% in top goes pretty high on my measly 30mbit connection.

    If you run out options, you can grab Vicetek (Tomato RAF?) I believe he still has the upload-only QoS which would be less demanding.

    I would also uncheck all those small packet priority crap, probably does more harm than good.
  30. Touche

    Touche Reformed Router Member

    I'm currently on Shibby v114. I'll try playing with some options and maybe flash a newer version and start fresh, but I don't believe it will help as I could see the load increase as download speed was increasing. It wont be too bad to loose QoS, though, as it seems it isn't needed with these speeds.
  31. JustinChase

    JustinChase Networkin' Nut Member

    I've started having some issues where, after some time of downloading torrents, my network comes to a crawl, and sometimes I can't even connect to the router. I'm downloading the files on a server, which is connected via Ethernet to the router, and am trying to connect via a laptop on wifi, where I'm experiencing slowdowns/failures. Often, if I disconnect the server/ethernet cable, I regain useful service on my laptop, but this morning, that didn't even help. I had to manually reboot the router with the switch, since I couldn't access the router GUI. I was connected, but my connection was so slow, it was useless.

    I'm using an e3000 router, with Tomato Firmware v1.28.7506 MIPSR2Toastman-RT K26 USB VLAN-VPN

    I'm using the stock QoS rules, with some tweaks to speeds/percentages, but I have not changed any rules/criteria.

    My internet is 60 Mbps, which I understand can put strain on the router, having to manage so much more incoming data.

    I've read a lot, and sort of understand the thinking behind the QoS rules, and am somewhat familiar with Linux and command line, but I'm not knowledgeable enough about routers, TCP/UDP, ports, classes and such to completely follow all the discussions.

    I want to use QoS so that I can have torrents and/or nzb files downloaed on my server at their 'max' speed when no one is using the internet, but want them to self-manage to slower speeds as 'more important' traffic uses the bandwidth.

    So, I need some help/guidance to get my system working better, but I'm really not sure where to go from here. I see there is a newer version of Toastman's firmware, but I'm not sure it's going to change much, as I suspect it's either my router, or QoS rules causing me grief.

    Anyway, thanks in advance for any help to get my network under control!
  32. cloneman

    cloneman LI Guru Member

    Basically I'd suggest SSH'ing into the router and running top to see if your CPU is being maxed out while you're running torrents. Most likely, it is, especially as you describe the web interface of the router becoming slow. Another test, you can set your max download to 20mbps, this will cause your router to severely throttle you, and your symptoms should disappear.

    Of course this is not an ideal solution. You can try to disable inbound QoS completely by setting very high limits for download (no limit or 300mbps or something). You can try to delete all the rules and start over. You can try to switch to RAF/Vicetek which does not support inbound QoS properly afaik and therefore should be faster. You can upgrade to an ARM router which has more CPU (but also incomplete inbound QoS as I've been told)
  33. Nick G Rhodes

    Nick G Rhodes Addicted to LI Member

    I doubt your E3000 is fast enough.
    I have an E4200 running Tomato-Toastman and with QoS it can only manage 40Mbit/s single ftp with a maxed out CPU (SIRQ at 99%) and that does not leave any spare CPU for Wifi and other duties.
    To tame your torrents you need to find the maximum speed with the minimum number of connections that won't put a heavy load on your router. For example on my 30Mbit/s connection I only need 150 connections in my torrent client to max out my connection. I would suggest trying this as a base setting before trying to bump the max torrent speed or connections up.

    Cheers, Nick
  34. JustinChase

    JustinChase Networkin' Nut Member

    I have a new ASUS RT-N66W Dual-Band Wireless-N900 Gigabit Router sitting in a box here in the house. I bought it for another reason, which didn't work out, and was planning on selling it, but I could use it now if it would resolve this issue.

    Is it worth opening and using this new router, or will I likely suffer the same issues?

    Is there any other change I can make to the QoS rules to get this under control? I saw mention of a couple of rules that have little effect, but use lots of CPU, but am not sure it's worth pursuing at this time.
  35. Marcel Tunks

    Marcel Tunks Networkin' Nut Member

    Faster CPU, better WiFi performance, and difficult to brick. Definitely keep the N66 and use it as your new gateway router. The faster CPU will be particularly helpful for this issue. No need to make any specific rule changes, just the hardware change. If the CPU is still maxed out (unlikely at that speed) then consider disabling any other features that aren't in use (ethernet port states, UPnP, ip traffic monitoring, USB, etc...)

    Use your old router as a wireless access point, backup device, toy, whatever.

    Sent from my HTC One_M8 using Tapatalk
  36. cloneman

    cloneman LI Guru Member

    I've been using tomato QoS for VoIP for some time now. Even though my only VoIP rule is "UDP port 5060", it somehow magically detects the RTP traffic as well and attaches it to my voip class; it just works, I never had to do it myself. How does it "know" that the RTP traffic is related to that connection even though it runs on a completely random port?
  37. mw333

    mw333 Networkin' Nut Member

    Sometimes the L7:rtp class function works magically. :)
  38. cloneman

    cloneman LI Guru Member

    Trouble is, I deleted all the default rules, and I have no L7 Rules, and it still works. In fact, I just moved my port 5060 rule to a different class as a test, and the RTP traffic followed.

    It's magic that the RTP traffic gets thrown in the same class as my port TCP/UDP 5060 rule :)
  39. mw333

    mw333 Networkin' Nut Member

    Perhaps you have another rule, such as by mac address or ip?
  40. mw333

    mw333 Networkin' Nut Member

    Another thought, have you looked at your firewall rules? Something like iptables -nL will give you a listing of the chains. There should be one labeled L7in. It may not be empty. Please find below a section of mine:

    Chain L7in (1 references)
    pkts bytes target prot opt in out source destination

    2443 492K RETURN all -- any any anywhere anywhere LAYER7 l7proto rtp
  41. jbeightynine

    jbeightynine Reformed Router Member

    Is it possible to configure the in/out class limits' % using the scheduler

    For e.g. configuring:
    1_Service to 5% - 30% inbound.
    or 4_WWW to 20% - 70% outbound

    My connection varies from 0.35 - 2.00 Mbps depending on the time of day and some scheduled tweaking to these values would be useful.
  42. Porter

    Porter LI Guru Member

    This should be possible. The values for inbound and outbound bandwidth in the nvram are being set in qos_ibw and qos_obw. You would just have to find out how the correct nvram commands are to manipulate those values and how to restart the QoS system with these values. Maybe you should first do a search in this forum because there might have been users who already did this.
  43. jbeightynine

    jbeightynine Reformed Router Member

    Got a little help and thought I'd share it. This is how its done at least on Shibby:

    #Enable (preconfigured) QoS
    nvram set qos_enable=1
    service qos start
    #Reconfigure Incoming Bandwidth to 512kbps
    nvram set qos_ibw=512
    service qos restart
    #Reconfigure Outgoing Bandwidth to 128kbps
    nvram set qos_obw=128
    service qos restart
    #Disable QoS
    nvram set qos_enable=0
    service qos stop
    #Change Inbound Limits for QoS <Class 4> to "20% - 70%"
    set qos_irates=10-20,5-60,20-60,20-70,5-0,5-0,5-60,5-0,5-60,1-19
    #Change Outbound Limits for QoS <Class 9> to "4% - 30%"
    set qos_orates=5-90,5-30,5-30,5-70,20-100,5-70,5-30,5-30,4-30,1-30
    I tried doing the same thing for bw limiter but keep getting a syntax error. Found this line with "nvram find qos" so I tried:
    nvram set new_qoslimit_rules=XX:XX:XX:XX:XX:XX<80<160<128<160<4<40<5>YY:YY:YY:YY:YY:YY<160<272<128<160<3<50<10
    This is the error:
    /tmp/.wxtpkAzh: line 5: syntax error: unexpected redirection
    Its obviously the wrong way to do it. No help from search apart from this.
  44. Porter

    Porter LI Guru Member

    Please keep in mind that you cannot use QoS and B/W-Limiter at the same time!
  45. jbeightynine

    jbeightynine Reformed Router Member

    I've also read that it can "break" qos but with testing, both qos and bwl seem to be working as they should.

    Specifically, qos is able to distinguish between different classes run from the same pc, while bwl keeps it's speed pegged just under the specified 0.39Mbps (it was 0.37). Both web and file transfer classes were set to 100% upper limit from a 2.00/0.25 Mbps max in/out limit for the test. This was on Shibby v128 build 123 for an RT-N66U.

    Any suggestions for the system commands to configure bwl rules?
  46. Porter

    Porter LI Guru Member

    It's not a myth that they don't work together, it's a fact. Even if you can't see anything going wrong, it's not working. QoS and B/W Limiter don't know of each other and therefore nobody can predict how your connection will react. Decide which system you want to use and then configure the scheduler accordingly. I don't know the variables for B/W Limiter. You'd have to search with "nvram show".
  47. jbeightynine

    jbeightynine Reformed Router Member

    Figured it out. It was only missing quotes:
    nvram set new_qoslimit_rules='XX:XX:XX:XX:XX:XX<80<160<128<160<4<40<5>YY:YY:YY:YY:YY:YY<160<272<128<160<3<50<10'
    Also thanks @Porter I wouldn't have realized something was broken if you hadn't insisted.

    Basically when both bwl and qos are enabled, a bwl set for 1 device in br0 will be applied across all devices in br0 despite greater qos class limits. lower qos class limits will still be enforced.

    Guess I'll stick to qos and figure out iptables for tcp/udp limits.
    HitheLightz likes this.
  48. mvsgeek

    mvsgeek LI Guru Member

    Is there any way to classify traffic based on a partial URL? For example, anything containing 'aaplimg'?
  49. theboyk

    theboyk Addicted to LI Member

    Just wanted to say a massive thanks to Toastman, and everyone else involved in this thread, for putting together this wealth of information! I'd never been all that comfortable with the QoS in Tomato (been running Tomato for years, but only played with the QoS system in it here & there), but over the last couple weeks, our 100Mbps line (up/down) was getting hammered and becoming totally unusable at times because of saturation. So, I spent a day+ reading all the posts in this thread (and then reading them again), and after some short tests, I put the new router (running Toastman) into production and after I enabled the QoS system (using the default as the basis and then adding my own required ports/rules/etc), our connection has never been so stable! It's only been a day thus far, but so far so good! Anyways, again—massive thanks to all those involved in this post (and in the QoS implementation in Tomato, and in Tomato as a whole)!
    Last edited: Nov 19, 2014
  50. careh

    careh Addicted to LI Member

    What type of router did you put in? What are the download speedtest speeds are you seeing with QOS turned on?
  51. theboyk

    theboyk Addicted to LI Member

    It's running on one of my RT-N66Us. Originally, I was going to put it on a RT-AC66U, but I wanted to run the Toastman build, so ended up on the N66U.

    As for a speedtest...well, that's a bit tricky to answer 'cause once QoS is on, speeds are affected by QoS itself. But, before things really got going this morning (there was some network traffic going on during these tests, so they're not 100% raw), I ran the following tests:

    Test #1
    - QoS disabled.
    - Download = ~90000 kbit/s
    - Upload = ~80000 kbit/s
    - Ping = 1 ms

    Test #2
    - QoS enabled, but limits on the FileXfer class (where most of the speedtest data ends up) ramped up to 70/90, so as to allow a pretty wide pipe for the test. As well, I ramped up the bandwidth limits to 85000 kbit/s on the download and 75000 kbit/s on the download, so as to use as much available bandwidth as possible.
    - Download = ~80000 kbit/s
    - Upload = ~70000 kbit/s
    - Ping = 3ms

    Test #3
    - QoS enabled, FileXer class limits returned to the original settings (inbound set to 5/60; outbound set to 5/70), bandwidth limits reset to 70000 kbit/s on the download and 55000 kbit/s on the upload.
    - Download = ~40000 kbit/s (theoretical max would be 42000 kbit/s)
    - Upload = ~35000 kbit/s (theoretical max would be 38500 kbit/s)
    - Ping = 3ms

    By the time I got to test #3, it was after 9am and the studio/network was buzzing, so I wasn't the only one using bandwidth.

    During test #1, even surfing the web, loading a basic website (, and, was delayed (for up to 5 seconds or so). But, during test(s) for #3, I was able to surf the web, stream an HD video on YouTube, etc., without any noticeable delay.

    So, while I've only been running QoS on this router for three days now, I can say it's definitely been a positive experience and has definitely resulted in a more stable network. While "max" upload/download speeds are "down" (in that a file transfer no longer uses the full bandwidth), the overall experience for all users is much, much better! I'd put off implementing QoS for way too long (I never felt comfortable with my own understanding of the Tomato QoS system), but after coming across this post, I felt a heck of a lot more confident enabling/configuring it, and man, I'm so glad I did! I've been glued to the graphs for the last three days, adding specific ports/rules as needed for our specific requirements, tweaking things as I go and I couldn't be happier with the decision to finally go ahead with QoS in our environment. So, again, a massive thanks to all involved!

    Last edited: Nov 21, 2014
    Toastman likes this.
  52. Monk E. Boy

    Monk E. Boy Network Guru Member

    Since I haven't seen this mentioned in this thread (I may have missed it though), one way to band-aid load issues is to simply lower your QoS bandwidth values to ones your router can handle. It will "ignore" the extra bandwidth on the connection, but at least your router won't be immolating itself, yielding a stable, faster connection than if you try to force at higher speeds. While it's not ideal, I would certainly run this way for a few days while, say, waiting for a new router to show up.

    Also, I was rather surprised to find yesterday that an RT-N16 can handle over 45Mb down and 8Mb up with QoS enabled. Granted I have most of the other services, even wireless, disabled so there's more CPU free than normal. But still it's amazing to me that it's capable of shoving data around without issue at these speeds. The load is near or at peak, but it's not hiccuping and falling over.
  53. phlibby

    phlibby Networkin' Nut Member

    Since I can not delete this post, I will use it to explain what I am trying to do. I am one of the House Managers (and officially appointed the "IT Guy" by my boss) of an apartment building that serves as temporary housing for people just getting out of prison or jail and they need a hand up to get back on their feet. There are 7 apartments (3 across with 3 on top of that and then one at the top level) with a maximum of 5 people in each one. All they need to be doing is job search, e-mails - just normal internet stuff. I have a main Comcast gateway router in the middle. I leave that as close to default as possible @ I come out of port 1 to the WAN port of a Linksys E2000 with Tomato @ This Segment I call PrivateNet. It is the one the House Managers share. There are 6 of us. Then, out of port 2, I go right 30ft to the WAN port of a Linksys E900 Tomato router segment ( I call ClientNetRight. Then, out of port 3, left 30ft to another E900 WAN port to make the last segment @ I call ClientNetLeft. Those are the 3 segments or subnets. The main gateway router and the PrivateNet router are close together. I do not use the wifi in the main gateway. I also have another wire running to each E900 from a makeshift patch-panel near the Gateway so I can log into each one. Just want to say a little about my set-up before I go back and come through the pages one more time. There is still a lot I do not understand. The QoS seems to be working fine. The 3 Tomato routers have Shibby Tomato Firmware 1.28.0000 MIPSR2-123 K26 Max and the rest, you know.
    Last edited: Dec 7, 2014
  54. phlibby

    phlibby Networkin' Nut Member

    Since I can not delete this post, I will explain my situation so the questions I ask might make more sense. I have an Arris TC8305C supplied by Comcast. It has 4 physical network ports. I am coming out of each of those(well using 3 at the moment) to Wireless N routers. 3 network segments. One on, one on and one on When I set up the Qos, I set the total bandwidth - 15% divided by 3 (or 4 if I used all 4). It all seems to run good. No complaints.
    Last edited: Dec 7, 2014
  55. phlibby

    phlibby Networkin' Nut Member

    It will not let me completely delete my post I had here. So, I guess I'll just put my question back in here. On my Classifications page I have 47 default rules. Not a one of these refers to a p2p/bulk class yet on my Basic Settings page my default class is p2p/bulk. Is this supposed to be this way? Or is something wrong. I added all my rules via System Command, copy, paste & execute per Toestman's latest Rule Settings. I have some special settings not related to QoS so I would rather not do a reset to default if I don't have to.
    Update: I just looked at every screenshot I can find showing the Classifications page and I do not see a rule for p2p/bulk. Now that I know it is not supposed to be there can I ask why? Is it because anything making it through the rules without being classified falls into this class? How can it if it isn't defined? Or is it inherently there but not there? Kind of like the Clipboard in Windows? If no one answers I will take that as a yes.
    Last edited: Dec 7, 2014
  56. phlibby

    phlibby Networkin' Nut Member

    Or here either, so I've decided not to ask stupid questions and waste any ones valuable time until I can get a good grasp on all this. It's a lot to assimilate.
    12/7/14 - I read through the 12 pages again and now feel I have a little more of a grasp on the concepts. My biggest question I am most concerned with at the moment is - does the main router HAVE to have Tomato? Can the first/main router be the ISP's router? My Comcast router has 4 physical ports. I use each one of these ports as a network segment (, so on). I don't want any chance that one segment can see another segment or get into the others files. I don't care if each device on a segment gets into the files on a device on that segment. The ISP router is left on default at Each of my segments is governed by a Tomato Router w/QoS and all seems to run good. I'm just wondering if there is anything wrong with this setup (besides the double nat I already know about that). Thank you Porter for answering me when no one else did.
    Last edited: Dec 7, 2014
  57. Porter

    Porter LI Guru Member

    As far as I remember you asked about the missing default class. I don't have an explanation for that, other than that you might have a Tomato version that doesn't have it in the default settings or your config somehow got screwed up.

    Either way, to use QoS, please flash a recent Toastman or Shibby firmware and just use the default settings for QoS. Then you will have all the classes and all the filters.
  58. Toastman

    Toastman Super Moderator Staff Member Member

    The default CLASS is set to P2P. I think, you actually DO have a default class. The misunderstanding is .... we do not have any rules to classify P2P, because essentially there is no rule that can classify P2P properly. That is probably the most important thing to understand about Tomato's QOS. We address every other service that we want to use, and everything else (including P2P and any other stuff that we may not even have anticipated on our network) drops past the rules and into the DEFAULT class. And that has been proven to be the ONLY effective way to handle P2P.
  59. phlibby

    phlibby Networkin' Nut Member

    Thank you Toastman for your reply. I actually feel like I've just talked to a Movie Star - kind of thing. This little group you've got; You, Victek, Shibby, Porter, Monk E. Boy and the lot are doing a spectacular job! I am currently running E-2000 and E-900 routers with the latest Shibby build. I have all 47 of your rule sets and I can understand almost half of them - ha ha. I've read through this thread twice now and going for a third time. Networking is my weakest point. I have a degree in electronics and I have been made "The IT Guy" by my bosses at the non-profit company I work for simply because they think I know what I'm doing and I work for almost no money. So, I am just learning this stuff. I seriously consider you as a QoS God! I can't even believe anyone could have as much patience as you do. I believe, after all these years of having QoS off in the background, you and your group are the ones that are going to finally make it work the way the idea was intended. So far, my network works great. Ever since I implemented your firmware and QoS system, my phone has stopped ringing. I have not had a single problem yet. Although, the only things I have seen so far out of the ordinary are YouTube, Netflix and FB Games. I am learning how to read the Pie-charts and Details so I can see what is going on and I am taking my laptop around to different locations inside the building while running WiFiInspector to test the quality and range of my AP's. So far so good. My hat is off to you guys. I can't wait to do the same thing over in our office building.

    Tomato 101 Student

    A Huge Thanks.

  60. Toastman

    Toastman Super Moderator Staff Member Member

    We have much in common. Several years ago I took over admin of a large apt. block internet system because it was completely unusable. We have so many people hiding in their rooms watching movies, downloading "stuff", and general surfing, mail etc. that any one of them could have been responsible for bringing the whole thing to a grinding halt. Tomato's QOS made it possible for over 200 people to share the same ISP link and not even be aware that they were doing so, by enforcing rules on everyone. Nowadays the main challenge is to give better connectivity by adding more or stronger access points, with the limited number of channels available, and several dozen interfering wifi points right next to us.
  61. phlibby

    phlibby Networkin' Nut Member

    Thank you for making me feel welcome here. I just made some major changes to my network I am building here but before I go into that, I would like to explain why I ended up here. My son is a Web Developer and he chose Tomato for us a long time ago. He has gone onto the bigger and better (and a lot more costly equipment) but I stayed with the SOHO's and Tomato. That was when I became the reluctant "IT Guy" for BTC. I had already read somewhere that you and your group were developing and implementing a QoS system that really works. Then came an opportunity to test it. Out of the blue, my phone starts ringing "the internet is out" and upon troubleshooting I discover, not one, but two of my E2000 routers have gone out simultaneously! I tested them without putting them on the network and they seemed to be fine, but as soon as I put them back in the network they started failing again. I had never heard of the connection storms up to this point so I was just baffled. So I order up some E900's to replace them (because I was short on cash and they were on sale). So, I made it on through, with numerous power cycling and resets. Time went by and, after a week or so goes by, I was thinking of dissecting the two E2000's to see if there was any visual damage. That is when I ran into this forum and your Tutorial & Discussion and started reading. When I came to the topic about connection storms. All the sudden the light came on in my brain and I saw clearly what had happened. I had no QoS system turned on yet and someone on the network was running torrents or something similar and probably causing the routers to crash. I still don't know what it was for sure, but ever since I flashed them up with Shibby's latest build and put your latest rule sets and prioritized classes into action, all complaining from my clients stopped. My phone was ringing at least 3-4 times a week. Now it is completely silent. Sorry for rambling on but I just want to make it clear that my situation is proof this QoS works and I am going to implement it at our office and in all our houses. We run 17 houses in 3 Counties here in Portland, OR. This just solves so many problems for us, I am going to learn this and never buy another router I can't put Tomato on. This is kind of long and I was going to explain the recent changes to my network but you have already posted advice to me in this thread:

    Anyone struggling with a similar setup as mine might want to check it out and follow along watching Monk E Boy, Porter and now Toastman guide me along as I plan to expand my network the right way.

    Thank you so much Toastman for EVERYTHING!

    Tomato 101 Student

    A Huge Thanks.

  62. Derbo

    Derbo Serious Server Member


    Is ip6tables included in QoS in the current builds?

    I wanted to run ipv6 and qos but it looks like in a previous reply in this thead (page 10) that iptables are only for ipv4.
  63. Moogle Stiltzkin

    Moogle Stiltzkin LI Guru Member

    actually toastman i was wondering the same thing.

    unlike you, my network situation is only for home usage, and i'm the only heavy user mostly torrenting, twitch streaming, or even just plain http download using internet download manager. The other users on my network fewer than 5 mostly only use casual web browsing e.g. facebook lelz....

    so does it make sense for me to use qos ? if not to limit my torrent download/upload speed, but at very least using the packet prioritization, would it make any difference ? or none at all ?

    I'm using a similar setting like touche

    tested torrent and this is my result for 10mbps ftth (in speedtest it says i get something like 11000-13000 kbps in dl/ul) but in tomato i set to 12000

    and this is when downloading multiple torrents at same time

    also here is the qos chart when using both web browser and heavy torrent at same time

    regular web browsing seems to have stopped to a crawl. loads sure but very slowly. Any ideas how to fix casual web browsing while heavy torrent usage ? I don't mind dropping down download speed by say 1mbps at most (preferably less if able), but i would like the download speed for torrent to go back to 100% when web browsing is not using the bandwidth needed to operate smoothly. Is such a configuration possible ?
    Last edited: Feb 10, 2015
  64. Toastman

    Toastman Super Moderator Staff Member Member

    I don't have time to reply in full to your message - but I would encourage you to read the qos thread and try to follow/understand it. The absolute first thing that it teaches you, is to measure the ISP up/down speed using speedtest and then set the MINIMUM reading you get less 10% at least.

    You mention "in speedtest it says i get something like 11,000-13,000 kbps in dl/ul) but in tomato i set to 12,000"

    The minimum speed you measured is 11,000 so 11,000 less 10% = 9,900, so why did you set 12,000? You know, if you do not follow the recommendations then it simply will not work.

    The best way to find out if you really need QOS is to simply turn it off and see what happens. If your and your peers web browsing becomes too slow, then you can benefit. Nobody here can predict what may or may not happen in your specific system. In general, you will always find smoother and more consistent operation with QOS running and correctly configured, but you will suffer a loss in your maximum throughput exchange for this stability. This isn't negotiable!
    HitheLightz and phlibby like this.
  65. Moogle Stiltzkin

    Moogle Stiltzkin LI Guru Member

    because despite speedtest results i note in torrent without qos it can reach those rates (at max i've seen 13k download, 14k upload) roughly. don't really understand the variance between torrent and speedtest values, so i set as 12k just to be safe.

    though i'm 10mbps officially, they did over cap me a bit to ensure a 10mbps consistency that is why.

    I was testing how torrent download speed would perform when i tweaked 12k to lower and higher by 1k increments to see which would net me the highest download speed.

    qos on shibby firmware didn't work as it caused boot loop for me. i've already reported to shibby the issue here

    so until shibby can fix that i leave it off for now :X also managed to figure out my other issue regards ipv6 mtu.

    this in script wan up fixes it
    sleep 1
    echo 1492 > /proc/sys/net/ipv6/conf/br0/mtu
    service dnsmasq restart

    still got to wait 10-15minutes before i get a wan ipv6 address but at least i don't have to manually edit ipv6 mtu anymore since the script automates it for me on boot ups.
  66. Mr.CTT

    Mr.CTT Serious Server Member

    Personally, I don't like many different speed tests and find their ups and downs to be inaccurate using window's Resource Monitor/ another utility i have. The most accurate speed test i have found is you can even control the size of the file you download(to improve accuracy), and it tells you what your speed was at every % of the way. you never mention how you are testing your speed so i thought i should bring that up.

    QoS is something I see as unnecessary for 90% of people who control their own network. Why you ask?

    -you know the speed you pay for and have done the tests so you know your normal bandwidth.
    -You can see who is torrenting and using bandwidth real time in tomato, and limit their speed.
    -You can block access to unwanted websites/servers/IPs

    The main use for QoS is for VoIP (or similar) in my opinion. This is the only reason you would really want to enable something that buffers ALL the traffic in and out of your network. QoS although minimal does create latency(very very tiny), and you do not gain over all speed from it, however you do gain more bandwidth for certain applications should something had been pulling high amounts of bandwidth and caused lag to the item you were concerned with.

    With QoS if done per the guide, you limit your bandwidth to less than 100% it's full potential, the only thing you are protecting yourself from(beyond the above mentioned stuff) is dropped packets/lost info that happens when your throughput gets into the burst-ed levels to and from your house which are higher than your sustained level. Most ISP have a buffer for peoples traffic so that there is less of information while negotiating speeds(this is a little inaccurate but the most simple way to put it without writing a book), your router also has this buffer. when you limit your outgoing and incoming speeds so that it does not max out your connection with QoS, you protect yourself from the losses that happen and prioritize the traffic so that the higher priority stuff is even less likely to have data loss should your connection be maxed out, and it is the first to be transmitted in all cases.

    If you have a Very congested network, or one that relies heavily on VoIP or similar running all the time or needs to be available all the time (such as a buisness for example), then QoS is a decent idea if you fully read the guides and understand what your doing(by this i mean you actually log all your traffic for like a week or more and see what is what and how much each uses then give a good classification to each based on your needs). If not, then IMHO it isn't really that useful. There are benefits to it, don't get me wrong, but nothing that outweighs it's cons as far as I am concerned in 90% of all situations since the history of time. Most people think QoS is some magical feature that makes everything better, by clicking a box, your pings magically drop to 1ms, you always have maximum speed, your router is smoother, you "killz deh pplz with your L33T skillz brah" when playing online games, when in reality unless you program it properly and do your research, it slows you down.

    I may be biased because I don't like to limit or buffer anything so take this with a grain of salt. But when you set %s for stuff, you are limited to that % for that item, I'm a get shiii-stuff done kinda person. The bandwidth limiter is a lot more useful for me, because when i need to control speeds of specific devices (like my PS3 to attempt to ensure there is never buffer bloat and to limit my speeds so i don't play against people on fiber and get killed because they have way better sustained speeds and lower pings) I have a way to do it without limiting everything so strictly (sort of).
    Last edited: Feb 10, 2015
  67. Mr.CTT

    Mr.CTT Serious Server Member

    I apologize in advanced for double posting but i feel it makes it easier.

    Here is how I Rationalize QoS

    This image is without QoS. There is no priority, just Raw bandwidth, and an amazing consistency I might add.


    You can see my connection Average is ~56mb/s, My max is 57.07mb/s during this 40 second interval the test took place.
    This image is what QoS does with that bandwidth.
    Each color represents one of the classes you set. Now instead of torrents/downloads/stuff having the ability to use all 56mb/s, you LIMIT your MAXIMUM possible bandwidth of ALL your devices on ALL of your network in it's entirety to that solid Red line Per the guide, because you don't always get 56mb/s sometimes it is 50 or 45 or 52 or 62.3 or etc. so you reduce 25-35% off your max or as @Toastman says 10%+ off your minimum to ensure you never go above what your given. (be careful, the end of the speed test is misleading because some downloads slow down as they near completion and do not reflect your bandwidth as shown in mine.

    Each dashed line with a color represents a class you set inside that 80-90% and its maximum potential usage speed. The class is limited to the amount of bandwidth that is made up of it's height (only it's color's height) and shares that for all devices that transmit data in that class, you create this when you set %s for a category.

    Good practice is to set your totals for your upper % to as close to 100% when added together as possible, Best practice is to keep that added up upper to 100% to ensure smoothest possible operations, however that wastes bandwidth, but going above 100% can lead to issues later discussed and lag. It is essential to keep your low end %s to total to less than 100% when added up so they have a default speed to drop back to should your router need to reduce speed to compensate for another class speeding up. *note* I am a little fuzzy if it can go beyond the upper % you set if there is unused extra bandwidth because one or many things aren't using all of their bandwidth, but i would assume it cannot go much beyond that % because should another class start or require more or all of it's bandwidth, speeds would need to be re-negotiated, and you can experience lag and possibly may have data loss.*/note*

    The Results are now instead of being able to download torrents at 56mb/s you set it to 10% of the number that is 90% your total untouched speed and you get ~5mb/s at max speed for that class

    You have to give up something to get something, everything comes at a price with QoS.

    -If you set a class too big, you waste useable bandwidth.
    -If you set it too small, you lag.
    -You always waste your bandwidth that is available to you over the number you set it to.

    -Your network is super smooth, but never uses all it's available bandwidth that you pay for.
    -VoIP is perfect all the time
    -Streaming always works and never times out
    -Everything always works.

    Please correct me if I'm wrong on stuff people who know more than I!
    Last edited: Feb 10, 2015
  68. lancethepants

    lancethepants Network Guru Member

    Back when qos was essential for me, I could play games, and torrent, or whatever else would normally (near)max out my connections, all fine.

    The thing to understand, is that you have to follow Toastman's guide to the T. You also have to understand that running effective qos means you will never actually 'max' out your connections. You absolutely have to set your limits lower than your lowest measurement, as Toastman states. Qos absolutely needs to have some wiggle room and overhead to work with, otherwise it will completely and utterly fail.

    While you may not squeeze your connection of every last bit, qos ensures that your essential services are left un-interrupted, voip, web surfing, and gaming, which is far better than nobody being able to do anything, except for your torrent leeches. My gaming pings didn't even suffer, or at least not more than a few, but no more than 5, and that's only if my connection was pushed to its limit.

    Mr.CTT beat me to some of this in his last post, but re-interates much of this.
    Holy_Hunter likes this.
  69. Mr.CTT

    Mr.CTT Serious Server Member

    Haha yea, i was a little on the thorough side.

    Other people that i believe should use QoS are...

    -people with slow connection speeds combined with congested networks based on that slow connection

    -People with fiber because 1Gb/s is freakin fast... when you have enough that you could [set basically every class to 90mb/s (DONT DO IT LIKE THIS IT WOULD BE STUPID)] using QoS is a way to optimize your network to run more smoothly while never really making a noticeable difference (when you have "God Mode" speeds like fiber)

    If you have internet via 3g/4g (and are moving) or internet comes to you via satellite/radio by your ISP, your connection is to unstable to use QoS properly and should not use it ever IMHO. It could help a little when done right, but your speed is so variable that you will probably end up hurting yourself in the end. Use the Bandwidth Limiter deal with the bad peers that hog it all.
    Last edited: Feb 10, 2015
  70. lancethepants

    lancethepants Network Guru Member

    "through" Do you mean 'thorough'?

    I've given up on trying to be thorough when Toastman's ginormous qos thread explanation is sufficiently (and possibly excessively) so. It's actually extreme insightful and simply put, the alpha and omega of qos. I just refer them to his thread, and they can just be SOL if they can't read and follow its simple instructions. Most times people want to cut corners, or find or develop a "better" method than Toastman's, where one does not exist, so their qos is lousy and fails.
  71. Mr.CTT

    Mr.CTT Serious Server Member

    Yes Sir/Mam, thank you for pointing that out. I fixed it. I'm dyslexic so i can look directly at that kind of stuff and never catch it, bleh. haha

    His guide is amazing, I learned a lot from it actually while having years of experience in networking/computing, however I think it is a little on the high level side at times for the average user / below average user that doesn't know much or anything at all about networking or does not think enough in Boolean logic to really make the proper assumptions on things but they are given more than enough information to be able Google what they don't understand. Don't get me wrong it is an amazing guide, he goes really really far into detail which he didn't have to do. I actually linked @Moogle Stiltzkin to that guide originally when he posted his question about QoS in the shibby firmware section.
    Last edited: Feb 10, 2015
  72. Toastman

    Toastman Super Moderator Staff Member Member

    I'd like to warn people against some of the online "speed tests". Before you use them you need to understand what we are trying to measure, some of these sites don't help us to make our measurements. Why ?

    What we need to know in order to set up QOS correctly is NOT the speed to some distant server in another city or country. The ideal speedtest would be done on a server based at our own ISP, most ISP's do have a server to assist their own engineering staff. We need to know the upload and download speeds that we get along that little bit of wire/cable/fiber FROM THE ROUTER TO OUR ISP. No more, no less. I can't emphasize this enough. What the ISP does with it afterwards isn't of interest to us, nor is the speed at which it sends and receives data to Facebook, Youtube, Timbuktoo, Hong Kong, China, or the Horsehead Nebula.

    ONLY the speed of that little portion of the network between our modem/router and our ISP is what concerns our QOS, it is the rate at which the router can send/receive data to our ISP.


    @ Mr.CTT ...

    I just spent a few days playing about with at different times of the day. It variously reported my download bandwidth as between 1.6 Mbps and 2.3 Mbps. and my upload bandwidth as 380-530 Kbps - most of the time. This is completely useless and false information. My download speed is usually 17,300 Kbps and my upload speed is just under 1Mbps. I would recommend people avoid this and similar websites. And ignore people who talk about speedtest using cached information - maybe they do, but since it is on the other end of our link to the ISP we can use it with no problems.

    If I had used this site to set QOS I would be using about 2 Mbps of my 17300 Kbps downlink and only half of my uplink.


    Did I understand you correctly when you said to set the TOTAL of the "maximum" class settings to 100% ? That would rather defeat the whole point of QOS, no ? Perhaps I misunderstood ? And no, the whole point of setting a "maximum" for a class is to prevent things going above that speed whether other classes are, or are not, in use.

    I can't help feeling that you need to read through things again until you really understand them.
    Eri likes this.
  73. Mr.CTT

    Mr.CTT Serious Server Member


    I think you may have caught an error in my grammar / wording? You didn't quote me, but i am assuming you mean the quoted text.

    (more focused at low side)
    What I meant to say was for both in and out bound, you have two selections, a Low and a High for each class that it varies between when running. The low is the lowest possible speed that class would be (theoretically) allotted bandwidth for and supposed to be able to run at 24/7. So when you total that column for the section (inbound for example) up, the %'s should add up to less than 100%.

    The reason for this is because our internet is variable and should our max bandwidth available be less than that of what we have specified in our QOS settings for Max Bandwidth Limit and we hypothetically have all classes being used to the above minimum, there is a setting for bandwidth to immediately (but slightly-delayed because it cant make changes before or the moment of its detection of over-usage of bandwidth and I do not know how much is buffered nor do i believe i am smart enough to calculate a speed vs time vs size of dropped packets to specify that any further) drop back to in order to run everything without negotiating speeds to find a spot where it isn't fighting it's self to stop dropping packets and maintain the bandwidth.

    The Upper % side of the QoS for an entire section should be as close to 100% as you can get(within your preferences and based on your usage of each class). It is impossible to determine a perfect % for each class, because we do not know how much we use / need specifically to be perfect, and each class's usage more than likely varies constantly. If you were to set all classes to 100% that would be kind of dumb, as each of the classes will have to be re-negotiated continuously when using all available bandwidth and ones of the same level will have to fight for the bandwidth when calculated. For some of the classes, I don't believe it possible to hit 100% for most faster connections (for smaller networks), nor do i think outrageously high %s for a category will help anything.

    I actually believe the opposite in fact, because a spike of speed for something to load almost instantaneously (like WWW), then stop because a web page isn't that big to sustain a need for that speed, would negatively affect your bandwidth and cause other classes to be slowed down, then speed back up in the event WWW was higher priority(please used for concept only). This is probably a kind of bad example because that would be fast enough to not re-negotiate speeds significantly for long enough durations for other classes(a guess no factual proof), but I hope you see what I am trying to say with something that would cause this or sustain a speed long enough to cause it then drop off. I did not write the QoS system, so i am un-sure how aggressively it re-negotiates speeds, but I could see certain spikes that are short causing this, and setting a lower top speed for that class would prevent it(Ex: say 50 ms lowering 1 or more class vs 400 ms lowering 0 classes, that is a rough example used for concept because i am unsure how to say what i think so only use the concept please don't critically analyze it based on time). If you were to get as close to 100% in total for the accumulated totals for the upper column of all classes for a section (for example inbound), then you would over all increase performance and stability in my eyes and decrease cpu usage, as it is performing less calculations of rates and changing speeds less aggressively.

    The minimum should be sustainable at all times should there be a severe drop in total bandwidth available. The Maximum should allow one class to use the most amount of bandwidth available to it, should other classes not be in use or used enough to accumulate to the over all total available from the ISP. But the upper % column can be a double edged sword, if you set all classes at 100% or too high, then you run into the above mentioned problems. It is a give and take relationship that also depends on CPU availability as well.

    I hope this is what you referring to and I clarified what i meant well enough, please let me know if I am incorrect about my logic / understanding. If i made any offenses, my intention was not to offend at all and sincerely apologize if i did so.

    Please note that processor speed could greatly affect speeds also because QoS is very CPU intensive and that could be a bottleneck factor.
  74. Eri

    Eri Networkin' Nut Member

    As I understood, QOS has been working so nice for me for 5 years.
    Thanks a lot for Toastman, Shibby and other MOD'ers for the great contribution.
  75. cloneman

    cloneman LI Guru Member

    My stance has always been that you should only use QoS with a few rules that are important to you. For most people, this is about 3 rules...

    - something to guarentee BW for VoIP/Xbox/PS3
    - something to push HTTP slightly above other traffic
    - default

    Having a whole lot of rules only makes sense if you're managing a large public access point with many users. (Like Toastman's tenants.) Maybe all those rules do work well for some people, but I just see it as a troubleshooting nightmare if traffic gets misclassified.

    Not using QoS:

    You can self-manage your housemates to share the bandwidth properly, this works reasonbly well. Sooner or later someone does something that interferes. .

    I have 2 rules that I like, I give all UDP traffic higher priority (but limited to 40% of the total bw), which covers gaming and voip.

    I de-prioritize to a lower bandwidth any http or https download that has used more than 30mb of bandwidth.

    All members are assigned a port that they must use for their torrent client which is deprioritized. This is the only "weak point" that requires collaboration - but the upside is huge, everyone can do anything online without hurting anyone else.

    EVen in a worst case scenario, HTTP traffic (<30MB) is always prioritized over a misbehaving torrent user who doesn't use their assigned port.

    As always, anything high priority has a upper limit set so that even traffic marked as important cannot saturate the whole line.

    Another point of clarification - some people cite the loss of max speed as a disadvantage for QoS. I find this is not the case most of the time. When you have no QoS, you're scared to do certain activities for fear of slowing down other things. As a result, you never do big downloads and you're always self limiting yourself in your activities, so your average enjoyment and bandwidth capacity is lowered. Before implanting QoS, I would always wonder if I should download or upload something and how it will affect other people, if I should wait, or warn then first. You're really getting less bandwidth for your money, from a certain perspective.

    The only time I ever turn of QoS is when the only thing I have to do is download 1 file and I'm really impatient and need to go to bed - I'll turn off QoS completely for 15 minutes just to regain that lost 15%. This is a fringe case and 99% of the time I'm glad I can have a bunch of downloads running at all times of the day and I can still browse the internet at a much more "snappy" speed and without weird issues like pages not loading due to momentary packet loss caused by congestion
  76. Mr.CTT

    Mr.CTT Serious Server Member

    Haha not me, I let her rip! I have been seeing short spikes every few minutes during a constant download of 150mb/s on torrents. I think something is breaking TWCs QoS at times because I only pay for 50mb/s. They have a ridiculous amount of wasted bandwidth every second... such a waste...

    I guess my connection is fast enough that it doesn't matter a lot since i mainly d/l 1-3gb sized files randomly. if it were 100gb... then i would prolly hear some complaining. Then again i use vlans, so I feel like they have a small overhead that allows little things to still run ok when im torrenting. I know everyone has netfllix so i don't feel too bad eating the bandwidth for a few min. The buffer on that with our speed will cover any downtime they would see.

    I feel like for 5 of us, we use more than average. Is this a lot?

    Last edited: Mar 1, 2015
  77. sandspike

    sandspike Network Guru Member


    On Shibby 1.27, export does not work and it's throwing errors with save locations.

    Any help?
  78. Thomas Begley

    Thomas Begley Networkin' Nut Member

  79. Porter

    Porter LI Guru Member

    @Thomas Begley
    Yes, those rules have been updated. Please download one of the most recent Toastman firmwares and just enable QoS. The newest rules are intergrated in the Toastman builds.
  80. Thomas Begley

    Thomas Begley Networkin' Nut Member

    Im using the Shibby builds as I have a AC66U and it is not supported by Toastman, but im trying to incorporate the toasman qos system as i have read in many places that toastmans qos is the best.
  81. Porter

    Porter LI Guru Member

    Shibby should have the most recent rules, too. Maybe just give Shibby's rules a try first.
  82. Thomas Begley

    Thomas Begley Networkin' Nut Member

    I have tried shibbys defaults but i cant seem to get the rates/limits working right. For arguments sake with shibbys rates youtube videos constantly stop to buffer no matter what i do. When i followed that toastman link youtube played fine thats why i was wondering if there was a more up to date set of rules and limits i could follow.
  83. Porter

    Porter LI Guru Member

  84. Tony Arnold

    Tony Arnold Serious Server Member

    It's been mentioned a couple of times here on the forums that QoS is broken on ARM. If you delete any L7 rules, the remaining rules will work as long as they don't rely on the KB transferred (which also don't seem to work).
  85. Porter

    Porter LI Guru Member

    @Tony Arnold
    Thanks for mentioning it. Seems the ARM part isn't mentioned in every chipset description because I looked that up... But unfortunately you are correct!

    @Thomas Begley
    For the time being you will have to settle with the contrictions Tony is descriping or you are switching routers.
  86. vibe666

    vibe666 Network Guru Member

    I'm trying to get QoS working (again). I had it set up perfectly for years, which I'm pretty sure I managed via a script or screenshots from shibbys guides, but for the life of me I can't get it working now.

    I think part of my problem was that I used to have a 100Mbps/10Mbps FTTH connection, but I recently moved Perth and now I have DSL providing approx. 6Mbps/600Kbps which I'm finding a challenge having spent years with decent internet speeds previously. :(

    I'll be totally honest here, I've tried reading through this thread (and others) and a lot of this is going over my head. I used to 'get it' at least enough to set it all up originally, but since then I've spent a lot of time on some very heavy medication for some major back problems and I just don't have the brain power any more to make sense of it all.

    I have an E3000 running Tomato v1.28.9054 MIPSR2-beta E3000 USB vpn3.6, but when I set up the modem (an old ADSL2+ ADSL2MUE), I made a complete balls of it and ended up unable to connect to the E3000 at all and had to do a factory reset (stupid brain!:confused:) and lost all my QoS settings, which I did previously have backed up in my config, but which was on a laptop which had a failed SSD two weeks previously.

    I've been trying to set it all up again manually, but I just can't get QoS to function properly with the slower connection and I'm now stuck with pretty much a dead internet connection if I even try and run one torrent. :(

    Is there a quick and easy way to try and get it working properly again that my drug addled brain can comprehend?

    I've seen a few pre-made scripts to alter QoS settings, but none of them seem to be able to do what I want and torrent speeds fall through the floor with anything other than the default settings, but the default settings cause everything else to fall over once I start any torrents.

    I have a feeling the sweet spot is going to be to edit the conntrack settings and QoS to make the most of the (limited) speed (as i'm pretty sure that's what did it last time), but as I said, it's pretty much beyond me at this point to figure out exactly what that should be.

    Can anyone help without turning my brain into even more jelly. :(
  87. Porter

    Porter LI Guru Member

    You don't seem to be using one of the most recent Toastman build so please upgrade. When doing so please make sure that you securely erase your NVRAM so that you will get the most recent config, too.

    With a lower bandwidth it is imperative to get the bandwidth right. So please make sure to measure correctly and deduct about 15-30% from your values.

    To be honest, you should just be able to enable QoS and use the default rules. No need for scripts. What you might find useful ist to set your Bittorrent client to a specific port which you can then match for in your ruleset. Use some higher Port in the 50000 for this. The rule on you classification page needs TCP/UDP and "Src or Dst". As a class you can use Bulk/P2P. Put this rule right under all the rules for the "Service" class.

    You could also have a problem with the amount of concurrent connections (full conntrack table) but with just one bittorrent user this doesn't seem very likely. A typical sign of this would be a very high load of your router, when bittorrenting. Look for the load on Status/Overview. Of course you can also monitor your connections on Advanced/Conntrack. Just click on "count current...". If the connections almost reach the maximum, you might run into problems.
    vibe666 likes this.
  88. cloneman

    cloneman LI Guru Member

    Once you are on the latest version, be sure to enable the setting "Settings for DSL only". it doesn't really matter which number you choose for the overhead value, so long as it is turned on. This will allow the QoS to work on ADSL connections when downloading and uploading at the same time.

    Once you have followed Porter's other recommendations, if you still have problems, my advice would be to delete all the ~40 rules and start from scratch, using only 3: Default, Torrent, and high priority.

    Before losing your mind altogether with testing. you want to make sure your DSL connection is working properly. If your latency is HUGE (>400ms) while uploading there may be an issue with your DSL line (in which case you'll likely experience disconnections/loss of sync as well).
    vibe666 likes this.
  89. vibe666

    vibe666 Network Guru Member

    Thanks for the tips, I think that might have got it sorted, thanks a million! :)
  90. jochen

    jochen Network Guru Member

    Are there chances to get QoS working when using TOR? (I think no, because all traffic looks the same to the router)
  91. KyleS

    KyleS LI Guru Member

    Why not just use Tor on the device itself?
  92. jochen

    jochen Network Guru Member

    You mean on the router? This has some disadvantages. I'm using Tomato Shibby and the TOR client in Shibby can only redirect port 80 to TOR, not 443. And when activated it forcibly redirects all Traffic through TOR, but I need to reach some sites without TOR (because they block TOR), so I need to switch with Foxyproxy based rules.
  93. Grimson

    Grimson Networkin' Nut Member

    Then set it up by hand instead of using the GUI.
  94. jochen

    jochen Network Guru Member

    I don't know how to do that.
  95. Grimson

    Grimson Networkin' Nut Member

    You have access to the Internet, learn it!
  96. Chris OConnor

    Chris OConnor New Member Member

    EDIT - Seems my issue has been covered, ARM issue. Sorry for the post
    Last edited: Jun 1, 2015
  97. phlibby

    phlibby Networkin' Nut Member

    I get it. I get IT! I finally get it!!! I spent the longest time trying to learn linux CLI commands and Putty and finally I got my mind to jump the barrier - whatever it was? It Is Easy! - but only when you know where the output is. I was looking all over the place and must have been doing something wrong cause I didn't see it appear "on screen". Well, at least now I know how to do it. This is going to make expanding my network SOOO much easier! Thank you Toastman. I only would like to know how to transfer the Wireless Client Filter? I don't see one for that?

    Tomato 101 Student,

  98. cloneman

    cloneman LI Guru Member

    Would it be possible (either as CLI, or in a future release) to classify packets based on packet size? It would be pretty nice to give small UDP packets a higher priority, as a catch-all for games and VoIP.
  99. Porter

    Porter LI Guru Member

    Yes, that's possible. You'll just have to learn the syntax for tc. The only thing I'm asking myself is whether putting it into the WAN Up-section would be OK. I don't know when the QoS-system gets loaded and I'm not even sure whether this is important. You could run into problems, if the QoS-system does run after WAN-up and does a reset of all the classes and filters. But that's speculation. You might be able to see which tc filters are being loaded by using tc -s or something, which should show you the statistics to check for your new UDP filter.
    cloneman likes this.
  100. cloneman

    cloneman LI Guru Member

    okay, I did a bit of noob investigation.
    From what I'm gathering this would be something goverened by the classify page, which I'm seeing is managed by iptables. Presumably, this marks the packets and later on another process (tc?) applies the rules

    (I'm looking at /etc/iptables)

    In there, there's stuff like
    -A QOSO -p udp -m multiport --dports 20000,20001  -j CONNMARK --set-return 0x800102/0xFF
    which Means, classify udp ports 20000 and 200001 with a marking of 0x800102/0xFF, which I've determined corresponds my class #2.

    Reading up on netfliter, I've come up with the following rule for my use-case
    -A QOSO -p udp -m length --length 0:970  -j CONNMARK --set-return 0x1100105/0xFF
    Which I think means take all udp packets less than 970 bytes and mark them for class #5

    tl-dr; How do I add this rule, even temporarly, for testing? This is a more of a "how do I make changes to IP tables in tomato" question. I tried vim'ing /etc/iptables but I assume that's not how its done in the tomato world.

    EDIT: progress.

    The u32 matching seems to correctly add itself to the ip tables rules
    iptables -t mangle -A QOSO -p udp -m u32 --u32 "2&0xFFFF=0x2:0x0100" -j CONNMARK --set-return 0x1100105/0xFF
    But... I don't think this is being matched if I look at the QoS graphs.

    Anything I do with "length" matching throws the error:
    iptables: No chain/target/match by that name

    Firing in all directions here, no idea if I'm making any sense.

    EDIT: I got it to work! more testing needed.
    Last edited: Sep 7, 2015
    Toastman likes this.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice