Simplifying QOS for faster speed

Discussion in 'Tomato Firmware' started by danielhaden, Nov 1, 2018.

Tags:
  1. danielhaden

    danielhaden Network Guru Member

    Code:
    Name     Promotion, Limit/Cap  Classifications
    01.Exempt      15%, 100%       DNS Service
    02.Max         13%, 100%       VOIP and Game ports
    03.Premium     10%, 100%       less than 1k, 80,443
    04.Express      7%, 100%       Streaming video
    05.Quick        5%, 100%       1k to 512k, 80,443
    06.Medium       0%, 100%       512k and more, 80,443
    07.Default      0%,  75%       less than 1k, unlisted
    08.Slow         0%,  50%       1k to 512k, unlisted
    09.Crawl        0%,  25%       512k and more, unlisted
    10.Punish       0%,   1%       L7 microsoft, 80,443

    init script
    Code:
    echo "microsoft" > /etc/l7-extra/microsoft.pat
    echo "(Microsoft-Delivery-Optimization/|.mp.microsoft.com)" >> /etc/l7-extra/microsoft.pat

    Notes:
    *The promotion (minimums) column dynamically hinders speed of everything else, so that's why I kept the percentages small (and the sum of them not more than 50).
    *The limit/cap column is not dynamic--it is a fixed limit (persistent slowdown) for only the matching classification, so that's why I didn't put a limit on web browsing (100% is full speed). However, for gaming use, it is necessary to use limit column for deciding what 'pays for' using QOS, and what doesn't.
    *Items marked unlisted, go at the bottom of the classification list and in the same order.
    *Put L7 microsoft near top of the classification list, above other 80,443. That limiter lets you decide how to use your bandwidth instead of giving it away to microsoft.
    *It is possible to fill the standard Tomato QOS with these replacement names and values.
    *The main intent of this simplifying is for speed; but, after that you can add to it.

    I've been using this QOS at home, along with very few classification rules. The effect of this simplifying/streamlining has been web browsing much faster--quite competitive with other routers.

    Accessory firewall script
    Code:
    iptables -I INPUT -s 192.168.1.0/24 -m connlimit --connlimit-above 150 -j REJECT
    This prevents router stalling from retry floods, badly programmed torrent clients and other connection runaway. To use the script, you'd want to set your timeouts, udp 60 or less, and tcp 600, so that disused connections are cleaned out before the allotment fills. I've been using this script at home.
     
    Last edited: Nov 8, 2018 at 11:09 AM
    cloneman, M_ars and Techie007 like this.
  2. danielhaden

    danielhaden Network Guru Member

    Opposite of the automatic example above--here's extreme favoritism instead.
    Code:
    Name     Promotion, Limit/Cap  Classifications
    01.Exempt       1%, 100%       DNS Service
    02.Max         48%, 100%       VOIP and Game ports
    03.Premium      1%,  95%       less than 1k, 80,443
    04.Express      0%,  85%       Streaming video
    05.Quick        0%,  90%       1k to 512k, 80,443
    06.Medium       0%,  80%       512k and more, 80,443
    07.Default      0%,  75%       less than 1k, unlisted
    08.Slow         0%,  50%       1k to 512k, unlisted
    09.Crawl        0%,  25%       512k and more, unlisted
    10.Punish       0%,   1%       L7 microsoft, 80,443
    You could set the master download rate at actual speed, so there's no packet drop for exempt and max. Then, the QOS margin is made up for with the limit column. You could put a work-from-home VPN laptop (or game) with a Max priority classification. This example only works if you can identify an application that is more worthy than everything else.

    Unlike the example in the first post, this one will show a slightly lower speed on internet speed tests and file downloads; however, it won't slow down web browsing unless a Max priority classification is in use.
     
  3. cloneman

    cloneman Addicted to LI Member

    Interesting post. Why do you give so little "Promotion" though? This seems to me like it would starve those 0% classes. On shibby the minimum is 1%, I don't know how you got 0%.
     
  4. Monk E. Boy

    Monk E. Boy Network Guru Member

    Based on my experiments with connlimit for SMTP back in the good old days when the worst you could expect an infected to do was send spam, the connection limit is based on whatever the source is of the rule is. Since your source is a subnet, that would mean the entire subnet is limited to 150 connections.

    Based on how the rule is written it would seem to indicate if the source is 192.168.1.0/24 then there are only 150 connections allowed. It doesn't say 192.168.1.1 is allowed 150 connections and 192.168.1.2 is allowed 150 connections, all of 192.168.1.0/24 is allowed 150 connections.

    Back in the day I had people unable to send mail because the total number of SMTP connections had exceeded my limit of 25. If this was a per client basis 25 is plenty. Gobs even. But if you have more than 25 people trying to send mail at the same time then connections start getting rejected.

    Based on some searches just now I discovered the option --connlimit-mask which seems to do exactly what we'd be after.
    https://serverfault.com/questions/755992/what-are-these-iptables-rules/755998

    Set it to 32 to limit conntrack to monitoring hosts individually. Set it to 0 to apply to all traffic. (255.255.255.255 vs. 0.0.0.0 subnet masks)

    The trick is to test it and see if it works before sticking it into firewall/scripts. Paste it into an ssh/telnet session and see if it blows up. If it does, reboot, rule goes away. If it works as intended paste into scripts, save, reboot, enjoy.
     
    danielhaden likes this.
  5. danielhaden

    danielhaden Network Guru Member

    0 is for those versions that have an off/none in the list (for lighter cpu load). However, if your minimum is 1, just raise the example by that much. The promotion/minimums column works by slowing down everything else; so, if the sum total of that column is as much as 100, it means to slow down everything. This is checked with page load timings. You can check with a photo archive site and auto scroll with the mouse middle button, to see if there's blank screens scrolling Or lots of photos flying by.
    Example: http://catsncats.tumblr.com/archive
    click center mouse button, see how fast you can scroll without missing a photo, and then what is your speed in cats per second?
     
    Last edited: Nov 5, 2018
  6. danielhaden

    danielhaden Network Guru Member

    It is really hard to tell if adding '--connlimit-mask 32' does anything differently from the connlimit default behavior, which is, per client. Would you help in testing it? Currently in my router, I've got:
    Code:
    iptables -I INPUT -s 192.168.1.0/24 -m connlimit --connlimit-mask 32 --connlimit-above 150 -j REJECT
    This doesn't prevent Tomato, DD-WRT and Gargoyle from reporting in excess of 150 connections in the gui. However, it does prevent persistence of a connection overload, by cutting the overload timeframe down to about 2 minutes.
    So, not exactly what we thought it would do; yet, very helpful indeed.
     
    Last edited: Nov 7, 2018 at 10:54 PM
  7. cloneman

    cloneman Addicted to LI Member

    :eek::D

    Do you have any simultaneous competing traffic when you do this test? Your "promotion" column should not have any effect unless you are running traffic in multiple classes at once
     
  8. danielhaden

    danielhaden Network Guru Member

    Thank you for quoting my funny page speed test. Timing-centric tests like that are a great way to judge success or costs of QOS. It is a lot more relevant than a bandwidth test or a bandwidth expenditure, since responsiveness has an effect on utilization.

    I did try with and without other concurrent workloads.
    However, my main effort wasn't in comparing Tomato with itself.

    Actually, I was working on the problem of Tomato QOS not competing well against DD-WRT/broadcom and Gargoyle/atheros QOS, for speed. The problem is the Tomato QOS default settings is a hoarder's mess that costs performance. However, most attempts to clean up Tomato's QOS results in highly competitive performance.
     
    Last edited: Nov 6, 2018
  9. Monk E. Boy

    Monk E. Boy Network Guru Member

    It's rather amazing what a simple size-based QoS system will do. So category 1 are all connections under 512KB. Category 2 are connections 512KB to 1MB. Category 3 are connections 1MB to 5MB. And category 4 are connections over 5MB. So as a connection transfers more data it progressively gets lower priority. This generally works well because connections that transfer small amounts of data generally require quick responses (e.g. DNS, NTP, ICMP) while the largest connections (e.g. streaming video) are more tolerant of minor bandwidth hiccups. You can skip the whole TCP/UDP and just use any protocol any connection and strictly define them by the amount of data transferred.
     
  10. danielhaden

    danielhaden Network Guru Member

    Like this?
    Code:
    1.   10%, 100%    up to 1k
    2     3%, 100%    up to 512k
    3.    1%, 100%    up to 1MB
    4.    1%, 100%    1MB and more
    _.   n/a,  n/a    no rules assigned to it
    _.   n/a,  n/a    no rules assigned to it
    _.   n/a,  n/a    no rules assigned to it
    _.   n/a,  n/a    no rules assigned to it
    _.   n/a,  n/a    no rules assigned to it
    _.   n/a,  n/a    no rules assigned to it
     
    Last edited: Nov 8, 2018 at 10:23 PM
  11. danielhaden

    danielhaden Network Guru Member

  12. ghoffman

    ghoffman LI Guru Member

    @danielhaden - where do you enter the table with promotions, limits, and priorities?
     
  13. danielhaden

    danielhaden Network Guru Member

    The Tomato QOS page lists promotions as Minimums (those are dynamic as in nearly automatic).
    It is probably best to keep the limits column at 100% (no limit), at least during initial QOS setup (limits are fixed reductions, so avoid that).
    The remaining portion is found on the QOS classifications page, and that usually needs simplifying.
    On the classifications page, each rule may cost against cpu speed; so, what you will enjoy most are few rules that spare the cpu, as in 'earn their own keep'
     
    Last edited: Nov 13, 2018 at 6:10 AM
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice