Need some clarification on how QOS works

Discussion in 'Tomato Firmware' started by zhenya, Jun 10, 2007.

  1. zhenya

    zhenya LI Guru Member

    I've been successfully running Tomato and my voip provider for some time now, however, I had been accidentally limiting the bandwidth of my other services. What I'd like to do is keep my voip service as good as it has been, but give the rest of my services access to that bandwidth when it's not being used.

    My current setup:
    I have a 1Mb up link, and I have VOIP for my home phone. I'd like to make sure that ~100kb is always available for that line, and give other services and applications the rest of the bandwidth. Currently, I have my voip adapter classified as 'highest' by mac. The only other service in this classification is DNS.

    Here is my current Basic setup:


    First, my understanding is that by giving the 'Highest' classification access to 100% of my bandwidth, and limiting all other services to 90%, 10% of my bandwidth is always in reserve for the phone.

    Second, I'm not sure I understand what the first column is controlling. I believe that it controls the 'portion' of the available bandwidth that each classification can use, if all bandwidth is being used. Based on that assumption, I am giving my highest classification access to 20% of that 850kb, or 170kb total, because the only services classified as 'highest' are my phone and DNS - they could never use more bandwidth than that. I have then apportioned 50% of it to the 'highest' classification, as I have bandwidth intensive applications is this class, that also need priority. I've filled in the lower values, adding up to 100% total.

    So the question; is my understanding correct? I'm believing I'm missing something, because this is very different than the examples I've seen posted here, where everyone has the 'highest' classification allocated 80% of their bandwidth.

  2. zhenya

    zhenya LI Guru Member with working picture!! :)
  3. jochen

    jochen Network Guru Member

    I'm also very interested in this topic. No answers yet?
  4. rcordorica

    rcordorica Network Guru Member

    Although the semantics of the OutBound Rate/Limit are not given by Jon (it would be a good question to ask him) I think it works like this.

    Outbound Rate (First column):
    This controls the "guaranteed minimum" in percentage of bandwidth that QoS will give to that specific class.

    Assuming you are using 100% of your bandwidth by various applications, QoS will try to give your classes the Outbound Rate you set up.

    Limit (Second Column):
    Obviously the upper bound limit. QoS will limit a classification from using more than this limit. 100% is equivalent to having no-limit (the maximum upload speed of your connection).

    Usage in the real world:
    I don't usually saturate my upload bandwidth, because bandwidth hogs like P2P are limited to using 60% of my bandwidth. They will never use more.

    That leaves 40% of unused bandwidth to be divided up among my classifications, and usually they are only using a fraction of that.

    Web-browsing for example has "on-off" behavior. When downloading a web-page or downloading a youtube video I have given port 80 (HTTP) a 80% upload limit, but only a 4% Outbound Rate.

    This doesn't mean HTTP only transfers at 4% of the total. This doesn't mean it transfers at 4% of 80% either.

    It means that HTTP will use 80% of my bandwidth if it can when it is available. I'd rather burst the data at 80% of maximum to finish it ASAP if I can.

    The Limit field is more important to prevent congestion in the first place.

    The Outbound Rate field is only important if your connection is already being maxed out on upload.

    Your understanding seems to be like mine.

    I am not sure if the 1st column needs to add up to 100%. Ideally you can do this to define a strict set of rules for the QoS to follow under constant 100% bandwidth usage.

    But usually there is free bandwidth, so QoS will follow your order of precedence to determine how much bandwidth to give (strict rule ordering).
  5. zhenya

    zhenya LI Guru Member

    Thanks for the response rcordorica. It seems like we both have pretty much the same understanding of how this works, so that, combined with the fact that my phone line has been perfect for months is encouraging. :thumbup:
  6. bengali

    bengali LI Guru Member

    This is exactly the post I was looking for.

    I PMed rcordorica on another thread he had posted, with some questions along these lines. I believe that his interpretation of most of the settings is fairly accurate, that being said though I'm not sure the numbers he gave make optimal use of bandwidth.

    To continue on this question I would like to pose the following. I'm not sure how much this is in sync with what has already been said, but I have been looking everywhere for specifics on this. Only going back and actually looking at the default GUI do I believe I understand it:


    One can only assume that the firmware's default settings are designed out of the box for a pretty good level of functionality. I think the key is actually in the wording of Rate/Limit as well as the fact that for the major classifications (Lowest - Highest) the Rate adds up to 100% and the Limit is 100% on all. (I am sort of throwing the A-E classes out the door here, but they seem to be secondary optional classes anyway, and my explanation will show how these can be made to fit in).

    With that being said, the simplest I can word it is this:

    "Outbound Rate (the first column) is the percentage of bandwidth that will be used by the class assuming 100% of upload bandwidth is utilized."
    - These should add up to 100%, but don't necessarily have to.

    For example: Assume we are using 5 applications, each one classified in one of the major 5 class levels (Highest - Lowest). Also assume we have a 250 kbps upstream speed and that any one of these applications would eat up the whole pipe if allowed. With the default settings (shown above), this would limit the app classified as "Highest" to use no more than 200kbps (80%) while still allowing the "Lowest" 5kbps (2%), with the other classes using their defined percentage of the available bandwidth.

    Having this number add up to 100% (theoretically) guarantees that every application will have at least some bandwidth available to them at all times. For instance, if I put 30% as the Rate in all 5 classes that would add up to 150% which we obviously can't reach. Because of this, at least two of the classes will have to perform sub-optimally (under 30%) causing these two classes (and corresponding app) to get little or no bandwidth. Or possibly, to mess the entire algorithm completely, making it difficult or impossible to determine actual allocations.

    So why allow us to set these to numbers that don't equal 100%?
    Well, theory is different then practice. In practice I might not be using all 5 applications at the same time, or they might not actually attempt to take up all available bandwidth. In the default scenario if I took my Medium application offline I would in effect not be utilizing 5% of my available bandwidth. Or, in other words, I free up 5%. What class gets to utilize this 5%? Answer...any of the 5 classes. Why? Because they are each already utilizing their "guaranteed" Rate and they also each can use up to 100% of available bandwidth (defined in the Limit).

    So the crux is now how do we control which of the 4 other classes gets to use the 5% that is freed up? Answer: By setting one of the classes with a higher Rate. If we increase Low from 3% to 8% then when we shut down our Medium class application we can be sure that Low gets that extra 5% of bandwidth. Of course, when we bring Medium back online it is now getting only 5%, whereas Low is getting 8%, so you have to look at the numbers carefully to avoid unexpected results.

    So, if you want all 5 applications to equally share the bandwidth at all times (when pipe is maxed), you should set them to 20%/100%. If you take one offline you free up 20% to used by the other classes. Which other class gets it I cannot say as I would think that Highest only gets its priorities from the numbers stated (and in this case Highest is set the same as all others). One might assume that this would get evenly split between the remaining 4 classes, but that would only be one possible way to handle it.

    In most cases, we don't want to equally share bandwidth, as one App needs more bandwidth then another (like VOIP). In this case we would (for example) set VOIP to the default Highest at 80% ensuring that whenever we are using our VOIP app we will take 200kbps. We will NEVER take more than this if the other 4 classes are also competing for bandwidth, but if they aren't, then we can max out our VOIP all the way to 100% (250kbps).

    Based on the above descriptions/examples I can't see a good reason to ever set the Limit column under 100% as long as your Rate columns add up to realistically acceptable numbers. Doing this will only cause you to lose available bandwidth that might otherwise sit idle. If for example you set Lowest to 1%/55% then when your pipe was maxed by all 5 Classes your Lowest would only get 1% of bandwidth. When everything else was idle however, you would only be giving 55% to the Lowest, with the remainder of your bandwidth (45%) doing nothing.

    One possible reason for setting below 100% (and in this case it would be moderately less than 90%), would be because of the latency that QoS would have in shaping a new connection. For instance, if you have your Lowest application/class set to 5%/100% and it is the only thing running then you are using 100% of bandwidth for it. Let's say that you start your VOIP on Highest classification. QoS now needs to reshape your traffic to give VOIP its guaranteed Rate. To do this it will start to limit your Lowest application. This takes some time and means that for the first few seconds of your call you might experience poor quality. If you had limited lowest to 90% then we would have had an extra 10% leeway while reshaping, enough possibly to avoid the issue.

    On a similar topic I tend to agree with rcordorica (from another post) concerning Max Bandwidth. He suggests setting it to your actual upstream bandwidth, as opposed to 90% as some other tutorials/posts will state. If you set to 90% you will limit ALL outbound traffic to 90%, effectively losing 10% of your upstream traffic permanently. Of course if you grossly overestimate let's say you put 400kbps instead of 200kbps then your classes set to 50% will actually consume 100%... not good for QoS. So, use 100% or very close (96-98%) to be efficient, but make sure to thoroughly measure your speed properly.

    Well, there it is... I think that I am on the same page with some of the previous posts; however I haven't honestly read a description that puts it all together clearly. Hopefully my examples explain it a bit more adequately.

    Please let me know what you all think, and if you disagree with my explanations please present a salient argument, as I would like to have at least one thread here with complete information on how this all fits together.

  7. azeari

    azeari LI Guru Member

    good job here. this ought to be stickied, though i'd disagree on the outbound bandwidth allocation.

    Reason being: Most Internet Service Providers(ISPs) NEVER give you the full bandwidth stated on your contract. i.e. a 10mbps/1mbps line might turn out to be 9.5mbps/0.85mbps.

    Also, attempting to utilize more than 9.5mbps downstream or 0.85mbps upstream in this hypothetical case, will trigger the ISP's own traffic shaping routines, and this usually results in undesirably high latency.

    Overall, i'd first suggest performing a reliable bandwidth test to determine your actual inbound/outgoing bandwidth. Afterwhich its up to you to choose between maximizing available bandwidth and minimizing latency. I'd usually choose the latter. (=
  8. bengali

    bengali LI Guru Member

    azeari, thanks for your reply.

    I don't disagree with your statement, which is why I mentioned to "make sure to thoroughly measure your speed properly."

    I didn't mean to infer that you should put your maximum up/down at what was stated in your contract, but rather what you reliably measured in speed tests.

    I can't really comment on your statements regarding using above 90% of your bandwidth in terms of triggering your ISPs traffic shaping. I personally haven't experienced this, and I guess it is subjective to your ISP.

    Also, I'm coming from a broadband cable connection, where my upstream speed is in the 350kbps range. On upstream speeds this slow, I feel that shaving off 10% of your available bandwidth is quite substantial, and I would rather have the extra 35kbps. However, I do recognize the latency issues, which is why I do make reference to allowing some leeway (96-98%) seems like a better balance to me.

    However, if you have a much greater upstream speed (512kbps+ range), I do agree you have more play in minimizing the latency.

    Thanks for your support!
  9. rcordorica

    rcordorica Network Guru Member

    This is a reply to begali's PM to me. I figured I might as well put this information up for everybody to see.

    Yes, meaningless to VOIP that can't take advantage of all the reserved bandwidth. But in my case the HIGH category is used for port 80.

    Yes, I am purposely putting a limit on every other category besides my HIGHEST category because I only use my HIGHEST for latency sensitive operations, i.e. FPS games, VOIP, and DNS. Now this is a waste.. but the reason for doing so is to provide guaranteed low latency to newly started HIGHEST operations. For example, when I start a FPS game, I know that there will always be available bandwidth for it, since all other category are not allowed to use 100% of the available bandwidth. I don't even have to wait for netfilter to make the classification.

    Of course, the real question then is "How fast does netfilter classify a connection and begin packet shaping?" If it does it so fast that I don't even notice lag during heavy P2P, then I am needlessly wasting bandwidth. I am basically being cautious and assuming that netfilter doesn't work all the time.

    I did it for the same reason as above. By progressively lowering the inbound limit I am making sure that netfilter obeys my QOS setup when several different classification priorities are all fighting for the same bandwidth. By progressively lowering the inbound rate I effectively created QoS for load conditions by the way of dropping packets more often for lower priority classifications.

    Is that necessary? Well considering QoS already follows the order of precedence, I am not sure.

    I would have guessed that NONE turns of the limit for that priority. Honestly I've never tried it.

    Yup, I'm still learning how to optimize QoS.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice