QOS Limit problem

Discussion in 'Tomato Firmware' started by belliash, Jun 22, 2009.

  1. belliash

    belliash Addicted to LI Member


    I used WRT54GL router with 1.23.8515.5 RAF mod by Victek with 1024/256 kbps internet connection in PPPoE techology. Everything worked fine since today, when i migrated from 1mbps to 2mbps and actually i have 2048/512kbps. ISP is the same - no changes.

    As i share internet for 5 computers i decided to use QOS limiter (IP/MAC BW Limit). I changed downlaod and upload bandtwitch from 1000/250 to 2000/490 and increased speeds in the table. You can take a look at that here:

    The actual problem is that i ( cannot take any advantage from this conenction... When i set:
    DLCeil to 512 -> i have ~60KB/s
    DLCeil to 975 -> i have ~90-110KB/s
    DLCeil to more than 975 -> transfer is not faster than max 120KB/s

    What i have notices is that if biggest DLCeil i have then bigger transfer fluctuation i have and bigger cpu load (max i noticed 0.97/0.64/0.42) :(

    If i disable QOS then transfer is stable as rock (about 254-268 KB/s)...

    Anyone came across something like that? Any solution? I have already tried disconnecting from net and rebooting router. It has been reflashed few days ago (thorought cleaning -> flash -> thorought cleaning -> settings -> speed up net -> qos settings -> issue)

    P.S. Sometimes im also able to get 250K/s with QOS enabled but freezes after a while, speeds up to over 1MB/s freezes again and then slows down to ~120KB/s forever (i need to interrupt file download and restart it to get 250KB/s again, but then it will freeze and slow down again ;/)...strange for me

    P.S. 2: speedtest.net reports that i have over 2mbps (2.1-2.2mbps) even when i set DLCeil to 512kbps
  2. davexnet

    davexnet LI Guru Member

    It's my understanding that download OQS is almost worthless - almost everything
    I've read says not to bother with it. Can you disable the download QOS
    and just leave the upload part? Then see what happens?

    I reported a problem myself a few days ago but mine was the reverse of yours.
    In my case, QOS seems to be limiting upload B/W when it shouldn't.
  3. Toastman

    Toastman Super Moderator Staff Member Member

    Belliash is using the IP/MAC bandwidth limiter, which isn't actually QOS as such. You might have more success with tomato QOS and it will also allow any user to make use of the unused bandwidth, when not being used for anything more useful.

    From the CPU stats you describe something seems to be seriously wrong, the load should not be that high!

    davexnet - the people who talk about "download QOS" don't seem to understand how the QOS system works. Or, to put it tactfully, they are idiots. The incoming traffic shaping, which is what they are referring to, is invaluable as a tool, and is PART of the QOS system on the router. The other important part is prioritisation. Using the two strategies together, can actually achieve a lot. Just be careful what you believe when you source information from the internet.
  4. davexnet

    davexnet LI Guru Member

    I must admit, I've never played with the download part. (I'm using Tomato firmware
    v.23) I've never seen the need to limit it, I can always do it from
    the applications if I need to.

    Programs such as Utorrent with a download speed setting seem to be able to
    get very close to the chosen speed. How do they do this? Are they delayinig
    acknowledgments to the sender? How does the router do it?
  5. Toastman

    Toastman Super Moderator Staff Member Member

    Hmm. I'm not very knowledgeable on how the P2P stuff works internally. But yes, they utilize the standard TCP backoff and retry mechanism to control download speeds, which is exactly the same as QOS uses. It is a standard and fundamental congestion avoidance mechanism which is fundamental to every single application using TCP on the internet, without which the internet could not exist at all.

    The idea of most of the clients and systems is that to reward uploaders some sort of rating system is used to "bump" seeders in priority. But those (millions) of us on ADSL cannot upload much, so it would seem we can't make it up the list. So other strategies are used. Since we can't get much gain from seeding, then we really need to limit the seeding and devote what little uplink bandwidth we have to getting downloads, which is generally what we are really interested in. If you search google on how to increase download speed you'll find this is top of the list. Bit Comet is supposedly more aggressive than others at downloading, and many sites ban it's use. In any case we are really at the mercy of the server end - a good seed will be fast, while bad seeds cannot be improved. I use uTorrent, with 10k upload limit in the client, no download limit, 1000-2000 connections, and usually 50+ downloads at a time. Increase the number of half-open connections in Windows if necessary using the standard patch (google again). Doing this I can max out the incoming bandwidth with P2P. Perhaps that would translate into about 100 movie-sized files a week. All of this is using the public servers.

    As for the router, that would take too long to go into and this other article covers it already: http://www.linksysinfo.org/forums/showthread.php?t=60304
  6. davexnet

    davexnet LI Guru Member

    Hi Toastman,
    Wow, you download a lot of stuff, I don't keep mine running 24*7, but
    I use some private trackers, so keeping up the "ratio" becomes more of a factor.
    What I was really asking was this:
    Assume my true download speed is 500KB (without QOS active).
    If I start Utorrent and don't limit the B/W, and I get some very busy torrents,
    the 500 KB (or very close to it) will soon be used up.

    However, if I want to limit Utorrent download speed directly, I can adjust it in
    Utorrent and it's very accurate. What is Utorrent doing under the covers to
    slow down the incoming data ?

    Regarding download QOS - this is the section I was referring to.
    Have you used it? I don't see the point of it myself. Is this section actually
    usable ?

  7. Toastman

    Toastman Super Moderator Staff Member Member

    On ADSL I don't see how anyone can "keep up" with seeding to attain a decent ratio. Sadly :frown:

    The section you show is not "incoming QOS" - it is merely a bandwidth limiter. It is used in conjunction with the rest of the QOS system. It is best to consider it as PART of the router's QOS.

    For example, if you set 200k of upload for P2P - you have no idea if this will generate incoming downloads of 1Mbps or 5Mbps - it all depends on the seeds, the servers, and other things which are completely outside your control. So it's highly probable that your incoming bandwidth will become saturated and screw up your Web browsing etc. if you don't prevent it from doing so. Setting a limit here will cause the excessive traffic to be dropped when the limit is exceeded, resulting in the remote server resending after a wait period. This wait period increases exponentially each time packets are dropped until the link stabilises. In other words it is a way to slow down that class by using a mechanism which is part of the TCP protocol. This is well covered in the QOS thread at http://www.linksysinfo.org/forums/showthread.php?t=60304. I recommend you read through the thread.

    The whole theory of TCP congestion avoidance, the TCP retransmission timer, etc. is covered in many hundreds of articles, google for info.

    But you can easily see for yourself what this section does. Just enter a limit and traffic in that class will reduce to that limit after a short period.
  8. belliash

    belliash Addicted to LI Member

    Actually everything works correctly except uploading...
    I can limit upload speed to eg 256kbps (i have 512kbps upload) and im uploading about 30KB/s - so it seems to be ok.

    But when i am uploading data i have troubles with accessing web sites.
    Looking for address.com -> freeze
    Waiting for address.com & transferring data then goes ok...

    It freezes at looking for address.com...
  9. Toastman

    Toastman Super Moderator Staff Member Member

    Perhaps DNS lookup is being delayed. Is that still the top priority or did you change it? Try adding as an additional DNS server in case your ISP servers are slow.
  10. belliash

    belliash Addicted to LI Member

    I use OpenDNS... It occures only when i am uploading data (eg 20KB/s on 512kbps what is less than half)... I noticed that when i go to QOS->Basic, disable and enable QOS and press Save, then it fixes that strange situation...
    Fixes (but only when i am still uploading in time when i make changes)....
    nvram set qos_enable=0
    nvram commit
    vmram set qos_enable=1
    nvram commit

    does not help... I think this is related to service restarts... but i have not found /etc/init.d or /etc/rc.d or any file named tomato.cgi....

    Whatever.. it warks well since next reconnect (DSL session max here is 48h, so the router resets connection every 24 hours)... I thought i could add some WAN UP script to restart services... What do You think? What to restart? And how to do that?
  11. habskilla

    habskilla Network Guru Member

    what do I know?

    I've never understood this concept. To me, the only way to limited inbound is if your ISP controlled it before it hits your router or the receiving application (like utorrent) is able to control the amount of bandwidth it consumes. If you saturate your inbound connection, what good is it doing to reject packets once it's arrived at your router? Your line is still saturated from the internet to your router, all you've accomplished is limiting the packets from your router to your computer(s).

    Inbound QOS/bandwidth limiting is useless IMHO.
  12. belliash

    belliash Addicted to LI Member

    You are wrong... it is possible to limit in/out-bound traffic and the internet connection (eg telephone line) is not used at all then. speedtest.net is showing maximum speed that equals the limit You set. You are also not able to get bigger transfer rates... The graph/chart in tomato shows real time speed (and its lower than limit You set). This help eg dividing connection for 2 computers (1mbps -> 2x 512 for 2 computers)
  13. Toastman

    Toastman Super Moderator Staff Member Member

    habskilla - The obvious answer is - just don't allow your incoming bandwidth to become saturated. And that CAN be achieved by using the QOS system in Tomato. Comments like "inbound QOS/bandwidth limiting is useless" don't really help anyone and deters them from experimenting. While not perfect, it's far from being "useless". If it were indeed useless, then what you are saying is that every single application using TCP on the web is useless, because they all use exactly the same TCP backoff mechanism to stabilize links.
  14. habskilla

    habskilla Network Guru Member

    You're miss understanding the results you're seeing. Tomato is reporting the packets that passes through, hence your reduced speed. BUT, the packets made it to your router, just not to your computer.
  15. habskilla

    habskilla Network Guru Member

    Where is QOS doing its job? At the router. You've already saturated your pipe to the router. You're just dropping packets from your router to your computer.

    I did say, IMHO.
    You hit the nail on the head. It's up to your APPLICATION to be aware. Tomato can't do it.

    Don't LET me stop anyone from turning incoming QOS on.

    Like I said, what do I know?
  16. SgtPepperKSU

    SgtPepperKSU Network Guru Member

    There seem to be two camps on this.

    One that says dropping packets at the router makes things worse: The other end will retry, taking up the bandwidth twice (obviously this part is true, but see other argument).

    One that says dropping packets at the router makes things better: The other end will recognize this and throttle the connection until losses are reasonable.

    It would be interesting if someone could authoritatively say which is correct.
  17. Toastman

    Toastman Super Moderator Staff Member Member

    habskilla, please try to understand. Of course the traffic has to reach your router!! How else can the router have any knowledge about what is in transit? Are you expecting it to be clairvoyant?

    That is a ridiculous argument.

    The traffic hits the router. There are several ways for us to determine whether the link is saturated, and then the router can do something about it. Delaying or dropping packets will cause the TCP backoff protocols to slow down the traffic. What else do you think is going to do it? Fairies?

    The whole point is - of course the server WILL and MUST resend any dropped packet. But it will not do so immediately!!! It will wait a defined period of time, called a backoff period, before resending the packet. In other words, it SLOWS DOWN. If that packet is also dropped, it will back off TWICE as long the next time, and send again. If also dropped, the next time the backoff is again TWICE as long, or 4 times the original - i.e. the backoff is exponential. Thus very quickly the link will stabilize at a new level, at which the congestion is relieved.

    The first camp, mentioned in SgtPepper's post, do not understand that the other end MUST[/B] back off in time before resending the packet. Thus they are bound to make the assumption that nothing of value can come from a dropped packet and that the inevitable result will be congestion caused by retransmissions. That is simply because, to put it bluntly, they haven't understood even the most basic concept of congestion avoidance by the use of a retransmission timer, one without which TCP, and the whole of the internet, could never work.

    You MUST understand that he backoff mechanism is fundamental to how the TCP/IP protocol works to stabilise an end-to-end link! And thus, if this mechanism's existence is denied (despite being in every TCP primer on the planet) then the only assumption that you can make is that all TCP links don't work because of congestion. Which is clearly NOT the case, because you managed to post your replies.

    All TCP primers will contain something along these lines:

    Now, how can anyone say that it is up to the application to be aware, in order for this mechanism to work? The packet is dropped. Period. It does not matter at all WHERE it is dropped or the reason for it, the originating server receives no receipt for that packet, so it will back off and resend it. Note - this has nothing at all to do with the originating client application which has requested, and is now waiting for, that data to arrive, it is purely down to the server.

    ALL applications using TCP are treated the same way. After a period, the backoff will reach the point where traffic will no longer saturate the link and packets WILL reach the router, which will pass them on to the application. Which will then send an ACK - the remote server then receives the acknowledgement, assumes the link is now stable, and the backoff will now STOP, and the RTO will use the new value (I won't go into the mechanism used for speeding the link back up when the path becomes less congested).

    It is this fundamental and necessary part of the TCP protocol's behavior that also allows the router to set an incoming limit. And it can be verified by anybody in minutes that the incoming limit works, merely by trying it.

    The whole purpose of that incoming limit is to slow down the link, to relieve the pressure on the bottleneck at the router, and allow packets to reach the router - and hence the originating application. It may therefore be used to free up some bandwidth for other applications which may otherwise be slowed down by that particular class. And it it part of the QOS system used on many routers.

    You may take this statement as authoritative. I am sorry if I am little terse but really, this is kids stuff.
  18. belliash

    belliash Addicted to LI Member

    Toastman thanks for information.

    BTW... incoming works ok... i have troubles with outcoming traffic (upload) ;/
  19. yoda

    yoda Addicted to LI Member

    I wonder... Maybe there is somebody who can help but how accurate,
    the only thing I can say is if there were some way to find that answer
    from end user to the co
  20. Toastman

    Toastman Super Moderator Staff Member Member

    No need to wonder, yoda. Once again, this is kid's stuff. Please stop trying to make a mystery out of something simple. Google is your friend, and there are literally hundreds of articles explaining the fundamentals of the TCP protocol on the web. There aren't any mysteries here - this is all common knowledge. If you don't understand it, then that's your problem. Don't try to force your own uninformed opinion onto everyone else, please. Enough damage has been done by others already.

    The server does indeed resend. But since it does so only after backing off, this duplication of traffic lasts for only a short time, and then it's all over once the link stabilises at the new level. I'm talking in terms of several seconds here.

    As I write this, I have several Torrent clients online, plus my own - currently I am downloading at 161.5kB/s. If the others close down, my own useage will rise to about 450kB/s. Other traffic is unaffected and websites open at normal speeds. Some users are watching IPTV, there are dozens using Messenger, and a few games players. This does not happen by magic - it happens because the QOS works, and the incoming class limits also work. So, there really is no need to wonder, just go ahead and experiment ! If your experiments fail, then it is something you did not understand, and the cure for that is clear. Go use google, sit down and learn something.
  21. Ragnor

    Ragnor Addicted to LI Member

    A lot of the blame for the misunderstanding around incoming and outgoing QoS can be put on this site http://vonage.nmhoy.net/qos.html which shows up near the top of google searches for QoS.

    The page basically says only upstream QoS has a real effect which is bogus if you care more about good performance than dropping a few packets (which would probably happen anyway when you connection is saturated).
  22. yoda

    yoda Addicted to LI Member

    I totally agree with you
    while ago i had the basic package plan thru att our family decided to go with a voip provider. but, with our trials it seemed voip was impossible with our basic package plan. it would constantly had disconnects with minor voice quality issues. however, this only occurred when one of our computer go on game servers or used most of the total bandwidth at that time i found a setting in our 2wire modem that helped the issue called traffic shaping fortunately when my line was provision the traffic shaping was completely different from my actual speed to advertise speeds. Making the long store short i did test thru my isp's speed test and had both voip and gaming possible with my basic package plan... however, i don't know if that is still true to this day b/c we upgraded our package plan
  23. yoda

    yoda Addicted to LI Member

    Does anyone know how qos knows when to limit set speed by user?
  24. belliash

    belliash Addicted to LI Member

    OK, but...

    1) it is much easier to limit outgoing traffic - this works - i upload data with eg 20KB/s when i could do that ~65KB/s, so about 3 times slower
    2) actually i have 512kbps upload
    3) when i had 1024/256 conenction it worked like a charm!
    4) I have not changed ISP. I have 1mbps Down and 256kbps up ADSL throught telephone line and i had a telephone also
    5) actually i do not have telephone - internet only - 2mbps down and 512kbps up throught telephone line (still ADSL - the same ISP)...

    What is more... Now works download limiting and does not work upload... but last time it was vice versa... upload worked like a charm, but when i was downlaoding i get over 260KB/s and none site worked......... STRANGE for me

    As You see on my very first posts i have also other troubles... Some solved, another discovered, so i think it is regarded with tomato :( Yesterday noticed that i cannot join any game server (Americas Army, rFactor, etc... none works - join time out). Ports are forwarded and seems to work... Two days ago everything worked ok, powered off, gone sleep, yesterday couldnt join any server, but web sites and FTP works...
    In day-to-day ussage i use Linux, and windows for gaming and software that do not work on linux. I noticed that when i try to enter tomato website on windows - it freezes at 'waiting for' and then i have connection timed out... but on linux it works well... no firewall, Kaspersky AV (enabled, disabled - no difference)...

    I am going to go insane... i have no f****** idead what on the earth is going on :/
    Just waiting for RAF 1.25 mod with a hope that this will solve my issues... but Victek do not want to release it :p
  25. Toastman

    Toastman Super Moderator Staff Member Member

    yoda, consider this simplified description . There is a section in the firmware called connection tracking. It is there because all outgoing data from a NAT router carries only the WAN IP number and there has to be a way for the router to know which computer on the LAN actually sent it - so that it can send the replies to that particular computer. So conntrack keeps records of all packets sent with their identifying packet numbers, replies to those packet numbers can be identified as to which machine they should be sent to.

    You will see from this, that the router therefore has the information it needs to also keep a tally of the incoming data and by timing it, calculate the incoming data rate. When the incoming limit set by the user is exceeded the router has the option of delaying or not sending acknowledgements. The remote server thus backs off and resends as previously described using the normal TCP mechanisms, and thus the link stabilises at that user-selected limit.

    Is that what you needed?

    ragnor, I agree, that article has several misleading bits as well as some rather silly diagrams. Articles written by those with only half an understanding of what is happening are useless.

    belliash, there will be no magic cure with Victek's 1.25 - the QOS is the same, and there wasn't a problem with the previous version. So if it doesn't work for you, then you must find out what is wrong with your setup and address it. Start again from the beginning is always a good idea.

    Victek will release 1.25 when it's ready, and it's been thoroughly tested. A little longer yet...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice