Using QOS - Tutorial and discussion

Discussion in 'Tomato Firmware' started by Toastman, Dec 24, 2008.

  1. Toastman

    Toastman Super Moderator Staff Member Member

    Here, the crawl class is choking most of the UDP/uTP traffic. It still works. I have to kill it or it will totally wreck all of the networks.

    Ratio of upload to download varies hugely. Can be 20:1 or as high as 1000:1. I work on the 20:1 figure normally, but have been unpleasantly surprised on several occasions. Try to do a test, see if your app is consistent in any way. Or use the 50:1 figure if you're a pessimist :)

    I've always addressed RTMPT port 1935, but it hasn't had so much effect. I think the belt 'n braces approach is the only thing likely to prove successful!

    I am still finding my downloads to be very much worse using uTP. The best I have done with the uTP taking over most connections was about a quarter of my normal speed.

    What are you finding?
  2. Azuse

    Azuse LI Guru Member

    I don't have a crawl, I simply have a default then everything I deem important is raised above it. That's why I'm asking :). Anything not in one of the 9 classes is ether p2p (pretty much always), some app that needs classifying or some app that punches through/uses upnp and can't be classed by port e.g. msn.

    1935... Perhaps it may be time to have my eyes tested...

    Seems I'm practically 50:1 :eek: (48.8:1) which makes sense since 1.3MB downloads usually result in 30KB up. I just never though about it before :chuckle:

    uTorrent aside I was wondering if there is any real benefit since some much of p2p, read ever client bar utorrent, is heavily tcp. You basically have 2 crawl classes, 9 is tcp, 10 is udp. I tried separating tcp/udp like this in the past with no effect at all (since nothing could be classified) but with so much tcp on my line there's no benefit, even if utorrent is more udp now.

    My uTP experience has been good, well the initial release was crap but they've tweaked it alot, at least with the 2.0.1 build although the uTP fixes means the connections per second has risen from 5 to 10 which probably explains the decreased time it takes for downloads to reach full speed, although they do reach full speed and stay there. If anything, it's more responsive to other traffic. However my isp doesn't choke or throttle traffic, my line gets it's full speed 24/7 so I wouldn't expect it to slow down. The two faults - vandals :( - over the past month traffic was rerouted and the torrents slowed themselves but games didn't so I would assume uTP is throttling itself to prevent congestion as it should.

    You've tried it on a dedicated line without any qos/connection limits scripts haven't you? Have you considered the possibility that utorrent may be slowing your traffic down because it's detecting high latencies (target latency of 100ms unless it changed) because of your isp? uTP will only be as fast as TCP if the line isn't congested and not necessarily yours just one of the isps router pushing up pings...
  3. Toastman

    Toastman Super Moderator Staff Member Member

    Well, all international connections from here generally have greater than 250mS latency, even neighboring countries. This is made worse than it could be due to government proxy servers (blockers) just before the main international gateways. This is actually the case in a great many countries around the globe. Could be the reason I suppose. But for many, the majority of seeds will always be from another country and the target latency of 100mS is in any case physically impossible.

    I did a test on a dedicated line with no QOS and no other traffic at all. Best download speed with TCP only, 14.5Mbps total, reached after 5-6 minutes, incoming stream is fairly level. Same test with DHT and UTP enabled, best speed after 20 minutes was 2.8 Mbps (averaged), as the incoming stream is congested and jagged with big troughs every few seconds. Switching this UDP crap on is effectively screwing up downloads. Even worse, it is swamping the link with no obvious benefit whatsoever. I see no evidence of uTP controlling itself. Rather the opposite. My main concern is the tremendous amount of BANDWIDTH taken up by uTP and DHT.

    And why is that? Referring to discussions by ISP engineers, the problem is that the uTP protocol, which GREATLY increases PPS (packets per second) rate at the same BPS (bytes per second, "speed") rate due to small packet sizes. It is the PPS rate which determines the load on the ISP's equipment. The engineers say when the uTorrent 2.0+ was out and installed by lots of users the ISPs equipment got greatly overloaded which caused failures, glitches and delays of service. That was not because of improved file exchange and increased user traffic, but just due to the decreased size and increased number of packets. They say that in spite of any claims to the contrary, uTP is a harmful thing that produces lots of unnecessary problems to ISP's. As a direct result many more ISP's have now resorted to blocking P2P who had not previously done so. We have these idiots to thank for that.

    more stuff from engineers who actually work with this crap:

    Look forward to severe throttling by ISP's...
    Wolfgan likes this.
  4. Azuse

    Azuse LI Guru Member

    You wouldn't happen to have the disk overloaded problem when using uTP would you?

    Start your torrents then once they start slowing down go preferences > advanced > disk cache > override automatic cache size and increase it to 2/300MB and see what happens. Once the cache fills mine slows down but only when I'm doing 3 or more torrents. Seems uTorrents cache likes uTP less than you :)

    On the plus side it's worse in the alpha so it'll (hopefully) get fixed soonish..
  5. Toastman

    Toastman Super Moderator Staff Member Member

    Restoring config to the same OR another router.


    Backing up and restore after an NVRAM erase.

    Most people know by now that you should never use your old (previous) config file when you upgrade firmware. Often there are conflicting settings and this can result in some very unpredictable results. Also, routers often behave in strange ways after certain events such as power loss or brownouts. Erasing NVRAM and restoring is a time-honoured method of fixing many of these hiccups.

    Therefore please note:

    • You must erase NVRAM after flashing new firmware versions and reconfigure to be certain that config is still ok.
    • You can't just restore your previous config file, as that would be exactly the same as never having erased NVRAM in the first place.
    • And anyone who has tried it will know that a config file from one router cannot be restored onto another. You have to do it another way.
    We often want to do this when we buy a new router, for example.

    There have been many discussions on how to do this, as normally it's not possible to transfer the config from another router onto a new one. You risk a bricked router if you try editing the file etc. (I know this from experience).

    BUT - this is a way to do this which works without the hassle of other methods suggested! You can save the router's configuration in a form where it can be quickly restored to the same or any other tomato router. You do need to "cherry-pick" only the variables that are absolutely necessary to restore.

    You need to save a file somewhere on or off the router, with the following command from the command line.

    nvram export --set >config.txt

    This outputs the complete setup in useable text form, each line beginning with "nvram set". This means it's ready to input to the router without having to add the "nvram set" to every line. You could do this by ssh or telnet.

    NOTE - you can only restore the whole config file to the original router. To configure a different router you need to cherry pick the important lines (see below)


    nvram set clkfreq=480,240,120
    nvram set wl0_country=AU
    nvram set wan_domain=freewifi
    nvram commit

    Or - better still:


    Don't make your life more complicated than it need be!

    If you type "nvram export --set" into the "Tools/System command execution box in Teddy Bear's builds (or any version using it as a base) and click "EXECUTE" - it will give you all variables onscreen. These lines can then be individually cut and pasted into the same or even another router's system box and executed. When you are finished, they can then be committed to NVRAM with NVRAM COMMIT, again in the same system box. If you don't commit them, they will be lost when you reboot the router.

    It's actually very easy to set up just the important basic config by hand, but cherry pick only the big variables from the text file that are a real pain to do manually (QOS rules, Static DHCP and Access restriction lists in particular).

    You can use GREP to find the settings you need.

    e.g: experiment with these to get the feel of it:

    nvram export --set | grep dhcpd_static
    nvram export --set | grep rrule (or rrule1 - and so on)
    nvram export --set | grep qos
    nvram export --set | grep qos_
    nvram export --set | grep qosl
    nvram export --set | grep qos_orules

    NB - This has the added advantage that you don't re-enter any old unused variables from some long forgotten setup. Often you can reclaim several kilobytes of NVRAM. If you are running short on NVRAM space, try doing this to clean up your space. I often recover up to 10K. If you want to keep the config cut and paste it into a text file on your PC. No need to make complex scripts, keep it simple.

    You could also use a small script to send the nvram variables file to a PC cifs share every day using the Scheduler e.g. "nvram export --set >/cifs1/routerconfig.txt". So you can't forget to make a copy of your recent settings.

    Nowadays I usually get the list up in a browser window, leave it open, and then cut and paste whatever I need into another browser window on the target router.

    It's useful to keep a little script in a text file somewhere so that when you need it, you can collect the relevant details easily by pasting into the system box and execute. Here is one example.

    nvram export --set | grep rrule1
    nvram export --set | grep qos_
    nvram export --set | grep dhcp_
    nvram export --set | grep dhcpd_
    nvram export --set | grep ddnsx0=
    nvram export --set | grep lan_hostname=
    nvram export --set | grep lan_ipaddr=
    nvram export --set | grep wan_proto=
    nvram export --set | grep wan_hostname=
    nvram export --set | grep wan_domain
    nvram export --set | grep pppoe_username=
    nvram export --set | grep pppoe_passwd=
    nvram export --set | grep http_
    nvram export --set | grep router_name

    Try it! It's a doddle... Less than 5 minutes to transfer everything I need and configure a new router.

    related command: nvram export --quote |grep xxxx

    see also:

    There are some occasions where this doesn't work because of line breaks. Take a look at this post:
    AlterEgo, AllenJ, Dallas and 7 others like this.
  6. Latvian_guy

    Latvian_guy Guest

    Toastman ->


    at first, thanks a lot for great posts here in Tomato section, I just installed and configured Tomato - I would never be able to do this without these posts.

    I have few additional questions:
    1) Which is the best way to measure "real" download/upload speed to be used in QOS as Max Bandwidth. I tried, but speed varies grately based on server location (ping).
    2) I am currently using Your "LATEST AND BEST SO FAR" QOS example setup. I find it very good, but I would wish to attain higher uTorrent speed (we have quite fast internet connection in our flat, and only 3-4 computers are ususally connected). Which is the best way to do it? I find, that if I only change inbound/outbound Class E bandwidth % (default in Your example is 1%, i tried 3% and 10% instead), uTorrent speed jumps for a few seconds (while Tomato is saving settings and restarting processes), and then falls back. Permanent change in speed is attained only by increasing inbound/outbound Max Bandwidth number in QOS settings. I feel that there must be a better way to make P2P faster...

    See uTorrent and Tomato screenshots:

    Thanks a lot!

    P.S. All connection users have turned off DHT and made advanced settings number (forgot name) = 5 as You suggested for uTorrent 2.0+.
  7. Toastman

    Toastman Super Moderator Staff Member Member

    Hi LG

    1) I don't know of a better way than to use the online speed tests. The figure you need will be from the FASTEST one of them - the most local maybe, but not necessarily. What we need is the speed at which your router can dump stuff into the ISP's network, obviously you must use the fastest one as that proves that you can upload at that rate - and you should ignore the lower speed servers. Since your ISP doesn't run speedtest, (or does he? - some of them do have a speedtest page on their own websites) we can only use whatever we have available, and choose the fastest of them. If you absolutely have no way to measure speed at all, then until you do, I'd recommend you set 66% of the ISP's stated bandwidths.

    2) Change outgoing rate and limit to whatever you find works - for example try 5%-80% and set incoming limit on that class to 80% or whatever. You'll find as P2P bandwidth goes up your other apps will become more sluggish, and it's up to you to make a compromise. If you try to fill your pipe with P2P everything else will pretty well grind to a halt.

    I believe I *am* seeing an increase on your graphs, there's a dip when the setting is changed, a sudden burst of backlogged data, then stabilizes with a small increase. Maybe it's just not noticeable, so go for drastic increases as I suggested, and get a feel for what happens. Then a few intermediate settings. Set a ping session to your ISP gateway (or other reliable site) going in a window and watch what happens to your ping times while you vary things.

    Also, setting upload bandwidth in uTorrent is quite critical, here - for example, setting 1 gets lower downloads, 10 increases a lot, 15 levels out, 20 drops drastically. Depends on what your upload bandwidth is. But of course, if there's more than one P2P user on at the same time, it all becomes moot. All of this stuffs adds together in the same class on the router.
  8. Azuse

    Azuse LI Guru Member

    Pardon this, is can't remember :redface: the zero setting on the inbound acts like 0-100 doesn't it? If the rules are independent from each other then how would class D know to allow anything less than 100% if class C started downloading? Or does none simply mean no limit is applied?
  9. Toastman

    Toastman Super Moderator Staff Member Member

    Yes, it ignores the limit. The incoming section is nothing more than bandwidth limits on the individual classes. The "maximum" figure is not a maximum limit at all, it is just a figure used for calculating the % limits for the individual classes.

    EDIT - see Porter's comments on the QOS system, the lack of proper processing on incoming data, and the lack of an overall bandwidth limit. Soon I will add IMQ ingress system to Tomato which will (hopefully) rectify this.

    EDIT - DONE, Tomato QOS now has IMQ based ingress with priorities similar to the outgoing section, and also an incoming bandwidth pie chart! (2012)

    Thanks to TIOMO for this icing on the cake!

    What does this mean? It means we now have tighter control over incoming traffic, and the ability to make much better use of the available bandwidth without having to set low limits on classes.
    Last edited: Jul 16, 2016
  10. Toastman

    Toastman Super Moderator Staff Member Member

    New Toastman K26 version 7420 beta for MIPS2 routers posted May 6th
    AmyKurt likes this.
  11. fun.k

    fun.k Addicted to LI Member

    Thanks to Toastman's confirmation + kindness, I'm posting here my "rare" QoS needs (rare as I don't do any p2p nor gaming) as it may help someone else too.

    Here's some rather concentrated background: in my studio we share a aDSL 16000↓/1021↑ (true speed) line among 5 desktop/laptop Macs, a Skype phone and the occasional friends paying a visit and accessing the net for emails, but mostly I'm trying to find a good way to do QoS for 5 computers. Here's what I'm trying to achieve:

    1. Ipevo S0-10 Skype phone having a constant 128Kbps ↑↓ at any time and take priority over anything else when we use it.
    2. E-Mail (mostly IMAP) being speedy
    3. sFTP Uploads. I regularly have to upload medium-size (100-300MB) files over FTP. I read that FTP is tricky since 20-21 are the control ports and the actual transfer happens over randomly chosen ports.
    4. Web browsing. Besides the occasional this+that we have to check deployed web sites utilizing flash/streaming technologies.
    5. Occasional IM chats, included only the ones I use.
    6. iTunes Radio streaming. Most streams I listen to are 64-128Kbps. I wouldn't mind if radio streams would stutter form time to time, this is low priority for me.

    Here's what I managed to come up with after reading the thread. Looks like I got something right as the quality of the calls of the skype phone seems much better than before. Mr Tman, if you spot anything silly, I'd appreciate your thoughts :)

    Since I don't do any p2p/gaming, am I still better off lowering the TCP/UDP time-out values? I've left them @ stock values at the moment.

    [​IMG] [​IMG] [​IMG]
  12. Toastman

    Toastman Super Moderator Staff Member Member

    I suppose the first obvious comment is that you have given DNS lookup a low priority which may delay most sessions.

    Skypephone has been given a much lower priority than all of - Browsing, Mail, HTTP file transfers, FTP .... Is that what you intended? It would be more normal to give VOIP a higher priority.

    Timeouts are OK esp. as you aren't using P2P.
  13. fun.k

    fun.k Addicted to LI Member

    uhmm, just a minute, class names aside it's all about the percentage range, isn't it? I reverted back to stock tomato firmware and after reading your example setups I tried to adapt mine.

    Your Service class has a 5%-20% Out & 10% In range, including DNS, Time, NTP, RSVP.

    Mine is named Lowest has 5%-25% Out & 16% In, including the same services

    Also the stand alone skype phone needs 128Kps both ways for optimal quality, given my dsl values i just created a class for it (A) and placed it up first in the rules. Does it have to be in the "Highest" and give it all available bandwidth no matter what?

    Just trying to figure, if you think i'm way off, i'll start afresh :)
  14. Toastman

    Toastman Super Moderator Staff Member Member

    Ah, OK. I think it's a misunderstanding about how the priority works.

    The PRIORITY of the classes is Highest, High, Medium, Lowest, Low, A,B,C,D,E in that order.

    It also appears that the last 2 or 3 classes actually have the same priority, we'll address that later - it seems to be something that was overlooked after some experimentation.

    As you placed VOIP in class A it gets a lower priority to the ones above (in your example, HTTP. FTP. Mobile-me. Browsing. DNS)

    Getting the priority right is the most important thing.

    After that - the actual position of the rule in the classification list is important - rules are processed from the top down. So if two rules have the same port, for example, but are otherwise different, then you need to check whether the first (highest in the list) rule is intercepting all the traffic instead of allowing some to reach the second rule. (I'm probably putting this badly).

    Here's what I'd suggest.

    Set default as D (as you did)

    DNS to Highest - my own personal choice is that DNS always has highest priority.
    VOIP to High
    The rest to suit yourself, but as a guide
    WWW Med,
    Mail Low
    FTP Lowest

    As for VOIP - I would initially set it to allocate immediately sufficient bandwidth to carry the outgoing voice channel, 20% (128 k) - then allow it to rise if needed to double (256K) or even allow it to rise to 100% - i.e. 20 rate and 40 or even 100 limit, as you see fit. (The danger here being that a P2P application might use this class and take all the bandwidth).

    Don't set a limit on incoming class A, or make it quite large, so that if necessary the VOIP class can receive a fast burst of delayed or queued data from the ISP.

    Try to limit total incoming data to 66% of maximum bandwidth so that under any circumstances there will be excess bandwidth available for VOIP. 66% has been found to be the best figure. You can experiment with it later to verify.

    Now - a little description to illustrate the reasoning behind the priorities.

    Lets start up and switch on our skype phone. A DNS lookup takes place quickly (class HIGHEST) then 128k outgoing is allocated and maybe a conversation is started in HIGH class.

    Someone else starts up a MAIL session, a DNS lookup takes place, still fast no matter if a lot of bandwidth was taken by several VOIP etc sessions, because it is in the HIGHEST class. Mail session begins, in LOW class - lower priority than VOIP so doesn't affect it. If VOIP ceases the spare bandwidth could be used for MAIL.

    Now someone starts a WWW session, fast DNS lookup in HIGHEST class, WWW session begins in MEDIUM class, doesn't affect VOIP, but will take priority over MAIL.

    And so on.

    Hope this clears things up a little.
  15. fun.k

    fun.k Addicted to LI Member

    Thank you so much for your time Mr T :)

    I'll start afresh with your current suggestions. I know my QoS needs are much simpler than yours, wish each user could rename QoS classes at will, as it would only simplify things.

    Runs to do some more homework. Cheers!
  16. KapaT

    KapaT Networkin' Nut Member

  17. Toastman

    Toastman Super Moderator Staff Member Member

    fun.k - the reasoning behind the class names is simple. The order of the traffic is pretty much the same for everyone. DNS first, latency sensitive stuff like VOIP/Games, etc down the list. Some people don't need to use certain classes - so just ignore them. Then the priority is taken by the next one down. You should use all 10 classes and just ignore the ones you don't need, leave them empty - it does no harm. But these class names make it easy to see immediately what is going on, the silly default class namers, such as the "lowest" class - when it ISN'T actually the lowest class at all - has confused so many people I really, REALLY hate them.

    I will be adding the ability to set your own class names one day - just have to learn how to do it, but it looks reasonably easy to do.

    [ ADDIT - April 2011 - I did it - we now have configurable class names! ]

    Take a look at these posts, and ones shortly before and after. They were talking about a similar problem but in the IP/MAC limiter section of Victek's RAF mod. It's nothing to worry about though.
  18. jochen

    jochen Network Guru Member

    I'm having some diffuculties using QoS for Voip. I'm using Toastmans rules, which are generally working fine, but I don't know how to implement voip clients in this rule set.

    I want to give my voip client the absolute highest priority. I tried it with class "service", but service has a bandwidth limit of 20%, which is 80 kbit with my maximum of 400 kbit. This is not enough for voip.

    What can I do in this case? Should I increase the bandwidth limit for class "service", or should I use some other class for voip?

    What I need is about 128 kbit outbound for voip with highest priority. No other service should disturb my voip communications. Even games should be of lower priority.
  19. vibe666

    vibe666 Network Guru Member

    hi Toastman, thanks for all your work. i spent some time setting up the QoS on my wrt54gs v4 with the stock tomato 1.27 and it's working fine aside from one thing, which is that when I VPN into my office (using the nortel networks contivity vpn client) i can't get RDP to connect to my desktop machine in the office.

    I have other local connections (PC's on my home LAN) and they're all working fine, it's just the ones in the office over the vpn that don't connect. it starts to look like it's going to go and then just times out.

    do you think there's a setting in the QoS that could be edited to allow it to work?

    if i disable the QoS completely it works fine, so there's definitely something there that's causing it to fail, i'm just not sure what it is. :(

    i'm using exactly the same settings as in your guide (the most recent one that you said should work best), just adapted the max up and down to suit my connection.

    do you have any ideas what might be stopping it from connecting?
  20. Toastman

    Toastman Super Moderator Staff Member Member

    Sorry - connectivity is bad.

    Jochen, it might seem that because VOIP needs to be prioritised, that you need to put it in the highest class. But in fact, the most important thing of all is DNS. DNS doesn't take up much room. Just adjust your QOS to suit, but bear in mind that we don't need to LITERALLY give the HIGHEST priority to VOIP. Leave the highest class alone. Put your VOIP in the next one. Remember it is a PRIORITY based system. The DNS will be served in a fraction of a second and will not affect your VOIP class.

    To keep it brief: you do need to classify your VPN connections and place them in - say - HIGH or whatever. After you have a VPN connection set up, QOS has no effect on anything passing down the tunnel. It is all treated as just VPN class. Of course you need to give VPN class enough bandwidth - you have to decide how much. Perhaps start with 100%.

    Other than this - I have little experience of VPN.
  21. vibe666

    vibe666 Network Guru Member

    weird, seems to be working now.

    just having a look at your firmwares there with the labelled classes.

    do you know if "tomato-ND-1.27.7314Toastman-VPN.trx" will work on a wrt54gs v4?

    and can I just flash it over the top of my standard v1.27 build? i'm guessing i can just save the config from the standard 1.27 build and restore it as the majority of the code should be the same?

    thanks again for your work, it's much appreciated. :)
  22. Toastman

    Toastman Super Moderator Staff Member Member

    As far as I know it should be OK, but of course I can't give a guarantee. You may get away with using your existing config but it is always advisable to clear NVRAM and enter from scratch, as if you do not, unexpected things may happen.

    By the way, Official Tomato got left way behind a long time ago. One hell of a lot of the code is quite different in today's mods based on Teddy Bear's mod. I regard the "real" tomato as something prehistoric these days!
  23. vibe666

    vibe666 Network Guru Member

    cool, sounds like i've a lot of work to do then (again), but i'll re-do the config from scratch just to be sure.

    can you tell me which firmware file you'd recommend for my wrt54gs v4?

    i think it has slightly less ram than some of the other wrt's so i don't want to stick the wrong one on it and kill it. is there a build with all the good stuff in it that will work?

    also, have you thought about the possibility of a QoS script to load all the relevant settings automatically without every user needing to put them all in themselves?

    like a mini config file, but just for QoS? maybe even have a download as well as an upload feature so people can edit their config files and try each others QoS settings to find the optimal one for them based on how they use their router?

    just an idea. i'm all ideas and no actual programming talent unfortunately, so these are all things way outside of my area of expertise. :(
  24. Toastman

    Toastman Super Moderator Staff Member Member

    That router is pretty useless in my opinion, it has too few resources. One of my customers has some and has been regretting it ever since. She even paid MORE than the GL for a pile of useless poop.

    Normally, I think I'd stick to Victek's original RAF 8515.2 or the modded Toastman 7515.2 version - which also has wireless connection rates which I backported from Teddy Bear's firmware as well as labelled classes for the QOS example.

    Even later Toastman versions have the improved QOS and a host of other mods, incoming pie charts, per-IP connection data and graphs, etc.

    I haven't given any thought to the QOS script idea. The purpose of the QOS thread and example was really to try to help people to understand how QOS can work, and then change it to suit themselves.

    In future I may add these rules as part of the compile as a default. Then if someone already has a favorite configuration, the only thing that will change are the class labels. But if they erase NVRAM, then the rules would appear. New users would see it right away.
    Last edited: Jul 16, 2016
  25. Toastman

    Toastman Super Moderator Staff Member Member

    New class labelled version 7421 for MIPS 2 (RT-N16 etc) with mini DNLA posted, compiled from Teddy Bear latest sources.
  26. x-demon

    x-demon Addicted to LI Member

    I think if i have all traffic encrypted with openvpn client on router, qos is useless for me?
  27. Toastman

    Toastman Super Moderator Staff Member Member

  28. occamsrazor

    occamsrazor Network Guru Member

    In the way that you mean, if ALL traffic is going over the VPN, then probably yes.

    On the other hand if you have other machines at home doing stuff (e.g. bittorrent) while you are out, and want the remote OpenVPN client to have priority, then QoS is still useful. I do this by assigning priority to the OpenVPN port number, also by assigning a static IP to the remote client and giving that IP priority.
  29. ndoggac

    ndoggac Network Guru Member

    Toastman, great article on QOS. Dispels a lot of myths...much appreciated!

    Quick question for you...Since QOS classification rules are checked top to bottom, does it make sense to arrange your classification rules in order of number of expected connections (largest to smallest)?

    For example, P2P typically creates hundreds or thousands of connections...would you want to put that classification rule first in your list in order to catch the most connections as quickly as possible so you don't have to perform as many rule compares? In your example, your P2P classification is last so for 1000+ connections your performing many needless rule compares.

    Does this actually affect router performance at all, or does the rule compares take up so little cpu it doesn't matter?

    Thanks again for the great write-up!
  30. rhester72

    rhester72 Network Guru Member

    The problem with P2P is that it is almost impossible to explicitly class, thus the catch-all rule at the bottom. You are right in that it is wasteful to check (and negate) all the QoS rules for every P2P packet, but there isn't really a better way unless P2P protocols tag their content in some recognizable way (and that is not at all likely).

  31. Toastman

    Toastman Super Moderator Staff Member Member

    Agreed with Rhester72. We don't have a rule for P2P - that's the whole point! Letting it bypass all the OTHER rules and fall into the default class, is the only way to catch P2P. Even then, because P2P clients have been designed to be sneaky, they will often attempt to use other well known ports, anything - even those under 1025 (which is why the original default QOS is even more useless than it looks) and will therefore often show up in other classes, we just have to live with that, unless we can find another way to identify and classify it.

    You can spend a lot of your life worrying about what order to place the rules, but it often isn't worth the trouble to be too fussy in practice. For those who have very high speed internet access, at some point, the router will reach a performance limit when running QOS and you will either have to turn it off, put up with the reduction in throughput, or buy a much faster router (which will undoubtedly become available).
  32. ndoggac

    ndoggac Network Guru Member

    I see, thanks for the feedback. I guess I cheat a little on my own network, since I have a 24/7 seedbox, I just set the first rule to take everything from that MAC address and put it in the P2P class. There are some other services on there, but they are LAN services, so not affected. I was just curious if I set my MAC rule to the last in the list if it would make a difference in performance, I guess it does but probably not much.
  33. rhester72

    rhester72 Network Guru Member

    If you are 100% certain that the only P2P traffic you will ever encounter is from that MAC, and it represents more than 50% of your total traffic, by all means put the MAC rule first, as it will certainly reduce QoS processing overhead.

  34. jnappert

    jnappert LI Guru Member

    Hello Toastman. Nice Tutorial regarding QOS. I am running your last example sucessfully on my WDS-Clouds (7 APs) and finally - i understood ;-) (hope so)

    I am using an 1024/1024 SDSL Line, so i set my inbound-limit to 1024 and outbound-limit to 800. Is this correct or should i use 230 for outbound.

    BTW: Is there a beta14 EXT-build with labeled classes available? I didnt find an updated branch in git and you only uploaded standard build on your ftp.

    Thanks for your work ;-)
  35. Toastman

    Toastman Super Moderator Staff Member Member

    Greetings jnappert. You've got the limits right, 800 is OK. Do measure your speeds though, don't rely on what the ISP says it is.

    There will be an EXT version, but I had a compile problem. Fedor's pointed me in the right direction to sort that out though, so hopefully will be available later.
  36. Azuse

    Azuse LI Guru Member

    Well I've been tinkering with my line (again) and squeezed another few meg out of it. It's now 1400/13500 sync, actual throughput being 1240/11800.

    I'm curious where you'd set the limits if it were your line. Bearing in mind that controlling the inbound with outbound is not really an option (I'd have to sacrifice 4/5 of it) so there has to be an inbound limit (95/90/85%?). Mac limited, so restrictions will be adhered to. Thoughts? :)
  37. Toastman

    Toastman Super Moderator Staff Member Member

    In my own condos I'd probably set 1000/12000 but your situation is probably quite different. Remember there is no overall limit in Tomato's incoming settings - only the individual class limits. The max figure is only used for calculating the percentages for the classes.

    I'm currently getting 1000/15700 out of my 1000/16000 ADSL which is very good. Of course, that is only on local test sites. If only international bandwidth were consistently high it would be great.

    You can also think about using the bandwidth limiter in some mod's fiormwares, to artificially set a maximum incoming bandwidth, as Tomato doesn't have one of it's own.

    EDIT This might be of interest to some people - by using a different modem (Huawei MT-880 in bridged mode) my download speed is actually now 17,300 Mbps on the same line. For some reason this modem is a real winner.
  38. Azuse

    Azuse LI Guru Member

    Yes well, if I set the inbound to 12000 (11800) then the second anyone started downloading a large file it would choke up. I guess the real problem is I actually have to use the inbound qos because I'd have to sacrifice too much outbound to control things that way.

    I don't think a happy medium exists :wacko:
  39. thalaivar

    thalaivar Networkin' Nut Member


    I recently switched my internet service provider and since then have been having some issues with my QoS setup.

    The primary goal is really to get a decent setup for four things.

    My spouse works from home 3 - 4 days a week, so the VPN traffic needs to be stabilized to a point where work does not get affected. Currently, the VPN connection gets dropped atleast 5-6 times or even more during the day. The VPN can be setup through wireless or be wired.

    The main phone at home is VOIP so that needs to work. Presently, I have seen cases where the phone will ring but the called party will not be able to hear anything and vice versa. This happens when the VPN is active but can also happen when the VPN is inactive.

    Playing videos from youtube and other sites is also quite frequent at home so that is something that also needs to be a reasonable experience.

    Once/twice every week, we do use the webcam with yahoo/skype with family/friends.

    Lastly, in general the overall web experience should be something that works.

    We do not do any kind of P2P, or online gaming so that kind of traffic is non-existant.

    My connect speeds vary quite a lot, the downlink can be anything from 10.8Mbps - 16Mbps and the uplink can be anything from 2.5Mbps - 6.8Mbps.

    My setup is Cable Modem ---> Router ---> Switch ---> Switch ---> Devices.

    Both the switches are Gigabit Ethernet switches. There is very limited traffic on the network inside the home as I am in the process of wiring up the entire house. The first switch is a D-Link DGS-2208 and the second one is a TrendNet S50-TXE.

    All the connections are made over CAT-5e cable to make the best use of gigabit ethernet.

    The router is a Buffalo WHR-G54S that has been flashed with Tomato 1.27 firmware.

    My first question is, based on the layout given could my placement of the devices be the cause of my voice quality issues and the VPN dropping?

    I did read your thread a couple of times but because of the emphasis on P2P, I am not sure if what I have done is correct.

    Any pointers on what I am doing wrong would be greatly appreciated.

    Attached is a screenshot of my QoS settings as done on the router.
  40. thalaivar

    thalaivar Networkin' Nut Member

    Forgot to attach the files.

    Attached Files:

  41. jnappert

    jnappert LI Guru Member

    Hello Toastman.

    Did you find the time to make a build with labeled classes based on beta16 or could you point me to the place in git, where i can do this myself?
  42. Toastman

    Toastman Super Moderator Staff Member Member

    Hi thailavar

    Sorry to put it so bluntly, but your QOS setup is esentially useless. You do really need to read through the thread to understand what to do, because if you had read even the first page, you would not be using the original Tomato example QOS. In essence, hardly anything in your setup makes sense so I won't comment any further.

    The best thing to do would be to go to the QOS example here:

    Key it all in. See how it works especially the bit about default classes.

    Better still, later versions of Toastman Tomato have the rules built in, they will appear if you erase nvram after flashing. Reconfigure the router manually afterwards.

    Then mod it as you see fit for your VOIP by adding a new rule. For VOIP it is important to limit the incoming data to around 66% of your maximum bandwidth. Once working, you may be able to raise it. There have been several posts on the QOS thread about VOIP by people that use it, look back in the posts a bit.

    Good luck!

    jnappert, no. I have a problem compiling the new code additions, I have not had time to sort it out yet. Sorry ! You can however just use the new git RT code and then merge tomato-1.27-Toastman into it. There will be one file that needs fixing to remove duplicated entries, but then you can try it. The problem may have been fixed now, but I have been too busy to try again.
  43. thalaivar

    thalaivar Networkin' Nut Member


    Thanks for your candid response. Even though you might feel I did not read your posts, I did read them and also understood that the Tomato rules were useless.

    What I did not understand was if I need to flash my router with a new firmware or a modified firmware which corresponds to all the different settings that you display in the images?

    If that needs to be done, then I'm guilty of not reading your posts.

    Also, if a new firmware needs to be uploaded to the router should I pick any one of the first two from this url,

    My apologies for having to make you tell me to read your postings again.

  44. Toastman

    Toastman Super Moderator Staff Member Member

    No worries

    The classes are labelled in the version I made, but they are in the exact same order - you don't have to run anything special. Having classes labeled makes it easier to understand the examples though.

    Just convert highest=service, down to lowest=crawl

    If you do want to flash the one with labeled classes be sure to use MIPSR2 for RT-N16 and MIPSR1 for WRT and other routers. Later, I will try to post versions of this firmware without the labelled classes too, so people don't have to use them if they don't want to.

    Good luck
  45. jnappert

    jnappert LI Guru Member

    Thank you. Tried it yesterday but had no sucess merging. I checked out origin/tomato-RT and used git-merge using your branch with no sucess. Could you point me to the right syntax?
  46. Azuse

    Azuse LI Guru Member

    You know, I've been playing with uTP the past few weeks, and while I haven't been able to do anything very useful QOS wise dumping UDP traffic at the bottom stops DHT wasting bandwidth :).

    However I was always under the impression that traffic took the default class (D in my case) unless there was a rule for it in a higher class. When I add a rule for UDP in class E I expected it to do nothing, since the default was higher but I've got a lot of UDP traffic in there now. If Tomatos QOS operates in ascending order why is class D, my default class, not being given priority over class E (why is anything in there at all)?

    Have I missed some trick as to how QOS works?
  47. rhester72

    rhester72 Network Guru Member

    The rules are evaluated in listed order (top to bottom), not by class.

  48. Azuse

    Azuse LI Guru Member

    I know, I just found it odd that a rule lower than the default class would override it.
  49. fei2010

    fei2010 Networkin' Nut Member

    I am following post 135 to set up the QoS on my tomato. I've noticed that after the setup the download speed is dramatically dropped, to a point that even watch youtube is choppy.

    Here is my setting, the speed test(with Qos off) is 4800kb/s download and 630kb/s upload. I put 4500kb/s as max inbound and 500kb/s as max outbound. See below screen shot for the settings.

    I've changed the p2p crawl from class E to class D, which enable me to watch ppstream, and also brower internet without problme.

    Now here is my issue:
    1. watch youtube is choppy even not the HD one!
    2. speed test shows download speed is only 420kb/s, 1/10 of without QoS!

    could someone give some advice what I am doing wrong or this is expected?

    Basically my goal is that when someone in my family is watch video online(ppstream, youtube etc) or when there is p2p downloading, I am still have to browser internet fast enough, telnet works ok, VOIP call works ok, VPN to my company works ok.

    Thanks in advance.
  50. Toastman

    Toastman Super Moderator Staff Member Member

    fei2010 - Just had a quick look. You have no priorities for multimedia or youtube - you only have a partial QOS from post 135. So I would expect them to become slow and choppy.

    Is the default class set to D? Is "prioritise" ACK set to OFF?

    What is your ISP stated speeds for your link? Did you measure the speed with online test ( or similar) and enter that figure - less, say, 20% - for the maximum outgoing speed? You probably did - just checking.

    What speedtest shows is what it is capable of doing in the presence of other traffic, a QOS rule for HTTP (which is what the speedtest uses for the test), and really means very little. You must conduct a speedtest with no other traffic, QOS off, otherwise the result is meaningless.

    jnappert - don't try to use the command line is my advice, use the GUI's. Gitk and GitGui together make things very easy andf very fast. Smartgit looks good too, but can do entirely unpredictable things if you don't know exactly how to use it, it's bitten me several times.

    Azuse, this is a problem with your understanding of the word "default". Anything you don't address takes the default setting. That's why it is called a default. But you set your DHT to class E, so it no longer assumes that default.
  51. fei2010

    fei2010 Networkin' Nut Member

    Thanks Toastman! Yes the priorities for multimedia or youtube is missing! I added and but forget to click the "save" button, my stupid mistake.

    Yes default class set to D, "prioritise" ACK set to OFF.

    the ISP stated 6mb download, I used and did put less 20% figure on the max.

    Do you think the slow of speed test after apply Qos is normal?
  52. occamsrazor

    occamsrazor Network Guru Member


    I previously had my own QoS setup that I think worked fairly well, but I thought I'd give yours a go... I've saved my old config in case I want to go back to it.

    First off, mine is a home setup with a few machines, I use VOIP a lot, and Bittorrent almost continuously. Using Teddybear K26 latest beta MIPSR1 on WL-500GPv2.
    I have setup everything exactly as per post 135. I haven't done much testing yet but it seems fine. I'm not using your named classes build, but did make sure to put everything in the right classes.
    My line is adsl 1MB up / 4MB down, and I have outbound max set to 850kbits and inbound max at 3400. I'm using TCP Vegas with 2-6-2, but maybe it's worthless...

    I just wanted to ask about adding some extra rules that are maybe specific to my setup, and most importantly which class I should put them in, or how I should rearrange some of the classes (maybe)....

    1. I don't play any games ever, so I guess I can delete that class, or use it for the below?

    2. I see in your setup that SIP is classed the same as most "media" e.g. online video. I want SIP to get VERY high priority and as much bandwidth as it needs - even if that means everything else slows down. Basically when I am using SIP I'm not doing anything else. So I'm assuming I should separate 5060-5061 out and raise it somewhere?

    3. On my setup SIP UDP 5060-5061 is used for SIP signalling, but I think the actual voice traffic is UDP 16384-16482, as this is what is in my VOIP phone's "RTP Parameters". Can I put these in a higher class, if so where?

    Update: I just did a test and watched the connections while a call was in progress.... the traffic was on a couple of ports in the 16000 range and did not get picked up by the SIP rule. That's why I think better to classify by LAN IP address.

    4. Previously I also had a rule to prioritise traffic for this SIP hardware device (a Linksys SPA-3102), based on it's static IP e.g. source IP = How should I fit this in? Given that this is the most important device and the main/only source of SIP traffic, it seems like this would be a good way. I do also use SIP on my iPhone via WiFi (see below) but less frequently.

    5. I also have an iPhone using the WiFi with static IP which I would like to prioritise at the expense of everything else, or at least at the same level or just slightly less than SIP.

    6. I often remotely log in to my different machines and devices using various remote web interfaces each of which has a different specific port. I'd like to give these a very high priority as when I'm logging in remotely that means all the other stuff happening at home isn't that important. I was thinking to put this in the "remote" class, but I see it only gets 5-20% of bandwidth which seems like it's maybe not enough.

    Another related question... for remote access apps, if I am connecting to my home machine remotely using say port 5103, and the home machine is serving me a web page on that port etc, and I wanted to prioritise it, would I do this by classifying 5103 as source port, or destination port? The connection is coming IN from the remote machine, but I guess all the traffic is going OUT from my home machine to the remote machine.

    Finally, I just noticed a couple quirks on your (post 135) setup that may be errors or maybe there is a reason for it....
    Port 5100 appears twice - once in the media classification as (QT,Camfrof,VLC) and again lower down in the messenger class under MSGR3
    The same goes for port 5061 - it appears twice, once as SIP in the media class, and lower down in the MSGR1 class.

    Thanks for all your great tutorials....


  53. Kisch

    Kisch LI Guru Member


    I have these settings and it is working nice for me. Look at VOIP and NAS section :) For access to NAS from web I use 5000,5001 and 443 ports.

  54. occamsrazor

    occamsrazor Network Guru Member

    Re: post 272 above, these were my old QoS settings.... see attached.
    PS - I wish this forum would raise the limits on attached images.... 640 pixels maximum is way too low.

    Edit: Sorry I just realised those were actually from a much older setup I had a year ago. Anyway... it's the customisation of Toastman's post 135 setup that I'm interested in...

    Attached Files:

  55. Toastman

    Toastman Super Moderator Staff Member Member


    Once you have the QOS switched on, the speedtest result doesn't really mean much, no. You must stop other users from accesing the router while you do the tests, and turn off QOS. Otherwise the speedtest will itself be placed in some class you have in fact limited.

    I get this problem with users who try speedtest, not knowing that this occurs, and the results will vary with the traffic on of the router.

    What is really important is the speed that the apps load at, once QOS is on, don't worry about the speedtest. By now you should have developed a "feel" for it.
  56. Toastman

    Toastman Super Moderator Staff Member Member



    TCP Vegas doesn't work on any connection passing through a router. Turn it off as it can do some odd things..

    Delete, use it for whatever ..

    Yes, raise it up, you could move others down since you have the games class empty and rearrange the priorities.

    I'd try them in High ...

    If you know which machine it is then it's good. In my case I often have a hundred plus machines online, all or any of which could be using SIP or whatever, on any port, and over which I have no control at all. I have to try to address them by QOS. You have a free hand tho, if you know the ports to prioritize, since they are your own machines it's possible for you to also reconfigure those machines.

    As above

    Don't get too carried away - they will usually work OK in the same class as their bandwidths are not that high. Just make sure there's enough bandwidth for 2 simultaneous connections.

    Yes. One thing I am seeing here is that you have several things you are now mentioning you want high priority for. You really do need to define which is highest and stick to it. If you put too many things in a priority class the net result is none of them has a priority. (Gilbert and Sullivan - the Gondoliers .. "if everyone is somebody, then no-one's anybody"

    Source port - It is better to think of "local" rather than "source" I think..

    Yes, it's deliberate. It makes a rule self-sufficient. It makes it easier to turn off (disable) a class, move it up or down the list, or delete it without having to go re-define everything. As it is, although a Messenger rule is called for, it's actually diverted before it gets there and given a higher priority. But if that rule were deleted, it would still get addressed by the Messenger rule, so it would work. It's just a safe way to create rules ...
  57. occamsrazor

    occamsrazor Network Guru Member

    Thanks Toastman.... I think I will experiment with deleting the Games class and putting all my VOIP stuff in the High class.

    Another question though.... I notice the percentages in the Highest and High class are fairly low i.e. 5-20%. This runs very contrary to the default setting of Tomato which if I remember is Highest 80-100%, High 10-100%. In my case the 5-20% on a 1MB outbound line gives 42 - 170 kbit/s. My VOIP alone needs around 100kbit/sec.

    Can you explain the reasoning around this relatively low figure for the high/highest categories, or direct me to a previous post that does? I don't doubt you, am just wondering how it works. Would I be safe to change it upwards, or how else should I guarantee bandwidth for my VOIP?

    Do you have any idea how best to classify Skype? Are the L7 rules effective? Or port-based ones? I used to use both a port-based rule for the Skype port and an L7 rule, high up on the list, but I'm not sure if it was effective or not.

    Also, when I'm looking at the connections graphs under your setup, I'm seeing a lot of "unclassified" connections. I am pretty certain these are mostly bittorrent as most of them have 6881 as src or dst port. I tried to add a rule at the bottom TCP/UDP Src or Dst Port = 6881 > Class D. However I still see a lot of these connections unclassified. What I don't understand is if I can see the src or destination port as 6881 in the Graphs/Details.... then how come they're not getting classified? I hasten to say I had the same with my previous setup.

    Sorry for all the questions!! Thanks...
  58. Double

    Double Networkin' Nut Member

    Hi, i think is down and btw. i think im speaking for alot of people who are just lurking in this thread, that we really REALLY appreciate the work your doing Toastman. Thank you very much.
  59. qwerty01

    qwerty01 Networkin' Nut Member

    Hi. I'm using wrt54gl with tomato 3 years. QoS work fine but...
    I change ISP (20 Mb/s down, and 8 Mb/s upload). This ISP have local hub dc++. How configure tomato that it will not be slow down speed download/upload? If i want have big transfer i must turn off QoS. :(
  60. fei2010

    fei2010 Networkin' Nut Member

    Toastman, I really appreciate your help!

    where can we see the distribution of the inbound traffic? it is not in the graph section. In general how to do fine tune QoS?
  61. Toastman

    Toastman Super Moderator Staff Member Member

    Use the games class for VOIP?

    In my case VOIP isn't given any priority as we don't encourage people to use our bandwidth for phone calls in a shared environment. So it's sufficient. You can up this to suit your own use. My experiments in actually giving it priority have actually been quite positive, and I am adding VOIP to the Games class - which will now become Game/VOIP. Another important reason not to allow that class to take 100% if it's not needed, is that on occasions P2P can and will use those ports, and perhaps hog your bandwidth, without you realizing why.

    Not too confident of this, as I don't use skype myself, but the L7 rules seem to work quite well these days. The skype-to-skype L7 rule is OK, but the skypeout filter (to POTS telephone) allows so much P2P past that it's useless, and has to be disabled. You just have to experiment. Note - on the ASUS RT-N16 quite a few L7 rules can be loaded without any problem, but on a WRT54GL etc. too many will result in a noticeable slowdown of the router.

    Normally most unclassified connections are incoming attempts to connect to ports that have already been closed. Remote P2P clients etc. keep trying to connect regardless. They eventually give up but there will always be more opening. There's nothing you can do to stop this, it's just how it is.

    Be aware that some of these may be TEREDO or other connections associated with IPv6 (windows Vista, and 7) which is enabled by default. You should perhaps disable it on your PC by command line:

    set state disabled
  62. Toastman

    Toastman Super Moderator Staff Member Member

    double - just have to be patient. is a server on my main desktop machine. If that's off then it won't be available. Having said that, usually it will be on 24/7.

    fei2010 - distribution of the inbound traffic is something that Tomato lacks - and it would be really nice to have it. Tomato's incoming QOS is incomplete - there's no overall bandwidth limit and no incoming pie chart or graphs. Anyone fancy doing it?

    Qwert - you're on your own with that one!
  63. fei2010

    fei2010 Networkin' Nut Member

    one more question, how about the traffic start from router? do I need to set up rule? I have asterisk installed on the route and want to allocate necessary bandwidth to allow smooth voip calls.
  64. Toastman

    Toastman Super Moderator Staff Member Member

    No idea about that. I believe it might be difficult - there have been several similar posts recently, take a look at the forum posts.
  65. xemino

    xemino Addicted to LI Member

    ty for your qos rules toastman.
    unfortunately i can't play any team fortress 2 nor bad company 2. it's lagging like hell. it worked fine w/ my dirty old qos setup. any suggestions?

    you might wanna check your machine:

    this is what i get if try to access
    why don't you use dropbox to host the files?
    you can create a public folder w/the newest beta, just like your current setup, except that it's more secure for you and the visitors and it will be available 24/7

    should you need a dropbox account, feel free to give me and yourself more space:
  66. Toastman

    Toastman Super Moderator Staff Member Member


    If your games lag like hell, obviously any QOS rules you have for the games and correct control of any others are not working properly. In my experience most games can be successfully prioritized by port. There are also some L7 filters for games that work.

    Re the server ... I am not responsible for Google's blackmail. There are no trojans on this computer. And most certainly not in the very small directory FIRMWARE. Neither are there 288 pages and Google has no access to anything outside this directory. Therefore I would suggest that Google's warning is total CR*P because it is actually referring to the complete domain which consists of many, many websites. Many of them are porn sites and wares distribution sites. But to imply all websites are malware - that's ridiculous and unnecessary scaremongering - one might even call it sabotage. Google have gotten too big for their boots with this and with their cookies and espionage. Pretty soon I expect they'll start blocking anyone that doesn't allow access to their damned adservers. Even now, 50% of web pages are slow to load because they are waiting for googleanalytics ... thanks a bunch Google for trashing the internet ....

    I have no idea at all what this line means or how it is relevant "Malicious software includes 38 trojan(s). Malicious software is hosted on 1 domain(s), including " ---- WTF has this domain got to do with

    Feel free to contact them if you wish. Nevertheless, is now taken offline as of 15th July 2010, because I don't need this BS :)

    Screw Google.

    Anyone who wishes for latest compiles, feel free to compile your own versions with class labels from the source code I have already uploaded to I won't be uploading to any other site, nor will I be spending any more time on this - yhh.

  67. serangku

    serangku Guest

    Hi ...

    i have some question and hope you can give more bright to me ...
    in front of my wrt54gl is transparent proxy server ... so my router is client that proxy
    how about qos value should i have use for inbound and outbound ?
    as far i know main connection is 3 mbps down and 512 kbps up

    i ask this because it look wrt54gl+tomato act as primary gateway for connection
  68. onehomelist

    onehomelist Addicted to LI Member

    I use L7 flash blocking to limit youtube traffic on my network. But it isn't effective, seems like the rule is doing nothing. Is it possible to have a rule which lets youtube use only 50% of my bandwidth, however many users try to access it. Because youtube traffic uses port 80, is there any way to prioritize usual http traffic (web pages) over youtube traffic?
  69. bangkokiscool

    bangkokiscool Addicted to LI Member

    Can QOS control Spambots?

    A couple days ago something happened to me that has never happened before. Time Warner shut off my cable modem! When I finally figured out what happened (swapped out two routers, and then tried another cable modem that worked fine) I got a hold of TWC and they said they had suspended the road runner account because a computer on the network had contracted an "undetectable" virus that makes the PC send a continuous stream of emails. They said that the virus can be contracted when someone sees a popup on Facebook or Myspace that requests the user to install a virus software. Anyway, they said I have to bypass the router completely and hook up every client PC to the modem one at a time, run "" and then check if that IP is blocked at "". After 24 hours I can run another PC into the cable modem until i find the offending computer.

    Doesn't make much sense to me. If a client PC is indeed infected, how would TWC know the IP address of the PC if the PC is getting an IP from Tomato?

    Anyway, has anyone heard of something like this before? TWC says if they have to suspend the modem 3 times they can block the account forever (the rep admitted they've never actually done that). Can a QOS or firewall script take care of a spambot problem like this? I have Toastman's QOS post #135 settings as well as firewall scripts running. Level 3 tier support said this particular virus is not picked up by any antivirus software and the only fix is a total reformat of the client PC's hard drive.
  70. rhester72

    rhester72 Network Guru Member

    I'd recommend blocking port 25 outbound unless you're actually running a MTA.

    Yes, this _can_ happen, it happened to me. However, the compromised machine didn't have to be reformatted, and it's completely undetectable...though it does stealth from the vast majority of known AV software using explorer spoofing. Rootkits can be nasty stuff. Fortunately, Vista and 7 64-bit's driver-signing model make this form of attack all but impossible, so there's hope for the future. ;)

    If you're running XP (32-bit), there's a very good rootkit detector out there that might be able to help, but since it does present some risk, I'd prefer not to discuss it on-forum. PM me if you want to explore that further.

  71. bangkokiscool

    bangkokiscool Addicted to LI Member

    Thanks for the tip. If I block 25, does that mean no one will be able to send outgoing mail unless they use web-based mail? And how do I actually do that, just drop 25 from the QOS rule for mail? That still leaves 465,563,587,110,119,143,220,993 and 995 on toastman's example, could the mail be coming through those ports instead?
  72. rhester72

    rhester72 Network Guru Member

    Hrm...I'm assuming that this blast of outbound e-mails are going direct and not through your normal mailhop (something along the lines of If that be the case, it should be reasonably simple to block "unauthorized" outbound e-mail via port 25 (I've yet to see a rootkit not use 25, since it's the SMTP standard port and easiest to use), since you can block all outbound connections to that port where the destination IP *isn't* the one you're actually using with your clients.

    It has nothing to do with QoS - it will be a hand-crafted set of iptables rules that go in the Firewall script section.

  73. bangkokiscool

    bangkokiscool Addicted to LI Member

    Ah, got it. OK I'm willing to try it. Right now the only scripts I have are in the firewall section to limit connections (again, per toastman's suggestions). Can you give me a pointer on what to type in to restrict unauthorized use of port 25? Sorry, I'm not a programmer just a harried and hassled landlord with college student tenants who should know better! Thanks so much (and sorry for taking this thread OT since I know it's really supposed to be about QOS).
  74. rhester72

    rhester72 Network Guru Member

    Do you know the name or IP address of your ISP's outbound SMTP server (i.e. the one everyone "should" be using as their mail relay)?

  75. bangkokiscool

    bangkokiscool Addicted to LI Member

    I'm going to guess that it's
  76. rhester72

    rhester72 Network Guru Member

    I'd advise something like the following in the Firewall script page:

    iptables -t filter -A wanout -p tcp -d --dport 25 -j ACCEPT
    iptables -t filter -A wanout -p tcp --dport 25 -j LOG
    iptables -t filter -A wanout -p tcp --dport 25 -j DROP
    This will log all outbound attempts that fail, so you should be able to isolate (by IP/MAC) which machine is infected as well (or attempting to legitimately connect and in need of another rule!).

  77. bangkokiscool

    bangkokiscool Addicted to LI Member

    Thanks a lot, I'll try this!
  78. ray123

    ray123 LI Guru Member

    Could you post the output of this command: "nvram export --quote | grep qos_ " for us. Then somebody can directly import these settings, instead of having to go thru the GUI.
  79. Lassik

    Lassik Networkin' Nut Member

    Hey Ray I ended up doing it all manually. Here's an export for anyone who wants to import it. It's not the "exact" word for word layout but it's all there functionality wise. The class names are obviously different from Toastman's named builds. The conntrack settings are not included so be sure to set them manually.


    Contents of file:
    "qos_orules=0<<-1<d<53,37,123,3445<0<<0:10<7<DNS,Time,NTP,RSVP>0<<6<s<80<0<<<1<Remote Web Access>0<<-1<d<11999,2300:2400,6073,28800:29100,47624<0<<0:50<8<Some well known games>0<<-1<a<<0<flash<<6<Flash Video (Youtube, etc...)>0<<-1<a<<0<httpvideo<<6<HTTP Video (Youtube, etc...)>0<<6<a<<0<shoutcast<<6<Shoutcast>0<<-1<d<554,1755,5004,5005,6970:7170,8554<0<<<6<RTP,RTSP>0<<-1<d<1935,5060:5063,1719,1720,3478,3479,15000<0<<<6<RTMP,MMS,SIP,H323,STUN>0<<6<d<80,443<0<<0:256<0<WWW,SSL>0<<-1<d<25,465,563,587,110,119,143,220,993,995<0<<<3<Mail (SMTP,POP3,IMAP)>0<<-1<d<1220,1234,5100,6005,6970<0<<<6<QT,Camfrog,VLC>0<<-1<d<1502:1503,1863,3389,5061,5190:5193,7001<0<<<4<MSGR1 - Windows Live>0<<-1<d<194,1720,1730:1732,6660:6669,22555<0<<<4<MSGR2 - Chat Services>0<<-1<d<5000:5010,5050,5100,5222,5223,8000:8002<0<<<4<MSGR3 - Chat Services>0<<-1<x<20:23,6571,6891:6901<0<<256:<5<FTP,SFTP,WLM File Transfers>0<<6<d<80,443<0<<256:<5<HTTP, SSL File Transfers>0<<17<d<1:65535<0<<<9<P2P (uTP, UDP)"
  80. Lassik

    Lassik Networkin' Nut Member

    Scratch that previous post, I think this is a better way to set up Toastman's QoS settings without having to manually enter them in to the GUI which can take a very long time.

    In the GUI, navigate to (Tools > System) and paste the following lines in:
    nvram set "ct_tcp_timeout=0 1200 20 20 20 20 10 20 20 0"
    nvram set "ct_udp_timeout=10 10"
    nvram set "qos_enable=1"
    nvram set "qos_ack=0"
    nvram set "qos_default=8"
    nvram set "qos_fin=1"
    nvram set "qos_icmp=1"
    nvram set "qos_irates=10,10,60,90,0,70,70,70,80,1"
    nvram set "qos_orates=5-20,5-20,5-25,5-80,10-80,20-80,5-80,5-80,5-90,1-1"
    nvram set "qos_orules=0<<-1<d<53,37,123,3445<0<<0:10<0<DNS,Time,NTP,RSVP>0<<6<s<80<0<<<3<Remote Web Access>0<<-1<d<11999,2300:2400,6073,28800:29100,47624<0<<0:50<1<Some well known games>0<<-1<a<<0<flash<<2<Flash Video (Youtube, etc...)>0<<-1<a<<0<httpvideo<<2<HTTP Video (Youtube, etc...)>0<<6<a<<0<shoutcast<<2<Shoutcast>0<<-1<d<554,1755,5004,5005,6970:7170,8554<0<<<2<RTP,RTSP>0<<-1<d<1935,5060:5063,1719,1720,3478,3479,15000<0<<<2<RTMP,MMS,SIP,H323,STUN>0<<6<d<80,443<0<<0:256<4<WWW,SSL>0<<-1<d<25,465,563,587,110,119,143,220,993,995<0<<<5<Mail (SMTP,POP3,IMAP)>0<<-1<d<1220,1234,5100,6005,6970<0<<<2<QT,Camfrog,VLC>0<<-1<d<1502:1503,1863,3389,5061,5190:5193,7001<0<<<6<MSGR1 - Windows Live>0<<-1<d<194,1720,1730:1732,6660:6669,22555<0<<<6<MSGR2 - Chat Services>0<<-1<d<5000:5010,5050,5100,5222,5223,8000:8002<0<<<6<MSGR3 - Chat Services>0<<-1<x<20:23,6571,6891:6901<0<<256:<7<FTP,SFTP,WLM File Transfers>0<<6<d<80,443<0<<256:<7<HTTP, SSL File Transfers>0<<17<d<1:65535<0<<<9<P2P (uTP, UDP)"
    nvram set "qos_pfifo=0"
    nvram set "qos_reset=1"
    nvram set "qos_rst=1"
    nvram set "qos_syn=1"
    sleep 2
    nvram commit
    sleep 10
    Now click the "Execute" button and let the magic happen.

    The Sleep timeouts may or may not really be necessary but I put them in just to be safe.

    Sorry I forgot that the (Tools > System) section does not exist in the Standard build of Tomato. For Standard build users you can just SSH in to your router and paste those lines one by one. It's still easier than filling all of that out. Just skip the Sleep lines if you're doing it this way because there's no reason to enter those.
  81. Lassik

    Lassik Networkin' Nut Member

    If for any reason you want to go back to the default QoS settings without resetting everything to defaults, do the same process with the following lines:
    nvram set "ct_tcp_timeout=0 1200 120 60 120 120 10 60 30 0"
    nvram set "ct_udp_timeout=30 180"
    nvram set "qos_enable=0"
    nvram set "qos_ack=1"
    nvram set "qos_default=3"
    nvram set "qos_fin=0"
    nvram set "qos_icmp=0"
    nvram unset "qos_irates"
    nvram set "qos_orates=80-100,10-100,5-100,3-100,2-95,1-50,1-40,1-30,1-20,1-10"
    nvram set "qos_orules=0<<6<d<80,443<0<<0:512<1<WWW>0<<6<d<80,443<0<<512:<3<WWW (512K+)>0<<-1<d<53<0<<0:2<0<DNS>0<<-1<d<53<0<<2:<4<DNS (2K+)>0<<-1<d<1024:65535<0<<<4<Bulk Traffic"
    nvram unset "qos_pfifo"
    nvram set "qos_reset=0"
    nvram set "qos_rst=0"
    nvram set "qos_syn=0"
    nvram set "qos_obw=230"
    nvram set "qos_ibw=1000"
    sleep 2
    nvram commit
    sleep 10
  82. Toastman

    Toastman Super Moderator Staff Member Member

    For what it's worth, this is the output you requested ...

    nvram export --quote | grep qos_

    emote Access>0<<-1<d<11999,2300:2400,6073,28800:29100,47624<0<<0:50<3<Some
    known games>0<<-1<a<<0<flash<<2<Flash Video, (Youtube)>0<<-1<a<<0<httpvideo
    TTP Video, (Youtube)>0<<-1<a<<0<shoutcast<<2<Shoutcast>0<<-1<d<9,554,1755,5
    C>0<<6<d<80,443,8080<0<<0:512<4<WWW,SSL,HTTP Proxy>0<<-1<d<25,465,563,587,1
    3,3389,5061,5190:5193,7001<0<<<6<MSGR1 - Windows Live>0<<-1<d<194,1720,1730
    ,6660:6669,22555<0<<<6<MSGR2 - Chat Services>0<<-1<d<5000:5010,5050,5100,52
    23,8000:8002<0<<<6<MSGR3 - Chat Services>0<<-1<x<20:23,6571,6891:6901<0<<<7
    SFTP,WLM File/Webcam>0<<6<d<80,443,8080<0<<512:<7<HTTP,SSL File Transfers>0
    d<1:65535<0<<<-1<P2P (uTP, UDP)"

    Better to do " nvram export --set | grep qos_" though...
  83. Toastman

    Toastman Super Moderator Staff Member Member

    Bangkokiscool - Regarding the SMTP virus problem, this is one of the commonest virus problems. Did you try limiting the number of simultaneous SMTP sessions? I found this was the simplest solution. Of course, it might not keep your ISP happy. It does keep your system running though! Probably the best solution is to block port 25 altogether and use port 587 - the SMTP "submissions" port which exists for this very purpose. That's OK if you can inform your users, otherwise, difficult.

    #Limit outgoing SMTP simultaneous connections
    iptables -t nat -I PREROUTING -p tcp --dport 25 -m connlimit --connlimit-above 5 -j DROP
  84. ray123

    ray123 LI Guru Member

    Thanks, Toastman & Lassik, for the nvram commands. However, I see some discrepancies.

    The 5-20's are from Toastman.

    Unfortunately, Toastman's setting absolutely KILL my download speeds.
    Web surfing was pretty crappy, too. I'm guessing it's probably because his Inbound Max Bandwidth is 16000 kbit/s. I sure wish that I had a 16MBps internet connection!!
    20% of 16000kb is still pretty fast. 20% of 4000, not so much. :frown:

    So I guess that the Toastman settings may be good for a residential building with 400 computers, but not so much for a home network with 2 or 3 computers.
    Oh well, it's a starting point.

    Anyway, thanks, guys.
  85. Toastman

    Toastman Super Moderator Staff Member Member

    I've posted two compiled versions of 1.28 for the RT-N16 and similar MIPSR2 routers. The first is based on TomatoUSB RT code (beta 18?) from 8th August 2010 - and the second is based on the latest pre-Beta RT code from the git repository on 25th August 2010 - many thanks to Teddy Bear for his dedication.

    They were compiled for my own use, and should of course be regarded as very much BETA compiles.

    They are available in 2 flavors - with (a) "Toastman" class labels, now with Games/VOIP in the same class (b) the standard QOS class labels.

    All have the Victek RAF mods from RAF v8515.2 and may support up to 200 users (yet to be tested over 130 with ASUS). The amount of entries is limited by NVRAM space - so the more you have, the less of a problem.

    -IP/MAC Limiter
    -ARP Binding
    -CPU info and Clock Frequency Selection
    -Previous WAN IP Display
    -Access Concentrator ID Display
    -Additional Themes

    There is also a compile of Victek's 8515.2 with Toastman class labels and added wireless connection rates display backported from TomatoUSB.
  86. Toastman

    Toastman Super Moderator Staff Member Member

    To answer some of your PM's - the full ext +VPN build of 7426 seems to be "too large" to compile with RAF mods added, so it's not included.

    Ray - yes. It's a starting point, it's safe, it works most of the time, so I'm surprised it kills your connections (I have many sites with 500k/5Mbps ADSL). But then, I don't know what your useage is. But I'm sure you can see what the rules do and adjust them, the important thing is to give people an idea of what ports I've addressed for what applications, they can look up the ports and see why I did it. There may be things I've got wrong, or things that other people don't need. It's all about learning.
  87. bangkokiscool

    bangkokiscool Addicted to LI Member

    Toastman, your QOS settings continue to work wonderfully for my application (40 residents sharing one cable modem connection, managing roughly 12-13 GB of bandwidth (down) every day. I recently received a question from a user who says she can't use Apple iChat. I know Skype works well on these settings because I have users to use Skype every day, so I'm wondering if there's tweaks to allow iChat to work as well?
  88. Toastman

    Toastman Super Moderator Staff Member Member

    Eeek ... don't know anything about Apple iChat. I have very few Apple machines here, I think maybe one. I don't know if they use Apple iChat. Probably nobody does as almost everyone here uses MSN. I'll see what I can find out and you can try it. If you get it to work I will add it to my stuff also!

    I also see my users take 11-36GB every day - varies a lot. It's nice to hear that Skype is working well with quite high useage - do you know which of the two filters are most useful? I ask because the skypeout L7 filter does let a hell of a lot of P2P connections through, so for my purposes, is almost useless.

  89. Toastman

    Toastman Super Moderator Staff Member Member

    Classifying Apple iChat

    I'm assuming iChat clients will open needed ports on the router by UPnP or NAT-PMP (since it's Apple). If it doesn't open ports then maybe you're screwed.

    A lot of info here:

    Some ports are used for local traffic and don't need to be opened or given priority according to the blurb, so we are left with:

    U5060, T/U5190, T5220, T5222, T5223, U5678, U16384-16403 (T=TCP, U=UDP T/U=Both)

    I added a new MSGR4 rule for iChat - source and destination both TCP and UDP for these port numbers 5060,5190,5220-5223,5678,16384-16403

    Some ports are duplicated in other rules but this does not matter. This makes the rule self-contained.

    To maker the router more secure, the 4 messenger rules could be rearranged into a large TCP rule and another for UDP instead of often unnecessarily opening both. This would mean researching the whole subject again of course! I felt it would be simpler as it is, but it is up to each individual.

    Let us know how it works!
  90. bangkokiscool

    bangkokiscool Addicted to LI Member

    Thank you very much!! I added the MSGR4 line, and turned on NAT PMP. Will definitely report if the client has a better iChat experience now.
  91. Spyros

    Spyros LI Guru Member

    Can you help me with this, i don't know if it's a firmware or settings problem
  92. Toastman

    Toastman Super Moderator Staff Member Member

    Hi Spyros

    I tried installing TVU here - but will not work on any of my 32 or 64 bit machines with or without QOS. All other TV player and P2P TV Players work fine. This software, like most chinese-based software, seems to be full of bugs. so, I can't check it out. If it uses TCP port 80 then it should work just like any other HTTP connection. If outgoing data exceeds your upload "data transferred" limit of 512 k then it would drop into another rule below. The fact that it doesn't indicates that some other ports may be in use, and that may be the problem.

    Try disabling QOS rules one at a time to see if one of them is responsible. Maybe someone else who uses it can offers some assistance. Reading the many complaints about this software on the various forums, it seems that older versions were markedly more reliable than the current one.

    Take a look at JLC's Internet TV, and for online radio, Spider Player is a nice little gadget!
  93. Toastman

    Toastman Super Moderator Staff Member Member

    New versions 7616 (WRT K24)) 7816 (WRT K26) 7428 (RT/MIPS2) based on Fedor's latest code have faster page refresh times down to 0.25 seconds - useful for QOS graph, details, webmon - giving a more "real-time" feel to monitoring. The CPU utilization will rise, of course, so don't overdo it - just because the menu choice is there, you don't have to use it! However, an RT-N16 seems to have no trouble with faster refresh times. About 0.5 seconds is good for the graphs, the webmon list is nice when displaying 10/20 lookups (which fits a screen nicely) if it is refreshed a bit quicker. NB - These fast refresh times are for LAN use - over an internet connection they usually won't be able to update quickly enough. Have fun experimenting!

    NB - 7429 is development version of 7428 PLUS any mods or bugfixes from current git code.

    These last posted versions now have the complete set of latest QOS rules and short conntrack timeouts already entered as default config. Use this version if you don't want to spend hours keying the rules in manually. There is an extended + VPN build here too - [this one has the IPV6 support removed to make it smaller]. In future I will probably let the compiler run and make all variations available to share.

    Remember to save your old config and erase nvram after flashing, or you won't see the new default Toastman QOS configuration. And check back for later builds from time to time, be sure to check the compile dates. Rather than wait to issue a "release" if a bug is immediately found, I will occasionally just change the posted binary.

    The feedback so far - these builds seem to be very stable. A few errors have been corrected - check for a later release if you have problems saving CPU frequency and IP/MAC settings.

    For all you git users out there, I've totally rebuilt the git tree for these Toastman builds from the ground (RAF) up. You should find it tidier and much easier to use now. If you find any errors please let me know. "Save" bugs in CPU Frequency, Static DHSP, and IP/MAC limiter have been fixed (my error) and the CPU frequency box enlarged. Don't forget, when using the git, if you want RAF features merge RAF branch - if you want RAF features AND Toastman features, merge a Toastman branch. You may not want my labelled classes and QOS rules !

    For those who asked for it - I've added versions of 7428/9 with standard class labels
  94. Toastman

    Toastman Super Moderator Staff Member Member

    Just want to say something here. So many comments that say people tried my rules and it "killed" their downloads. Well, what do you expect? QOS is supposed to give everyone a fair stab at the pie. It will inevitably slow some things down and speed up others. What it does in your particular setup depends entirely on YOU. These rules are not cast in stone - they are there for you to learn - and change to suit your own useage. I'd recommend just disable rules you think you don't need and see what happens. When you change some parameter, wait to see what happens, don't change everything all at once.

    At the end of the day, the only way you will achieve what you want is by learning how things work, you must do this yourself. Nobody has the time to reply to hundreds of people all wanting someone else to do the job for them.

    Apologies for bitching - but some of the mail I get is unbelievable. Seems like some people still need diapers.
  95. peyton

    peyton Network Guru Member

    Will try it ! Thanks Toastman ! Will be easiest to erase nvram without re-adding every QoS rules.. :)
  96. Toastman

    Toastman Super Moderator Staff Member Member

    I used it for the first time a few days ago in a new installation. Wow, did it make life easier! Not only saved time but no mistakes in entry. Bliss.
  97. Toastman

    Toastman Super Moderator Staff Member Member

    Updated MIPSR2 firmware (RT-N16 etc) to v. 7430 using Teddy Bear's latest beta 22 release as a base. Judging by the way my server has been hammered all day, many people found it already :) Again, I posted compiles with standard QOS labels to keep some folks happy. All T/B updates plus IP/MAC now included in the list of expandable menus.
  98. peyton

    peyton Network Guru Member

    Could we ask for a victek mod one which include both VPN and Ext or it is too big ?
  99. Toastman

    Toastman Super Moderator Staff Member Member

    I thought the big VPN compile included the other stuff, but maybe I'm wrong. If not, it would be a matter of space, I guess.
  100. Toastman

    Toastman Super Moderator Staff Member Member

    Coffee shop and business with 2 vlans

    I helped set up a new installation on an RT-N16 using two extra vlans, vlan4 and 5. The main setup is a business office, the vlans exist to isolate other areas from the business network. Only one is in use at the moment, this is a data entry room on vlan4 and is on 10.0.5.xx subnet. I have two devices, a laser printer/copier, and a Belkin Skype phone on it along with 12 desktop machines and an AP for wireless access by laptops. The other subnet is to be used eventually for a coffee shop AP, with no access to anything except the internet. All assigned static IP's via DHCP. Firmware is my own build but based on Teddy Bear's latest ext. release.

    The normal LAN subnet on br0 (192.168.1.xx) functions as normal and UPnP ports open correctly, all lists and QOS functioning properly. The other vlans suffer only from a small cosmetic problem, the source and destination details in QOS/Details don't show correctly. I have QOS and UPnP working now.

    The setup here is taken from various articles on the forum - and I give it back the UPnP/NAT-PMP functionality which it seems is little known!


    sleep 10
    nvram set vlan1ports="2 1 8*"
    nvram set vlan3hwname=et0
    nvram set vlan3ports="4 8*"

    nvram set vlan4hwname=et0
    nvram set vlan4ports="3 8*"
    nvram set manual_boot_nv=1

    ifconfig vlan3 netmask up;
    ifconfig vlan4 netmask up;


    iptables -I INPUT -i vlan3 -j ACCEPT;
    iptables -I FORWARD -i vlan3 -o vlan2 -m state --state NEW -j ACCEPT;
    iptables -I FORWARD -i vlan3 -o ppp0 -m state --state NEW -j ACCEPT;
    iptables -I FORWARD -i br0 -o vlan3 -j DROP;

    iptables -I INPUT -i vlan4 -j ACCEPT;
    iptables -I FORWARD -i vlan4 -o vlan2 -m state --state NEW -j ACCEPT;
    iptables -I FORWARD -i vlan4 -o ppp0 -m state --state NEW -j ACCEPT;
    iptables -I FORWARD -i br0 -o vlan4 -j DROP;

    iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu;

    DHCP/DNS Custom




    New Toastman versions 7431 onwards is my solution to get UPnP and NAT-PMP working with vlans. It now appends the file "upnpconfig.custom" from JFFS if present. I set up JFFS on a USB stick and load the file from there. In this file you need to specify the listening IP of the vlans and the port ranges to allow forwarding.

    Future versions of Tomato will have a miniupnpd "Custom Configuration" box on the "Port Forwarding" page.

    You will see by default that Tomato allows only ports 1024-65535 to be opened by UPnP - this is common for security reasons. There is also an additional port 22 - often used by FTP - that is certainly a good idea to allow this too. You can add/delete/change these as you wish.

    Example upnpconfig.custom file:

    allow 1024-65535 1024-65535
    allow 1024-65535 22

    allow 1024-65535 1024-65535
    allow 1024-65535 22

    Thanks to the many people on the forum using vlans, whose work inspired me to get this working and post it.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice