Bufferbloat fix

Discussion in 'Tomato Firmware' started by geekjock, Jul 7, 2018.

  1. geekjock

    geekjock Network Guru Member

  2. cloneman

    cloneman Addicted to LI Member

    Enabling QoS helps tremendously with bufferbloat, using SFQ or fq_codel. There are a lot of settings to toy with, but the basic concept is turning it on and setting your upload/download to your line rate -15%.

    If you want to just have bufferbloat control without any of the Classes/Rules, in tomato, you have to delete all the rule that come preset.

    Unfortunately if you use fq_codel you cannot change any of the parameters in the Webui (such as target 5ms) but most people don't need to change this unless the connection is slow (less than ~2mbit). In that case you would have to use SFQ, which can also provide decent performance.
     
  3. geekjock

    geekjock Network Guru Member

    Thanks for the good info. I enabled QOS with fq_codel, set inbound and outbound max bandwidth limits -15%, and all classes to No Limit. Speedtest on DSLreports now has zero bufferbloat - yay!
     
  4. cloneman

    cloneman Addicted to LI Member

    Interesting, I don't know what No Limit does, nice to know that it has some positive effect for you.

    Typically what I'd have done is removed everything from the Classification page to have everything end up in "default", with something like Min 10% - Max 100% for that class.
     
  5. geekjock

    geekjock Network Guru Member

    Update. I guess I was too hasty. Further testing reveals that nothing is repeatable. Sometimes I get bufferbloat, with or without QOS, sometimes I don't. Can't find any correlation.
     
  6. txnative

    txnative Networkin' Nut Member

    Would you mind posting pics of what you have setup for both classification and basic settings? What is you connection type, fiber, cable, dsl?
     
  7. Onee-chan

    Onee-chan Network Newbie Member

    Try this QoS settings are very good:
    https://www.linksysinfo.org/index.p...-for-gaming-and-everything-else-tomato.73927/

    And test in DSLreports using SFQ and fq_codel to see which work better to you.

    For test in http://www.dslreports.com/ follow this steps:

    1. Create a Account http://www.dslreports.com/
    2. Go to http://www.dslreports.com/speedtest/preferences scroll down on the page and click on Reset
    3. Again go to http://www.dslreports.com/speedtest/preferences
    4. [OPTIONAL] Click in Auto Select "by Lactency"
    5. Add in "No. download streams" and "No. upload streams": 32 or 16 or 8
    6. Click in Advanse Setting and increase "Upload duration" and "Download duration" to 30 seconds.
    7. Click in Advanse Setting and active "dodge compression"
    8. Save and test. (always test using http)

    Nobody in your house has to use the internet when you are doing the test.
     
    Last edited: Jul 15, 2018 at 7:37 PM
    nodnarb91 likes this.
  8. txnative

    txnative Networkin' Nut Member

    You may ask, how much bandwidth is receiving from isp, mipsel or arm router he is using as those settings from the post my not work due to cpu if he has a mips router and also is it necessary to control the incoming since most device like voip and gaming would really benefit the outbound connection instead. For example my isp really has network instabilities on the inbound and sometimes outbound connection and while having bufferbloat test is fine but it's really only the outbound that would need to be properly corrected with qos, but the isp can sometimes screw with everyones bandwidth and for that reason I had to basically disable my inbound and only using the outbound portion to be optimized with qos, now this my example and current situation but just keep in mind that a network may not have to use qos unless you have crucial devices that need low latency, priority over other devices as i mentioned before voip and games.
     
  9. NotVeryClever

    NotVeryClever New Member Member

    For me, No Limit seems to bypass QOS (and prioritization) entirely, defeating the purpose.
     
  10. txnative

    txnative Networkin' Nut Member

    Using "No Limit" only on the outbound is done on my setup except for my voip/game class as I like to place 10240 kbps for outbound, the inbound classes are setup for lowest latency as possible on a five class setup mine is done similar to Testing demo as shown in that link, a user may always customize as needed to their own network needs and situations when using qos is needed, thank you tomato past, present and future teams.
     
  11. geekjock

    geekjock Network Guru Member

    I have Spectrum cable (legacy Charter), 70 Mb/s down, 6 Mb/s up. Router is an Asus RT-AC68U (reflashed TMobile).
    Capture.PNG Capture2.PNG
     
  12. geekjock

    geekjock Network Guru Member

    This is without QOS enabled. Test settings per Onee-chan. Tonight I am getting the same results with QOS enabled! But sometimes there is no bufferbloat at all, regardless. Capture3.PNG
     
  13. koitsu

    koitsu Network Guru Member

  14. txnative

    txnative Networkin' Nut Member

    Just using the no limit on all the classes is not how this is going to work. Note the link Onee-chann had posted if you follow those tips for setting up in that structure of classes, ports, names that should be enough to getting a better score from dslreports if that is what you want?. I see you are using Spectrum so hopefully you are not experiencing the same problems I am?(inbound fluctuations) If you are going to use "no limit" I suggest to "no limit" "100" on certain classes that way bandwidth will flow properly. and customize the classes also. The tip that was posted works with prioritized Services Ports, and so on meaning, you are balancing for network performance, for all devices. Mine is similar but Voip/Game is set to priority and high traffic is not set at all since Spectrum in my case has major problems with their inbound bandwidth that fluctuates like crazy and i don't have anything class setup for it. I'm not on my computer to show a screen shot as I'll post here in while. Good day
     
  15. txnative

    txnative Networkin' Nut Member

    My bandwidth is 100/10 meaning usually, 115/11 at first back in Aug 2017, now the inbound is sometimes over 150, 180 even 300Mbps while in outbound usually at the 11.5 Mbps once in awhile drops to 2 Mbps while the inbound will drop to 3, 20 90 but usually 3. I could go on about the detail but you get what I'm saying. My Qos is setup to deal with inbound and as the outbound stays steady I set it to 9000 kbits for gaming purposes and like many things you'll need to adjust to your own network needs and if you have rogue torrents programs someone maybe using you'll have to adjust to those as well. The dslreports bufferbloat test doesn't mean anything to me now, as mentioned before most to critical programs will use outbound bandwidth than inbound and a good Qos is a must to setup operate and optimize the network and give priority for those devices that will need it, but if getting a A or A+, you'll want to make sure you have an ARM router, pc router to handle in both directions, this why i use tomato qos, now if you have a lower bandwidth under 100Mbps like 75 and lower then you could use a mips router to handle both directions, my router is right now netgear r6300v2ch, I use the src mac for my PS3 a range of ports for the voip type ports as well as http, https that PSN uses and since setup is done this way priority is for my PS3. In order for me to check I would do a src mac for my pc that is connected to the lan network no ports class is voip/game give it short name and run some speedtest, run a ping test for fluctuations, adjust if necessary. Note that this is how I've done for my network for three people with no bittorrent crazy bandwidth hogging, four wireless devices using Spectrum's troubled network, yours will be quite different so adjust to your needs. Good day again editclasses_Screenshot.png
     

    Attached Files:

  16. geekjock

    geekjock Network Guru Member

    Update: I changed all classes from No Limit to 1% - 85% across the board (not prioritizing any particular traffic), and enabled TCP Vegas. Up and down max bandwidth limits set to values obtained without QOS. This seems to have greatly reduced bufferbloat, typically less than 20ms. A win, I think.
    Capture6.PNG
     
    nodnarb91 and cloneman like this.
  17. txnative

    txnative Networkin' Nut Member

    Keep in mind to use the country you are in, if you are in the US pick the US and so forth, to make sure you are receiving the proper results for whatever how many states you are testing from your home and to make sure you are the only user, disable wifi, unplug all other ethernet ports, just you. Are you using the default classes, basic settings, sometimes not doing anything to min-full bandwidth is fine as is, all a person is to do is place the proper inbound/outbound data in and that's is. The reason you are seeing bigh latency bloat is those locations are not anywhere near you if you are testing from the US.
     
  18. cloneman

    cloneman Addicted to LI Member

    A few points to help sort out some confusion:

    1. I consider The DSLreports bufferbloat test is fairly good, albeit not immune to problems, like any test or service. You can press the Cog/Gears button to change some of the settings of the test https://www.dslreports.com/speedtest/preferences , for example, choosing 1 server in particular.

    That said, best quick test for bufferbloat is observing continuous ping window while initiating some sort of competing traffic (e.g. file transfer or another speedtest)

    2. It appears I was correct initially in assuming "No Limit" is of questionable benefit. The proper approach in Tomato, at the time of writing, is to use "100%" when your goal is "use fq_codel/sfq , respect the global maximum"

    3. Don't use "Min 1%". This means that the classes above that one can use 99% of the bandwidth for themselves. If you don't want to deal with manual classification, a sensible solution would be to throw everything in default and use Min 25% , Max 100%.
     
    NotVeryClever and geekjock like this.
  19. Sean B.

    Sean B. LI Guru Member

    Out of curiosity, I tried these QoS rules, and they actually did yield a very noticeable reduction in intermittent stutter and lag for XboxOne game play. However, during said gaming my syslog filled with these:

    Code:
    Jul 18 02:19:19 Storage kern.warn kernel: net_ratelimit: 833 callbacks suppressed
    Jul 18 02:19:19 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:19 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:19 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:19 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:19 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:19 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:19 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:19 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:19 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:19 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:24 Storage kern.warn kernel: net_ratelimit: 1992 callbacks suppressed
    Jul 18 02:19:24 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:24 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:24 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:24 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:24 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:24 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:24 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:24 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:24 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:24 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:29 Storage kern.warn kernel: net_ratelimit: 3070 callbacks suppressed
    Jul 18 02:19:29 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:29 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:29 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:29 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:29 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:29 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:29 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:29 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:29 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:29 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:34 Storage kern.warn kernel: net_ratelimit: 3851 callbacks suppressed
    Jul 18 02:19:34 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:34 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:34 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:34 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:34 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:34 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:34 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:34 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:34 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:34 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:36 Storage daemon.warn dnsmasq[9090]: Maximum number of concurrent DNS queries reached (max: 150)
    Jul 18 02:19:39 Storage kern.warn kernel: net_ratelimit: 2319 callbacks suppressed
    Jul 18 02:19:39 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:39 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:39 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:39 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:39 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:39 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:39 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:39 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:39 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:39 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:44 Storage kern.warn kernel: net_ratelimit: 5806 callbacks suppressed
    Jul 18 02:19:44 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:44 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:44 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:44 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:44 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:44 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:44 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:44 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:44 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:44 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:48 Storage daemon.warn dnsmasq[9090]: Maximum number of concurrent DNS queries reached (max: 150)
    Jul 18 02:19:49 Storage kern.warn kernel: net_ratelimit: 5068 callbacks suppressed
    Jul 18 02:19:49 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:49 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:50 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:50 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:50 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:50 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:50 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:50 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:50 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    Jul 18 02:19:50 Storage kern.warn kernel: nf_conntrack: table full, dropping packet.
    I haven't messed with QoS much, but can only assume it's related rather than a coincidence.
     
  20. RMerlin

    RMerlin Network Guru Member

    Something seems to be generating an insane amount of DNS lookups on your network.
     
  21. Sean B.

    Sean B. LI Guru Member

    Yeah, noted. As that's the only time I've seen that happen, I'm wondering if the count is actually valid. Or if the QoS settings are somehow messing with conntrack, possibly filtering down to DNS queries multiplying from dropped/vanished responses. I'm thinkin' I should set a cronjob to monitor /proc/sys/net/netfilter/nf_conntrack_count every couple minutes that will dump /proc/net/nf_conntrack when some arbitrarily high value is breached. See what all IP's and their connections are in conntrack at the time.

    At the very least, it's a great justification to "have" to play some more Battlefield 4 tonight when the kids/wife hit the sack :D.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice