user defeats upload bandwidth limit

Discussion in 'Tomato Firmware' started by SunnyWest, Jul 25, 2014.

  1. SunnyWest

    SunnyWest Network Newbie Member

    Said user was first asked politely then warned more sternly about too much torrent uploading seeding (other users complaining about poor access to the shared internet connection, causing manager of the shared housing situation to attempt to remedy the issue).

    We wonder if the defiant user, who is somehow overriding an upload limit of 50kbits/sec (again we tried to avoid having to do that by gentle requests then stern warnings, to no avail)

    The user is getting 700Kbsec upload speeds. And still interfering with other users.

    We wonder if this defiant and somewhat antisocial user is engaging a vpn to defeat the 50Kbit/sec upload limit we set for him on our ASUS router (a new Asus RT-AC66U router).

    How is this user able to defeat the 50Kb/sec maximum upload bandwidth we have set in the "Bandwidth Limiter" of Tomato (version 1.28 by Shibby) ??
  2. rs232

    rs232 Network Guru Member

    Don't use bandwidth limiter, I have never managed to make it working properly.
    However do post your current setting and somebody else might be able to help you.

    My solution is:

    I suggest you assign him/her a static IP (via DHCP static allocation) and in QoS you match the SRC IP in the classification.

    Then limit his class to 50KB outbound.


    even if it doesn't get lot of bandwidth he/she can still slow down Internet as torrent clients open a consistent number of connections. Try to limit the connections on a user basis adding these lines under Admin/script/firewall:

    #Restrict number of TCP connections per user
    iptables -t nat -I PREROUTING -p tcp --syn -m iprange --src-range -m connlimit --connlimit-above 100 -j DROP
    #Restrict number of non-TCP connections per user
    iptables -t nat -I PREROUTING -p ! tcp -m iprange --src-range -m connlimit --connlimit-above 50 -j DROP 
    Obviously make sure you use the correct IPs according to your settings

    Marcel Tunks likes this.
  3. EOC_Jason

    EOC_Jason Networkin' Nut Member

    Unplug them from your network and make that person buy their own internet connection...
    mpegmaster and s44 like this.
  4. SunnyWest

    SunnyWest Network Newbie Member

    Thanks to both of you, the 'unplug from the network' is a possibility and we've let him know that the others on the network will only take so much.

    rs232, I'm going to next use your suggestion to use the "QoS->Classification" section of Tomato Shibby 1.28 and assign him to a low class. I have already limited TCP connections by using the "Bandwidth Limiter" section of the router. I first set them to 150 max. No effect on his behavior. Then dropped it to 100 tcp connections. Some moderate affect then defiance again. He has now been shunted to a max of 50 TCP connections. As I monitor the "IP Traffic->View Graphs->Connections Distribution" section at the top of the pie charts, he's clipped to that number of connections, so at least that part of the "Bandwidth Limiter" section works.

    Here's a question -- this guy is now using the following to broadcast messages to all the other clients on our Lan network -- maybe some kind of ping attack or something else to take revenge on them for complaining to management about his misdeeds?

    Why/what is he doing possibly by broadcasting to and to
    (the details below are from the "QoS->View Details" section in Tomato 1.28):

    ProtoSourceS PortDestinationD PortClassRuleBytes OutBytes In

    UDP192.168.1.3 [resolve] [hide]53838192.168.1.255 [resolve] [hide]1947Unclassified680
    UDP192.168.1.3 [resolve] [hide]53838255.255.255.255 [resolve] [hide]1947Unclassified680
  5. koitsu

    koitsu Network Guru Member

    That just looks like standard broadcast traffic, using UDP protocol. The ports involved -- source port 53838 and destination port 1947 -- don't really narrow anything down. Broadcast traffic is normal.

    The question then quickly becomes how much of this he's sending. I cannot tell because of the formatting of the data you copy-pasted. Based on my own guesswork, I would say the lines you pasted break down like this:

    Proto = UDP
    Source =
    S_Port = 53838
    Destination = {or for the 2nd line}
    D_Port = 1947
    ClassRule = Unclassified
    Bytes_Out = either 6, 68, or 680
    Bytes_In = depends on what Bytes_Out was, else possibly empty

    If all of this is decoded by me correctly, then 680 bytes is no where near 700kBytes/second (nor 700kBits/second; you said "kbsec" which is too vague).

    P.S. -- Limiting the number of concurrent TCP sessions has absolutely nothing to do with limiting speed or bandwidth (those two are also separate concepts, just for the record).

    What is often difficult for folks to understand is that QoS is a "prioritisation" mechanism in most cases, where you can say packets of type X/Y/Z are processed/handled at different priorities from packets of type A/B/C, and those with higher priorities are processed/handled first. This does not directly correlate with speed (at least not 1:1). Some QoS implementations offer the ability to limit actual bandwidth (total capacity limit) or apply rate-limiting (speed limit). The Linux utility tc can control these, as well as general QoS concepts; I forget if TomatoUSB uses tc or not (I do not use QoS of any kind, nor the exclusive bandwidth limiter). You can see an example of actual rate-limiting when matched against an IP address here. Don't forget that traffic is bidirectional, i.e. input and output are separate things. If he's chewing up bandwidth uploading (that means sending), then that means you probably want to rate-limit his outbound traffic, but you may also consider limiting his inbound (download). Your call. There are some other examples here (know that the interface/device names discussed will be very different on TomatoUSB).

    I cannot help verify or test or give example tc rules, nor do I know if they conflict with the TomatoUSB QoS model -- I have no familiarity with all that. I just know tc is what you use to achieve rate-limiting.
    Last edited: Jul 27, 2014
  6. i1135t

    i1135t Network Guru Member

    Create a guest or second network and only grant him access to that network. Then throttle that range down to the speed you want to set through bandwidth limiter. I won't matter if he tries to set a static IP, it will still be enforced.
  7. Marcel Tunks

    Marcel Tunks Networkin' Nut Member

    @koitsu You've hit the nail on the head by pointing out that limiting the number of connections will not effectively limit bandwidth.
    By the way, I believe Tomato QoS uses tc. It includes both prioritization and traffic shaping.

    My approach would be to:
    1. Talk to the user about your concerns.

    2. Try QoS because it's fun to learn about it and it may preserve low latency and higher throughput of desirable traffic types. Initially try the default classes and default percentages with your own throughput data. Then check graphs and other data from the QoS pages to see what types of traffic are flowing when problems arise so you can tweak the settings.

    3. If the user keeps changing ports for the torrent (or whatever traffic type) that's causing the problem, then create a rule and limit bandwidth for the user's ip addresses using the QoS or bandwidth limiter.

    4. Don't use QoS and bandwidth limiter at the same time.

    5. If all of the above somehow fails, or if your router's CPU can't handle your overall bandwidth with QoS or BW limiter enabled, then remove the user from your network.
  8. Porter

    Porter LI Guru Member

    @Marcel Tunks
    Yes, all version of Tomato use tc. The GUI is just a front-end to interact with tc and iptables (Tomato only uses iptables to mark traffic). You can always write your own script, but disable QoS for that or make sure you know what you are doing.

    I'm with Marcel Tunks on this. Try the QoS feature. Remember to measure your bandwidth (and then deduce 15-30% as a safety margin) and keep in mind that your bandwidth might fluctuate thourghout the day, so better play it safe because otherwise you will have times of the day when QoS doesn't work.

    If you can't identifiy the traffic he is sending, this is no problem. There is a default class for this where all the unidentified traffic goes and this class doesn't have a high priority.

    After enabling QoS and letting it run for some time, please check whether your CPU is powerful enough by looking at the status page under CPU Load (values should be below 1.0) and CPU Usage.

    If your user is torrenting a lot, he might open a lot of concurrent connections and thereby filling up the NAT table which leads to other users not being able to connect to the outside world. This is another problem and with such a new router this wouldn't be my first concern. It's just something to look out for. You can check the amount of concurrent connections under Advanced/Conntrack/Netfilter. It should only be a problem if the count reaches the limit.
  9. Mercjoe

    Mercjoe Network Guru Member

    I had that problem here.

    My solution:

    1) Use the crawl class; set it to something like 5% up and down

    2) Set the default class to class D (P2P/bulk)

    3) in classification, set the FIRST rule to be one that assigns EVERYTHING from his source MAC address to the class E. It must be the FIRST rule.

    Problem solved. No matter WHAT he does, no matter what he changes, it is limited to the QOS class. The only thing is if he changes his MAC address. There are other ways to handle that issue at the router level as well.

    That was what ultimately fixed all the shenanigans here.
  10. SunnyWest

    SunnyWest Network Newbie Member

    Thanks to everyone -- I've checked the cpu load, on the 1/5/15 minute periods that the Status page checks, it's at 0.00/0.00/0.01 so I'm fine there. The other advice above, there are are things we have not tried yet, there are several good suggestions there, I have noted them and those are now in our toolbox, thank you highly.

    We changed router access policy to require all users submit MAC addresses. Although we knew by device name at lease one of devices the person was using, the anonymity of nameless/faceless MAC/IP addresses meant that the potential remained for mischief.

    I read somewhere of another user in a similar situation with defiant torrenting users -- they made a new policy to require anyone who wanted to use the router to submit their MAC address.

    The goal was to eliminate the mischief by removing anonymity; when anyone now connects to the router they know their misuse of torrents is no longer hidden by anonymous nameless/faceless MAC and IP addresses.
    We figured the inability to hide behind anonymity would stop the torrenting user.

    That worked. *Almost.*

    We used the 'Access Restriction' section of Shibby 1.28.
    - we checked the 'Enabled' box
    - put the 'Schedule' to "All Day" "Everyday"
    - set the 'Type' to "Normal Access Restriction"
    - set the 'Applies To' the value "All Except..." then entered management's MAC addresses.
    - we clicked 'Save' on the Access Restriction page and rebooted.

    We did a remote reboot of the router (we're logged in remotely as admin) and using the 'Reboot...' on the main menu bar, and after a reboot of the router, we verified the new MAC restriction rule is present in the Access Restriction section in Tomato.

    At this point, shouldn't all MAC addresses be prevented from using the router except Management's machines, which are off-site, out of the router's range?

    Someone is still accessing the router.
    The 'IP Traffic->Real-Time' shows someone uploading at around 700kbits/s

    How is that possible?
  11. Mercjoe

    Mercjoe Network Guru Member

    You need them to submit MAC address's??? You get them when they are connected to the router.

    I think we need more info on your network infrastructure. Something is not adding up here.

    Just how many users are you supporting? What is your available bandwidth.

    Are devices IP assigned by DCHP, defined static IP, or mixed assignments? Can you just plug in (or connect wirelessly) and you are ready to go?

    Are you using VLANS?

    Is this a business, a college dorm, a household?

    In order to have a workable solution we need more info. Otherwise we are just making blind guesses
  12. SunnyWest

    SunnyWest Network Newbie Member

    The MAC address appears when anyone connects to the router, indeed. But there is no way to know which of the 19 people in our live/work loft 'belongs' to that MAC address.

    The entire point of blocking all router access was this:
    1) any user who wants to access the router must submit their MAC address in person
    2) we then add their MAC address in a new rule we created called 'allowables' in the Access Restrictions
    that (is supposed to) only allow MAC addresses on that list to access the router.

    This does 2 things:
    1) the user knows that we know their MAC and IP address (we have added each to the 'Static' list to lock each MAC address to a single local IP address)

    2) thus the user won't interfere with the other users because they know that we know who they are, and that we are monitoring the upload and download amount of each user

    Previously, we didn't require the users to identify their machine by providing a MAC address. We just gave them the network key. The overusing user just happen to use his real name on (one of?) his machine so we were able to identify that one person.

    Reality is there are other torrent uploaders here, but this new 'Only these MACs allowed' list should solve the problem for all of them, since they can no longer hide behind our inability to associate their MAC address with them personally.

    Because *very few* users out there these days put any kind of identifying name at all on their devices. So while we do get MAC addresses in Tomato, unless the user was foolish enough to name his device in a clear manner, we cannot know which of the 19 people own a given MAC address.

    By 'VLANS' is that virtual Lan? Wouldn't know how to set one up/use one. So the answer there is 'No.'

    All users, the first time we notice a new device in the 'Device List' section, get their MAC statically bound to the IP address they have at that time by way of the 'Static DHCP/ARP/IPT' section of Tomato.

    And yes, we thought 'everyone's courteous, and recognizes and respects the connection is shared with others' and when they moved in they were given the access key. And we did not restrict their number of devices.

    I don't know about you but we personally do not like micromanaging/babysitting the shared internet connection. But some 'adults' are not cool, sorry to say, and don't seem to give a rip about others.
  13. Mercjoe

    Mercjoe Network Guru Member

    Well babysitting & micromanaging advice is what you have been asking for here.

    There is no magic button in network management. If someone is bound and determined to abuse the system then you have to be bound and determined to catch them and stop them.

    In order to FIX this you are going to HAVE to micromanage it for a little bit. You are not going to fix this without nailing down who is doing this.

    Suggestion for a temporary measure to nail this down..

    Use DHCP but set the IP range to a small allowable range. Something like 100-110.

    In Basic --> static DHCP: Assign a IP to a specific MAC address. Give each hostname (you define this, not the user) a easily identifiable name. Bind each IP and assign to IP traffic. Make sure that the assigned IP is outside the DHCP address range you set above.

    In Basic--> static DHCP check the 'ignore DHCP requests from unknown devices'

    This is going to the following things for you.

    1) only the people who should have access to it WILL have access to the network. If they have not been pre-authorized they can not get a network access connection. Now you know EXACTLY who is using your network and what IP address they have. Just having the network key is not enough now.

    2) you will be able to look at IP traffic for each connection in IP Traffic. This will tell you who is using a lot of bandwidth.

    3) with a specific IP, you can go into QOS connection details and look at what the connections are. Is it a torrent or is it YouTube?

    4) now that you know EXACTLY who is doing this you can have an informed adult discussion with them about what they are doing and what the consequences of non-compliance will entail.

    If they ignore you, then cut them off.

    Either way, everyone ELSE who is on the network will thank you for making their internet experience more enjoyable.

    Is it enjoyable doing this? No. It is not. It can bee seen as intrusive as hell. Sure. If you want to be in charge of the network then BE IN CHARGE.
  14. abubin

    abubin Addicted to LI Member

    are you using qos and bandwidth limit together? There is a known issue that both of them does not work together. Try disabling qos and only use bandwidth limiter. See if that helps.

    Another way to solve this issue is to create a separate wifi ssid and network like someone suggested. Limit this whole different network to the speed you want. Give this key to the users you want to limit speed. I have tested this and it works. Something similar like this link
  15. SunnyWest

    SunnyWest Network Newbie Member

    I'm not using the 'QoS->Classification' except with the many 'Match Rules' that came by default on Shibby 1.28.
    I only turned QoS on so that I can see the pie graphs, I do not want to turn it off and lose those.

    Right now I have deleted all the Bandwidth Limiter entries I was using before we switched to the "you must provide your MAC address to get on the router" protocol.

    I think this is working, because only one user 'A' has provided their MAC address and I know they're out of town for the weekend.

    But I see some interesting things.

    1) when user 'A' was in town, checking the 'IP Traffic->Real Time' showed only their machine getting downloaded bandwidth.

    2) but on the 'Status->Device List', I see 5 (five) machines in the list with TX/RX Rate values that are changing!

    Can a VPN or Proxy get around Tomato's 'Access Restrictions' rules -- does my access restriction rule 'Only these MACs allowed' still work if the user is using a VPN (maybe with encryption too) or a Proxy?
  16. Porter

    Porter LI Guru Member

    Toastman is one of the people around here who really utilizes the QoS system to manage those types of users. If I recall correctly, he is a network admin for several hotels. What I'm trying to say is that the QoS system really might be what you are looking for. Just give it a try. But please restore your config so that there are no errors in the QoS config. This should be much easier than constantly checking for new users, registering them and so on...
    mvsgeek likes this.
  17. koitsu

    koitsu Network Guru Member

    Narrowing down who's doing it should be easy enough -- using tcpdump against the br0 interface when the issue is happening (look at bandwidth usage graphs -- make sure your WAN interface shows outgoing traffic as well as the LAN or Wireless interfaces! If you can't correlate this in advance, i.e. WAN shows high upload but nothing shows up on LAN or Wireless, then your router is what's spitting out the traffic, in other words someone may have compromised your router and is using it illegitimately) should be sufficient; tcpdump -v -p -i br0 -l -n -s 400 -w /tmp/capture.pcap would work just fine, using Ctrl-C to terminate the capture when done. But to do this you'll need to get tcpdump by installing Entware (usually the easiest method, as long as you have a router + firmware that supports USB flash drives) or by finding a tcpdump static binary (like from here) (and put it on the router using opposite syntax as below, ex. scp tcpdump root@ followed by getting on the router via CLI and doing chmod 755 /tmp/tcpdump then using /tmp/tcpdump to use the utility -- it will disappear, by the way, if you reboot the router (/tmp is in RAM)).

    Copy /tmp/capture.pcap off of the router (depends on what services your router is running, but the easiest it to enable the SSH server and then use scp root@ . to copy the file to your system, or if you're on a Windows system get PuTTY and use pscp.exe instead of scp (same syntax though)).

    Load that capture file into Wireshark, choose Statistics / Conversations, choose the IPv4 tab (I'm assuming the traffic is IP traffic and not 700kbit/sec of ARP -- that would surprise the hell out of me!), and you'll get an idea of what IP address is doing what and how much traffic (in bytes, or bits per second (last column)) is flowing per IP address. The IP addresses should be in the 192.168.x.x range (assuming that's what you've configured for DHCP delegation / what subnet the router is part of).

    If this is a MDU (multi-dwelling-unit) then expect there to be a lot of traffic, but you should be able to narrow down who's doing what. The analysis must be done by a human being and you must know what it is you're looking at. If you don't know how to do this, you can upload the capture here and one of us can review it. Be aware that up to 400 bytes of traffic per packet will be stored in this capture, which may include HTTP headers, websites people are visiting, etc.. There is no way around this. Just something to keep in mind before uploading something like this publicly.

    Once you know what IP address on your LAN is doing it, you should be able to map that to a MAC address and/or DHCP address, which should hopefully narrow it down for you (this is shown in the GUI under Device List in TomatoUSB). If you suspect someone using an illegitimate MAC or something along those lines, then you need to start physically unplugging cables (unless they're on wireless, in which case you need to switch to a deny-all-permit-only-specific-MAC model). You can also use arp -a from the CLI, but that information is identical to what the GUI shows.
  18. SunnyWest

    SunnyWest Network Newbie Member

    Koitsu good calls there, I like the idea of tcdump. Right now, everyone is cooperating and giving their MAC address. So as of now we know who owns which MAC address. And we're only allowing those to access the router by way of the Access Restrictions in Tomato. And since each MAC address is getting statically bound to just one local IP, we know who owns which MAC/IP address combo.

    So if there's any mischief, we'll know.

    One person who has not given a MAC address is the antisocial ahole who has turned me into a sysadmin against my will. Although I have to say I'm enjoying learning this stuff.

    Right now everything is fine -- nothing malicious happening.

    To my question -- anyone know if a user on a Tomato router will not appear in the 'IP Traffic' charts and pie chart and bandwidth counts (found under 'IP Traffic->Daily') ??

    If the guy uses a VPN, or a proxy, does the Tomato router still count his outgoing bandwidth, and incoming bandwidth?
  19. koitsu

    koitsu Network Guru Member

    With regards to the bandwidth graphs: the router counts any traffic going through it. It doesn't matter if it's encrypted with a VPN or classic plain text. Network traffic is network traffic. The data comes from interface counters, which the kernel (for things like vlan2/WAN) or network drivers (though it's kind of a combination of the driver and the kernel togehter) (for things like eth0) track. These are counters that increment from the last time the router rebooted (i.e. they are not saved across reboots); the bandwidth usage stuff in Tomato is stored elsewhere and saved across reboots (assuming you have those checkboxes enabled in the GUI).

    If you want to see the raw interface counters, from the CLI do cat /proc/net/dev. Packets (per direction) and bytes (per direction) are there. Example:

    root@gw:/tmp/home/root# cat /proc/net/dev
    Inter-|   Receive                                                |  Transmit
    face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop fifo colls carrier compressed
        lo:  270720    1494    0    0    0     0          0         0   270720    1494    0    0    0     0       0          0
      eth0:2177730299 461648311    0    0    0     0          0         0 3176612001 276215195  186    0    0     0       0          0
      eth1:1638875796 8875366   41    0    0 95693536          0         0 588100089 16702942 6570    0    0     0       0          0
      eth2:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
      imq0:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
      imq1:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
      imq2:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
    vlan1:3317422692 123929206    0    0    0     0          0     56705 855893170 144006605    0    0    0     0       0          0
    vlan2:3295436088 337573642    0    0    0     0          0   9846352 2268913170 132124155    0    0    0     0       0          0
       br0:4269517005 132615318    0    0    0     0          0    402675 191710155 159446206    0    0    0     0       0          0
    Do not worry about a semi-high error count or frame count for the physical wireless interface -- errors of this type are generally normal for wireless, given the nature of the medium. (And I see I have had 186 transmit errors on my LAN interface -- oh well, whatever :)).

    ifconfig -a will also show most (but not all) of this information.

    But none of this is useful when granularity is needed, e.g. correlating network utilisation and an individual. I personally don't use the IP Traffic feature because there are cases documented on the forum about it causing problems (router reboots I think? I forget), in addition to me dealing with network protocol analysis and related things as part of my job, so I'm used to "lower-level" tools and know I can trust those.
    Last edited: Jul 28, 2014
  20. Grimson

    Grimson Networkin' Nut Member

    Sorry, but this does not prevent anyone from gaining access to the network. They can't get an IP via DHCP but they can still set their IP locally on their machines.
  21. EOC_Jason

    EOC_Jason Networkin' Nut Member

    MACs are not routable, they are only visible / used on a local LAN. If you are connecting from a machine across the Internet, it has no idea what your MAC address is.
    koitsu likes this.
  22. SunnyWest

    SunnyWest Network Newbie Member

    Koitsu that helped. Bigtime. Knowing that if one of our torr(m)entors tries to evade detection by way of an encrypted VPN will not help them is reassuring.

    After a couple days of locking down the router by MAC address, the normal users are thanking us because the connection is reliable now. Funny thing, a few of the users refuse to provide a MAC address and thus are not using the shared connection. Dang it, dang it. The connection is there for everyone, yet there are some who cop an attitude when we try to ensure everyone can use the connection easily without being shunted by torrenting.

    Freaking sad, but regardless, we've solved the problem, and hey, this forum is a fantastic resource, I really want to thank everyone for helping us learn about this stuff. Really really appreciate it!
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice