issue with QoS / L7 / FTP

Discussion in 'Tomato Firmware' started by bogderpirat, Jan 2, 2007.

  1. bogderpirat

    bogderpirat Network Guru Member

    hi guys,

    i'm using tomato Version 1.01.0918 on my linksys wrt54gl v1.1.

    i'm just now trying to configure QoS properly for the first time on the box and i've stumbled over this issue:

    on one of the pcs behind the router, there's an ftp server running for which i have forwarded ports 20 and 21. everything works fine - people can download from the machine. when i however set a priority for connections using the ftp via the l7-"ftp"-setting and try to set those connections to "low", they will still be classified as "lowest" - probably being caught by the "bulk traffic" filter.

    this is my classification:


    this is what the details page shows:


    my ftp server program is filezilla ftp server. i have also tried downloading from a public ftp instead of having people download from me (= uploading), and in that instance, the router classified the connection as "low". is it some issue with the server software not marking the ftp packets properly, or am i missing something easy? whats going wrong here?
  2. GeeTek

    GeeTek Guest

    Something to try.... Under QOS basic settings, check the "Strict Rule Order" box. Then create 2 custom rules, one each for port 20 and 21. For address, specify "Any Address" and for ports, specify "Src or Dst". Put these 2 rules at the top of the QOS rule stack and see if that fixes the classification problem. If so, you can back track towards where you are now to see what the snag was.
  3. bogderpirat

    bogderpirat Network Guru Member

    thanks for your reply.

    strict rule order is checked. these are the rules:

    as you can see, i've switched the class to "medium", since at the moment, i haven't classified any connections to take the medium class. this is for better spotting any changes.

    connections distribution says:

    clicking on lowest (wrong class for any ftp transfers...) shows me the two ftp- and ftp-data-connections (the person has two concurrent downloads going on at the moment):

    what the deuce?
  4. GeeTek

    GeeTek Guest

    You did not show your other rules in the screen shot. If you used the arrows to advance these 2 rules to the top of the list above all the other rules, and re-boot before re-connecting, then you would appear to have found a bug. Let me know if you do indeed have the rules listed first, and I'll set up a parallel test to see if my results are the same. FileZilla FTP server is excellent. I use it a lot !

    Edit - Never mind. I've had a helluva day. Your screen shot shows the page header which is at the top, with the 2 rules just below. I'll try an FTP to see what I can find out.
  5. GeeTek

    GeeTek Guest

    My default FTP class was low. I created a rule and assigned it to medium and subsequent connections were medium. I changed the rule to Highest, and it became highest. Maybe you did not re-establish the connection and Tomato did not re-classify because the connection was open. If you want to clear QOS chart entries of closed connections, there is an expire early button under "Advanced".

    Attached Files:

  6. bogderpirat

    bogderpirat Network Guru Member

    up front: yes, rules will make ftp-downloads from a server that is on the internet change their priority class. that works here as well. i also mentioned that in my first posting.

    what doesnt work is setting the priority for upstream to the internet that comes from the ftp server i am hosting on my machine. if people download from me via ftp, these connections will not get affected by any rule, so long as they are l7-rules.

    in the meantime, i have done a *LOT* of rebooting, and i have had the following scenarios:

    - i rebooted after making a rule that made connections with "src/dst"=="21" change priority to low. when someone started downloading from me, not only the ftp-protocol connection (=port 21, over which ftp downloads are only !initiated! and not carried out) went into the correct and designated priority, BUT ALSO the connections it opened on other ports (3000+ on my machine, 60000+ on the remote machine) which are used to send the actual data of the ftp download.

    - i rebooted after making a rule that only used the l7 ftp-filter. priorities of neither the ftp-, nor the ftp-data-connections were put into the designated class. i have also tried the early expire button - but since connections shouldn't be tracked after a reboot of the box, that was quite in vain.

    i'm lost. why doesn't the l7 filter work at all? why does the port-filter work for more than it should? could anyone confirm my ftp-server issue?

    to clarify: obviously, my needs are satisfied with a rule that has makes 21-connections go to the "low" class (and oddly enough also influences connections that aren't based on port 21), i have however not been able to use the L7 filter at all. there are only three conclusions that i can think of: a) user error, b) a bug in the firmware/web frontend c) L7(-filters?) doesn't work with (ftp-)server applications/filezilla.
    i'm secretly hoping for a), thats the easiest to correct :p

    edit: ugh, now all four ftp connections (one protocol- and one ftpdata-connection for each of the two downloads) are, as explained in the last paragraph, in their correct priority. they are now however not shown as giving off any bandwidth in the "Bandwidth Distribution (Outbound)" pie chart diagram. the realtime bandwidth meter shows the upstream correctly. there is clearly something not working correctly.
  7. GeeTek

    GeeTek Guest

    Now you have me confused too. Until now, I thought all FTP traffic stayed on port 21 (or your custom port). I initiated an in-coming connection and discovered the return traffic going out on port 2964, and it was in the "un-classified" category. My port # 2964 should at least have been caught by the Bulk rule and set for lowest. Your in-bound and subsequent return connections are however being classified properly now ? Very strange...... I think I had better quit before I get any further behind.

    Attached Files:

  8. GeeTek

    GeeTek Guest

    I upgraded to Tomato 1.02 and get the same results. As soon as I set my bulk filter on ports 1024 - 65535 to include out bound traffic and use class "A", my remote desktop connection on 3389 immediately became class A. Inbound and return FTP traffic still is unclassified. I went back to the FTP rule and set it for L7 FTP, like it is supposed to be, the way you had it (and class B). My outbund FTP connections are only going to the default class (low). My in bound connections are still un-classified. I'm sure we must be doing something wrong. If not, we should blame the error on Linux.........
  9. neutralman

    neutralman Network Guru Member

    hey guys, I don't want to bother, just to say this is great topic IMHO, since I am QOS freak, I am interested in this. WRT54 is great little box, and I find Tomato the best firmware available because of mighty QOS, but it seems QOS doesn't work as it supposed to?

    what kind of filters apply for QOS (for example: data packet goes trough filters from #1 to #100; if #1 is the highest priority, and #100 block that kind of packets, where would that packet end?
  10. bogderpirat

    bogderpirat Network Guru Member

    uhm, it *should* set the connection to highest priority, but after what i/we've accomplished thus far, i wouldn't be too baffled if the connection wouldn't be classified at all. :)
  11. bogderpirat

    bogderpirat Network Guru Member

    alright, here's the solution.

    the phenomenon of the port21-rule also affecting other ports that are ftp-data was in fact a feature. it's called "ftp connection tracking module" and apparently marks ftp-data connections that go through random ports as regular ftp-connections.

    the issue with the l7 filter was - depending on the way you want to look at it - either a design error of the ftp pattern file included in the tomato firmware, or an issue that lies with filezilla ftp server.
    i ssh'ed to my box and took a look at the l7-ftp-pattern file:
    # cat ftp.pat
    ^220[\x09-\x0d -~]*ftp
    apparently, the firmware will look through the packets and prioritize any connections that start with "220" and have the string "ftp" in it according to the rule you use. in the ftp RFC, the 220-command (=welcoming message) however isn't required to be followed by the string "ftp" at all. there are a lot of servers that have "ftp" in there standard greeting line, however neither did the default welcome message of filezilla ftp server, nor did mine. just changing the entire welcome message under edit/settings/general settings/welcome message of the filezilla server interface to "ftp", or adding the word somewhere resolved the issue. ftp-connections, as well as ftp-data-connections are now being correctly prioritized.

    i have found an l7 pattern file here that contains a few more v8 expressions that will catch ftp connections on a more basic level, the l7-protocols-directory of the firmware however seems to be read-only, so there's no way of replacing them.
  12. bogderpirat

    bogderpirat Network Guru Member

    gah, double posting.
  13. GeeTek

    GeeTek Guest

    Dude, that is some pretty serious stuff you got into there. I'm very impressed. Thanks for posting back with what you found out.
  14. neutralman

    neutralman Network Guru Member

    @bogderpirat - great job!

    @GeeTek, others

    can we inform Tomato(Tofu?) to update L7 patterns in Tomato firmware?
    that would be great, so we don't have to add welcome message(es) and such things :)
  15. johnny2002

    johnny2002 LI Guru Member

    Please see my set for instance messenger QQ:
    After setting, I found QQ connection is still in lowest class. What's the problem?

    Attached Files:

    • qos.jpg
      File size:
      9.1 KB
  16. jeradc

    jeradc LI Guru Member

    That's very nice find. Definitely something Tomato needs to address, looking at the pattern file, it appears that there are 5 options to choose from, with only one at a time working.

    So, maybe Tofu can add a drop down, letting the end-user choose what level of ftp packet sniffing they want, as the more complex regex's will slow things down a bit.

    Edit: After digging into the Tomato Source, I see that that .pat file you linked to, is the exact file in the source ( \tomato\release\src\router\layer7\protocols\protocols ). So, giving the option of using one of the other 4 regex patterns in the file would solve your issue.
  17. bogderpirat

    bogderpirat Network Guru Member

    jeradc: the ftp-issue has already been resolved. the 1.04 changelog shows that jon has incorporated an extra space for custom L7 pattern files. i've even tried it - it works nicely.

    johnny2002: as i can see from the l7 pattern file commentary for QQ, only client-to-server-traffic can(?) be categorized properly. are you observing client-to-client connections? please check by resolving the login servers mentioned here and looking whether theyir ips are the same as shown in your QoS details.
    if they are, then something's going wrong.
    if they aren't the same, you're probably connected to a peer directly instead of over a server - and as of now, that kind of connection cannot be identified. at least if i'm not mistaken. also, i don't know QQ at all, but perhaps there is an option to let chat-traffic go via the server, in which case your already prioritized connection to the server can be used to prioritize your chats.

    but i'm just poking holes here.
  18. jeradc

    jeradc LI Guru Member

    well, that's what I get for not looking at the current interface! /doh :rolleyes:
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice