Tomato 1.19 SpeedMod

Discussion in 'Tomato Firmware' started by hardc0re, May 9, 2008.

  1. hardc0re

    hardc0re Network Guru Member

    Hi guys,

    I've released a modified version of Tomato 1.19 which aims to improve the router's performance under high load (many connections, more than 2000 or so).

    All the modifications are made to the Linux kernel. To summarize:

    1. Tuned route cache and conntrack hash sizes.
    2. Tuned route cache parameters.
    3. Updated conntrack and netfilter components to later versions.
    4. Modified hash function to a newer one.

    Get it here:
  2. nvtweak

    nvtweak LI Guru Member

    Hey, that's the one in my sig! :tongue:

    # uptime
    14:59:19 up 12 days, 19:49, load average: 0.00, 0.00, 0.00

    12 days uptime is pretty good considering SpeedMod has only been out that long :wink: No reboots since I flashed it!

    It's a great mod, thanks! :thumbups:
  3. DeCex

    DeCex LI Guru Member

  4. i1135t

    i1135t Network Guru Member

    Can someone take you speed-mod version and append the OpenVPN function to it also? I dunno if that's possible....?
  5. FRiC

    FRiC LI Guru Member

    Mmm, speedmod and openvpn, that's what I'd like to have too...
  6. regular

    regular LI Guru Member

    thanks for the mod! Been working great for the past few weeks on your modded fw.
  7. Mercjoe

    Mercjoe Network Guru Member

    I have a question for the modder,

    How does this mod affect memory usage? I run the hostfile mod on my WRT54g ver 2.2 and running your speedmod AND the hostfile, I found the router to be sluggish and rather unstable.

    In fact, soon after flashing the mod and letting it settle out, I had almost NO available memory. With the regular Tomato, I still have quite a bit left after even a week of usage. Yes, I know the kernel cleans up memory as needed, but it seems that the speedmod took it all right away and never lets it go. The whole network grinds to a slow drizzle after about 2 days. I give you this, it never crashed, but got to the point that is takes 5 minutes to load any web page. A reboot and everything works fine again for a while.

    Is there a way to configure the mod 'halfway'? So that I get SOME of the speed improvements and still retain the hostfile functionality? I have 5 computers being fed off the router and anything I can do to increase speed AND stability is greatly appreciated. I have found that the hostfile function is a sorely needed item that I cannot do without.
  8. hardc0re

    hardc0re Network Guru Member

    Hi Mercjoe,

    When you say hostfile, do you mean /etc/hosts and are you using JFFS2?

    SpeedMod does use up more memory, depending on how many max connections you set. About 1 Megabyte more on average. This is because of the larger hash table sizes.

    On a 16MB RAM router, if you don't go above 6000 connections you should still have about 2 to 3Mb free RAM left (free ram includes cache memory).

    If you haven't tried build 8500 yet, try it and see. It should have slightly better memory usage and reduced cpu load. Its not possible to go 'half-way' without changing some things in the source code and recompiling.

    It will help diagnose the problem if you can send the output of:
    # uptime
    # ps
    # cat /proc/meminfo
    # cat /proc/slabinfo | grep ip
    # cat /proc/net/rt_cache_stat

    It will give an indication of what is using up the memory.
  9. Mercjoe

    Mercjoe Network Guru Member

    Loaded into RAM at startup and updated on a weekly basis. Basic script has been posted on this forum before.

    Yeah, I have only 16meg of RAM. I have a limit of 2048 max connections in the config.

    I was using ver 8423 before. I will try 8500 and get back to you on it.
  10. hardc0re

    hardc0re Network Guru Member

    Great. There have been quite a few significant changes from 8423 to 8500. Hope it'll be okay for you.
  11. der_Kief

    der_Kief Super Moderator Staff Member Member


    i think i make this sticky :biggrin:

  12. Meffy

    Meffy LI Guru Member

    I realized an increase in CPU load by using this mod
    CPU Load (1 / 5 / 15 mins) 0.12 / 0.04 / 0.01
    I used to get 0.00/0.00/0.00 using the old firmware
    Theres no difference in memory for me since i'm using roughly 70% of the total memory only.Router seems fine thou:D
  13. GhaladReam

    GhaladReam Network Guru Member

    It's the opposite for me...

    Used to get 0.70-0.85, now I'm constantly at 0.25-0.35!

    Good improvement. I like this speedmod :)
  14. GhaladReam

    GhaladReam Network Guru Member

    Sorry to double post, but I was wondering if anyone else noticed that the SpeedMod 1.19-ND build 8500 archive is corrupt? I can't extract it with any archive app. I also tried downloading it using different browsers. No luck.

    The regular build 8500 extracts fine, it's just the ND build that seems corrupted.
  15. gotamd

    gotamd Network Guru Member

    When upgrading from Tomato (1.19), is it necessary to clear the NVRAM? I've already flashed and everything seems fine, I was just curious.
  16. hardc0re

    hardc0re Network Guru Member

    Hi GhaladReam,

    I just tested it and both archives seem okay. I've had a few people mention corrupted archives as well, but its usually because of a partial download, and can be made worse if your ISP has a transparent proxy.

    You could try to clear your download cache and then download the file again.
  17. hardc0re

    hardc0re Network Guru Member

    Hi gotamd, no need to clear NVRAM from stock Tomato to SpeedMod. No changes there.
  18. GhaladReam

    GhaladReam Network Guru Member

    My browser cache clears automatically when I close it (I have it set to that in firefox). Regardless, I just tried to download it again, and it seemed to work this time. Must have been something on my ISP's end.
  19. Toxic

    Toxic Administrator Staff Member

    GhaladReam you want me to email it to you?
  20. Meffy

    Meffy LI Guru Member

    Does this modify anything related to DNS?Like the cache size etc?
  21. hardc0re

    hardc0re Network Guru Member

    Hi Meffy. No this mod does not change anything related to DNS. DNS cache size is the same as default.
  22. mstombs

    mstombs Network Guru Member

  23. GhaladReam

    GhaladReam Network Guru Member

    Nope I'm good, thanks anyway. It eventually downloaded fine.
  24. Meffy

    Meffy LI Guru Member

  25. kulmegil

    kulmegil Network Guru Member

    *cough* Tomato SpeedMod VPN? ;P
  26. nvtweak

    nvtweak LI Guru Member

    the sources are available, just extract and compile.
  27. i1135t

    i1135t Network Guru Member

    I would try, but don't know the steps.. I have linux, but have no idea how to compile... any helpful suggestions??
  28. nvtweak

    nvtweak LI Guru Member

    Follow the instructions included with the tomato source code. For the VPN build, download the source and read the thread. I imagine the author of that mod made some instructions too.
  29. i1135t

    i1135t Network Guru Member

    Well, there are instructions on how to compile the source code to the BIN file, but what I'm trying to do is combine the two together, such as applying patches of one MOD to the other... any linux gurus here willing to help?
  30. nvtweak

    nvtweak LI Guru Member

    As of right now I can't even get Tomato Mod VPN to compile. I was wrong, as there are no instructions to follow in the Tomato Mod thread.

    EDIT - scratch that, roadkill gave me a tip and I was finally able to compile it. There's a build on the next page if your interested.

    Tomato 1.19 and SpeedMod compiles fine under Debian.

    Attached Files:

  31. Mercjoe

    Mercjoe Network Guru Member

    It has been a little over 3 days and things have started to get wonky on me and the missus. I ran the commands in order above. Here is the cut and paste.

    Tomato v1.19.8500

    BusyBox v1.2.2 (2008.04.25-09:18+0000) Built-in shell (ash)
    Enter 'help' for a list of built-in commands.

    # uptime
    23:25:47 up 3 days, 21:25, load average: 0.02, 0.02, 0.00

    # ps
    PID Uid VmSize Stat Command
    1 root 492 S init noinitrd
    2 root SW [keventd]
    3 root SWN [ksoftirqd_CPU0]
    4 root SW [kswapd]
    5 root SW [bdflush]
    6 root SW [kupdated]
    7 root SW [mtdblockd]
    39 root 264 S buttons
    77 root 300 S redial
    79 root 444 S nas /etc/nas.conf /var/run/ lan
    83 root 460 S pppoecd vlan1 -u XXXXXXXXXXXXXXXXXXXXX -p XXXXXXX -r 1492 -t 1492 -i 0 -I
    92 root 292 S telnetd -p 23
    94 root 368 S crond -l 9
    97 root 340 S rstats
    107 root 336 S httpd
    113 nobody 1624 S dnsmasq
    115 root SW [cifsoplockd]
    116 root SW [cifsd]
    158 root 248 S igmprt -f -i ppp0
    162 root 476 S upnp -D -L br0 -W ppp0 -I 60 -A 180
    2609 root 440 S -sh
    2682 root 348 R ps

    # cat /proc/meminfo
    total: used: free: shared: buffers: cached:
    Mem: 14876672 14147584 729088 0 1425408 4349952
    Swap: 0 0 0
    MemTotal: 14528 kB
    MemFree: 712 kB
    MemShared: 0 kB
    Buffers: 1392 kB
    Cached: 4248 kB
    SwapCached: 0 kB
    Active: 5516 kB
    Inactive: 2584 kB
    HighTotal: 0 kB
    HighFree: 0 kB
    LowTotal: 14528 kB
    LowFree: 712 kB
    SwapTotal: 0 kB
    SwapFree: 0 kB

    # cat /proc/slabinfo | grep ip
    ip_conntrack 137 1620 384 19 162 1
    ip_mrt_cache 0 40 96 0 1 1
    ip_fib_hash 17 113 32 1 1 1
    ip_dst_cache 478 5592 160 25 233 1

    # cat /proc/net/rt_cache_stat
    entries in_hit in_slow_tot in_slow_mc in_no_route in_brd in_martian_dst in_martian_src out_hit out
    _slow_tot out_slow_mc gc_total gc_ignored gc_goal_miss gc_dst_overflow in_hlist_search out_hlist_se
    000001eb 004b9836 000202a3 0000004f 00000000 00000211 00000000 00000000 0002c0e9 00006434 00000064
    00000000 00000000 00000000 00000000 0003a81c 00001dc1


    It has taken me a few minutes to get this posted to you. The whole router is sluggish as can be but it is trying, I will give it that.
  32. nvtweak

    nvtweak LI Guru Member

  33. hardc0re

    hardc0re Network Guru Member

    Hi Mercjoe. The problem does not seem to be related to the SpeedMod changes. Why?

    1. You are using very little connections, not stressing the route cache or conntrack at all. Conntrack is only taking up about 1620 x 384 = 644Kb RAM, and the route cache about 5592 x 160 = 895Kb RAM. Not much at all. The problem is not here.
    ip_conntrack 137 1620 384 19 162 1
    ip_dst_cache 478 5592 160 25 233 1

    2. You still have a lot of free RAM. About 4.9Mb.
    MemFree: 712 kB
    Cached: 4248 kB

    3. I notice that your dnsmasq is taking up a lot of RAM, about 1.6Mb RAM. In my router, my dnsmasq uses up only 0.5Mb RAM.
    113 nobody 1624 S dnsmasq

    Here are some suggestions:

    A. You seem to be using some custom configuration for dnsmasq, otherwise what could be causing your dnsmasq to use so much RAM? Try using the default config for dnsmasq.

    B. You are using rstats (historical bandwidth monitor?) and the CIFS client. These could be the culprits. Try turning off historical bandwidth tracking and CIFS.
  34. Mercjoe

    Mercjoe Network Guru Member

    I have the hosts file onboard my router. That is the dnsmasq issue probably. I also use OpenDNS for the DNS servers.
    Here is my WAN Startup script. It is the only time I mess with dnsmasq

    cut 'n paste of that script:

    logger WAN UP Script Executing
    sleep 5
    test -s /tmp/dlhosts
    if [ $? == 1 ] ; then
    echo -e "#!/bin/sh\nwget -O - | grep | sed -e '2,\$s/' -e 's/[[:space:]]*#.*$//' > /etc/hosts\nlogger DOWNLOADED\nkillall -1 dnsmasq" > /tmp/dlhosts
    chmod 777 /tmp/dlhosts
    cru a Gethosts "45 23 * * 5 /tmp/dlhosts"

    I have a routing set up so I can access my DSL modem thru the router as well.

    Just a simple: iptables -I POSTROUTING -t nat -o vlan1 -d -j MASQUERADE

    I also have two CIFS hosts open. I dump logs and bandwidth data to these two remote systems as well. I have teens on my network and I keep an eye on what they do. I do not sit over their shoulder, but they know that the logs are reviewed and if they are caught doing something they KNOW they are not allowed to do they lose it all.

    The host blocking and the logs are something that I can not live without unfortunately. I have to block what I can from the head of the network or the blasted ads and such from the pages they load would bog my whole network down. This keeps me from having to nag them to update the security software. (speaking of this, the XP SP3 has a 'computer health' monitor for network access. Any ideas on how to read it with the router and DENY them ANY network access till it comes up to spec??)

    Oh well, I was hoping that the speedmod would give me a little boost in performance. I know that I am pushing this little blue box pretty hard and any boost in performance is a plus for me. I know that with the stock Tomato firmware, I can run for weeks with this configuration and never have a problem with general speed.

    Thanks for looking into this.
  35. cpgbg

    cpgbg LI Guru Member

    I don't think that this thing with the hosts file is a good solution. They can always lookup the IP address using other DNS server and open the site by IP. You better use iptables script that blocks the IP addresses of these sites.
    But this still is not a total solution as there are many free proxies they can use.
  36. kulmegil

    kulmegil Network Guru Member

    Big thanks, although I won't be able to unbrick if necessary so maybe I'll wait a bit for some other testers..
  37. hardc0re

    hardc0re Network Guru Member

    Hi Mercjoe,

    Thanks for the input. Still got not much of a clue as to whats causing the problem though. There are too many variables. ;)

    Another user also reported high CPU load after about a week, and he seemed to be using CIFS too. I'm not sure whats the problem, but it could be the bandwidth monitor saving to CIFS that is the cause.

    Also, do you have logging enabled? (Its enabled by default). If you aren't using it, try turning it off. Its at Status > Logs.

    Still, the changes made in SpeedMod should not affect that. Anyway, I'll think about it more to see if we can find the culprit.
  38. nvtweak

    nvtweak LI Guru Member

    # uptime
    08:15:19 up 19 days, 13:04, load average: 0.00, 0.00, 0.00

    My current configuration; I have all of these things disabled. QoS, CIFS, Logging, Bandwidth Monitor (except real-time), Layer 7, Radio, UPnP, DDNS. Basically just a barebones router config, with JFFS2 enabled (for my custom Buffalo theme)

    This is free memory after nearly 20 days.
    Total / Free Memory 14.14 MB / 8,108.00 KB (55.98%)

    (As an experiment, I had downloaded some large files to the router which cleared out some of the cache)
  39. hardc0re

    hardc0re Network Guru Member

    Hi guys n gals, new build of SpeedMod 8501 is released.

    Get it here

    Change: Fixed ip route cache hashtable size for 32MB routers, its now the same as for 16MB routers.
  40. hardc0re

    hardc0re Network Guru Member

    New build 8502 out.

    Get it here


    * Modified IP route cache parameters to be more resistant to "dst cache overflow" and "Rusty's brain broke" errors.

    * Expect about 700kbytes more RAM usage if there are a high number of connections.

    * If you have build 8501 and don't want to upgrade to 8502, you can implement the same changes by putting these lines into your startup script:

    echo 16384 > /proc/sys/net/ipv4/route/gc_thresh
    echo 32768 > /proc/sys/net/ipv4/route/max_size
  41. hardc0re

    hardc0re Network Guru Member

    Build 8502:
    # uptime
    13:02:18 up 12 days, 10:33, load average: 0.00, 0.00, 0.00

    Currently average of about 2000 conntrack entries.

    Free memory is around 3.2Mb, and this is with Bandwidth history and CIFS client enabled. :)
  42. GhaladReam

    GhaladReam Network Guru Member

    Might I ask how you achieve such low load averages? I barely break 200 conntrack entries at any given time (using SpeedMod), and mine are constantly at 0.20, 0.30, 0.25 (on average). Even with no network activity they are this high. I've diabled pretty much every service in Tomato. It doesn't seem to be affecting performance, but it still concerns me, since everyone else's is much lower. Any ideas?
  43. hardc0re

    hardc0re Network Guru Member

    High CPU loads can be caused by a few things, for example:

    1. High WLAN usage (if you are transferring large files over WLAN, CPU load will shoot up for sure).

    2. Using Tomato's GUI features like checking QoS graphs, bandwidth graphs, etc.

    3. Your P2P software is actively making new connections very rapidly (like when you first start it up, or add a new file to transfer).

    Still 0.20 is pretty low and looks fine.
  44. bagu

    bagu Network Guru Member

    Is it planned to update this mod with the 1.20 ?
  45. nvtweak

    nvtweak LI Guru Member

    it was updated but he pulled it due to the problems everyone had with official 1.20.

    maybe have another go 'round when Jon decides to release his final 1.20 source with updated DNSmasq.
  46. fairuza

    fairuza LI Guru Member

    when will speedmod 1.21 be out? i think the victek one has the speedmod changes in it but that one requires you to clear the nvram at each update. i'd rather just use this one, i don't need the other stuff victek added.
  47. PeterT

    PeterT Network Guru Member

    Actually, I believe you only have to clear NVRAM on your initiail conversion to Victek`s mod. Once on his, there is no need to clear in future.

    I think it is in fact recommended to clear NVRAM on moving between ANY of the different famlies of firmware; however on upgrading within a family unless mentioned in the readme`s there is no need to clear NVRAM.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice