My utilities web site revived

Discussion in 'Tomato Firmware' started by rhester72, Apr 10, 2010.

  1. rhester72

    rhester72 Network Guru Member

    Give me a complete list of commands needed to reproduce the problem and I'll see if it happens on my side - if so, it's an issue with the compile, if not, it's a kernel incompatibility.

  2. i1135t

    i1135t Network Guru Member

    Just tried to build a new set using this:
    "./ipset -N private bitmap:ip range" or "./ipset --create private bitmap:ip range"
    This should build a new set based on the man file here
  3. oldpond

    oldpond Networkin' Nut Member

    Is the site down again?
  4. rhester72

    rhester72 Network Guru Member

    Yeah, for another day or so - renovations in the room housing the router, entire room is powered down. :/

  5. rhester72

    rhester72 Network Guru Member

    It's back, but may experience brief (less than an hour) outages over the next day or two as I relocate things. Sorry for the trouble!

  6. rhester72

    rhester72 Network Guru Member

    In both cases, I get this:

    ipset v4.5: Couldn't load settype `bitmap:ip':File not found
    Try `ipset -H' or 'ipset --help' for more information.
  7. i1135t

    i1135t Network Guru Member

    I am not sure how you are getting that but when I try to create the set I get "segmentation fault". I tried many variations here and here and still not able to get it to load within tomato after loading your modules successfully. I will have to try compiling in Ubuntu and see how it works there to know for sure...
  8. rhester72

    rhester72 Network Guru Member

    Again, I suspect that my modules are against a different source-code-level kernel than yours (even though they share a meaningless version, they are _not_ the same kernel), so the only way it will work for you is to self-compile. :/

    Such is life in the land of "there is no One True Tomato(TM)". All of my compiles are based on the tip revision of TomatoUSB (i.e. tomato-RT) - anything else is all but guaranteed to fail w.r.t. kernel modules.

  9. rhester72

    rhester72 Network Guru Member

    It's multimedia day at the Tomato utilities site. Enjoy! :)

    (I also got very close on mkvtoolnix and mediatomb...but not quite close enough, and my frustration with both has reached a level that I'm going to back off of them for a little bit. I shall return. =)

  10. rhester72

    rhester72 Network Guru Member

    fwknop and tcpflow up by request.

  11. rhester72

    rhester72 Network Guru Member

    After more than a year, I finally figured out why my web server wasn't auto-starting properly on router reboot (weird search path issue). Hopefully you'll see better uptime as a result. ;)

  12. awdark

    awdark Networkin' Nut Member

    Hello, I was wondering if it might be possible to get HDhomerun compiled for the tomato firmware. My router uses the MIPS v2 cpu so I'm not sure if the binaries are compatible but I'm trying to get to record for me and I got python working but will need the hdhomerun_config binary.

    If you could build it for me I would really appreciate it.

    Edit: Might not be needed, can't even get the python part of it working without errors. Will see if I can get help on that part.

    Edit 2: I got it from a different source on the hdhomerun forums.
  13. dmitrydn

    dmitrydn Networkin' Nut Member

  14. rhester72

    rhester72 Network Guru Member

    It may very well be that it's not 2.4-compatible (though I can't immediately see why). I get the following:

    tinyproxy: Could not open config file "/opt/etc/tinyproxy.conf".
    (which is normal and expected)
    Try populating a config file in that location and see if it still completely bombs out.
  15. dmitrydn

    dmitrydn Networkin' Nut Member

    No, seems worse scenario is here :)

    # ls -la /opt/etc/tinyproxy.conf
    -rw-r--r--    1 root    root            7 Feb  8 16:43 /opt/etc/tinyproxy.conf
    # /jffs/tinyproxy
    User defined signal 1
    I didn't fill out tinyproxy.conf with the valid information - it's just a stub but anyway it was not help.

    PS: as side question what else could be used as a very light HTTP proxy? Unfortunately I cannot use srelay (by the way it's worked fine on my linksys outside of work place) as websense on my work is prohibit the SOCKS traffic.
  16. rhester72

    rhester72 Network Guru Member

    Putting an open HTTP proxy on the Internet is a _bad idea_, if I'm understanding you correctly. I'd recommend using something like PuTTY ssh tunneling and SOCKS proxying through that (using something like MyEnTunnel on the client or srelay on the router) to avoid the Websense block, but be aware that since you're at work you're taking a risk no matter what. =)

  17. dmitrydn

    dmitrydn Networkin' Nut Member

    You are right Rodney, seems I've lost my attention on resolving the technical issue instead of in first think how it will work at the end :) To be honest I just wondered that the tool doesn't provide at least basic authentication. So thank you for your concern :)

    Will keep my current solution: work -> http authenticated proxy on my local comp -> ssh to tomato = putty tunneling :) Yep very complicated but ssh is blocked by work policy :)
  18. rhester72

    rhester72 Network Guru Member

    Is your work blocking SOCKS, SSH, or both? I'd be awfully surprised if they are blocking SSH to port 443 (for instance), which is how many handle this sort of transparent tunneling (because then all SOCKS traffic is limited to your local machine and tunneled through as SSH traffic rather than SOCKS traffic).

  19. dmitrydn

    dmitrydn Networkin' Nut Member

    As ssh as socks both are blocked. And I've tried to connect to my home router through ssh on 443 port - nothing (but in the same time I can do it in case I've plugged in through GSM network). The same with SOCKS - cannot connect but can do it through HTTP proxy.

    Seems Websense is doing deep packet inspection so the filtering is happen on both levels: by ports and by packets. Any way my solution is working :)
  20. rhester72

    rhester72 Network Guru Member

    Glad to hear it. :)

  21. rhester72

    rhester72 Network Guru Member

    I have good news and bad news.

    The bad news: The site is currently down on port 80, and will remain so for the next week or so. (It is still very much alive on HTTPS and FTP.)

    The good news: I've changed ISPs, and my upstream bandwidth is now approximately 10x what it was. =)

    I'll post again when 80 is back, but for now, enjoy the speed on 21 and 443! ;)

    There's also been a goodie or two put up in recent weeks that I haven't announced...see the CHANGELOG for more details!

  22. dailyglen

    dailyglen Networkin' Nut Member

    Hi Rodney,

    I enjoy using your compiled binaries!

    I tried fwknopd from and I get:

    -sh: ./fwknopd: not found
    Your other binaries work fine. Can you check that it was compiled OK? I was wondering if libpcap was also required but I couldn't get it to work after copying the .so over.

    If anyone has a tutorial on setting up fwknop on tomato please let me know. If not, I'll try to write one.

  23. rhester72

    rhester72 Network Guru Member

    I've (re)broken my head over this one for a couple of hours now - I have NO explanation why, but the compiler is embedding "/opt/lib" in front of the path to uClibc, and dynamically linking it even on a full-static compile (and I've confirmed libtool is doing what it's supposed to, I literally can't figure out where the breakage is).

    A short workaround is to create a softlink from /lib/ to /opt/lib somewhere in your init:

    mkdir -p /opt/lib
    ln -s /lib/ /opt/lib/
    I also updated to release 2.0 final from 2.0rc5.

    Good luck!

  24. rhester72

    rhester72 Network Guru Member

    And of course, I managed to figure it out not long after I wrote the above. Fully static builds of fwknop (and quagga, which suffered a similar but subtly different issue) are up. :)

  25. dailyglen

    dailyglen Networkin' Nut Member

    Hi Rodney,

    Thanks for the updated fwknopd! The binary is working now but I when I try to use it fwknopd it runs into an issue that iptables on tomato doesn't support '-m comment'. Unfortunately the 'comment' match is required for it to close the opened port after the timeout (I tried removing it from the code and tested it on my Linux box but it didn't work).

    I think I'll take up the question of getting iptables support for '-m comment --comment blah' to another thread.

    Thanks again.
  26. rhester72

    rhester72 Network Guru Member

    As promised...back on 80 (but with the higher speeds). Enjoy!

  27. rhester72

    rhester72 Network Guru Member

    dnscrypt seems to be fairly interesting (though in my opinion, unbound with DNSSEC is a lot more so), so I've gone with the flow and put a binary up. The already-described-by-others caveats regarding having the correct time on your router and the resulting race condition with dnscrypt *MUST* be heeded!

  28. lancethepants

    lancethepants Network Guru Member

    Thanks for all those binaries, I'll be having fun with some of them for a while.
    I had a question too. I like to compile the occasion application on my router. Could you tell me how to compile it so that it uses the libraries already found in tomato instead of the ones from optware? Also, could you tell me how to create a static binary so it doesn't have external dependencies on other libraries?
  29. rhester72

    rhester72 Network Guru Member

    re: many cases, you can't. The reason is that the libraries as included with Tomato are not complete - they are stripped to contain only the functions actually utilized by Tomato. You can compile against them, and things might seem to work, but sooner or later your app will likely crash once it hits one of the missing functions.

    re: static depends. For very simple builds, adding "-static" during the linker phase will have the desired outcome. For others, it may be a configure flag or something more hairy (like any linkage done by libtool). It's a case-by-case basis.

  30. lancethepants

    lancethepants Network Guru Member

    I know some of the libraries are pretty stripped down in tomato, like OpenSSL. In those cases I just use a separate library. Well I know when I cross-compile, it will automatically look to /lib for those libraries, like,,, libgcc etc. However when cross compiling it wants the optware versions. How can I make it look to /lib for those versions instead? and I can compile any crypto/compression libraries that may be stripped down.
  31. rhester72

    rhester72 Network Guru Member

    It should default to /lib (or /usr/lib), depending on how your JDK/gcc was built (are you using the Tomato built-in?). That having been said, setting the LDFLAGS environment variable to include various custom library search paths is often sufficient (with sane Makefiles).

  32. rhester72

    rhester72 Network Guru Member

    Forgot to add unbound to the update list. Works rather well!

  33. danringer

    danringer Serious Server Member

    I've been hunting for a working ipk of netatalk for tomato for a couple of days. I'm not able to cross compile on my set up, lack of knowledge. And I've tried your avahi binaries with success, and was hoping you can help with netatalk. The existing netatalk ipk from ipk-opt doesn't support shadow passwords, like tomato uses, and has some complications communicating with libdb.

    See comments re libdb errors with cnids
    I was able to install netatalk but had those cnid errors and could not auth against shadow only guest logins.

    Some good info here re: compile flags for shadow password support.
    When you have time I'd greatly appreciate you adding these to your static binaries tarball.
    Thank you for your dedication to these tomato boxes and our community.
  34. rhester72

    rhester72 Network Guru Member

    I've put an "expermental" static compile of the netatalk 3.0 binary, using libdb 4.8 and with shadow support, in PRECOMPILED-static. It is still using threaded libdb, and I can't see why that would be an issue, but if it does cause problems in your testing, please let me know.

    I'll promote it to STABLE after you've tested and we've resolved any issues.

  35. koitsu

    koitsu Network Guru Member

    The "proper" way to solve this is to use the -Wl,-rpath=/some/path/lib argument when passed to gcc (used during link-time only), or just -rpath=/some/path/lib if using ld directly. Reference:

    This causes the underlying ELF binary to actually have pathnames stored in it which it prefers first, and completely rids you of the absolute stupidity that is LD_LIBRARY_PATH. This is also how the Entware project (not Optware) deals with the situation. Proof is below (using opkg as an example, although it's not a very good example for reasons I'd rather not get into here):

    root@gw:/# readelf -d /opt/bin/opkg | grep RPATH
     0x0000000f (RPATH)                      Library rpath: [/opt/lib]
    root@gw:/# ldd /opt/bin/opkg
   => /opt/lib/ (0x2aac0000)
   => /opt/lib/ (0x2aae1000)
   => /opt/lib/ (0x2aaa8000)
  36. danringer

    danringer Serious Server Member

    Wow Rodney, that was fast. Thank you. With the other 2 binaries I used from your site I installed recent ipk and replaced the binaries with yours, it provided all other tools and vanilla configs. I'm unable to find a current ipk package for tomato/ddwrt for netatalk 3.0, do you have the ability to build an ipk of 3.0 including the other binaries in the suite, or could you build a 2.2.2, 2.2.0, or 2.1.5 set with those flags:

    $ CFLAGS="-O -DSHADOWPW" CPPFLAGS="-I/opt/include" LDFLAGS="-L/opt/lib -Wl,-rpath,/opt/lib" ./configure --prefix=/opt --with-ssl-dir=/opt/ --with-bdb=/opt/ --with-shadow --with-cnid-cdb-backend --enable-afp3

    Either way an ipk would be easiest. Optimally I'd like a 2.2.2 ipk with shadow with a matching libdb ipk to handle the cnid db.

    Sorry if the non programmer in me requires such hand holding. Tool chain failed to build on osx and I haven't had the chance to install Ubuntu in a virtual space to retry it.
    Thanks again for you amazing speed and dedication to the community.

  37. rhester72

    rhester72 Network Guru Member

    Honestly, I don't have the ability to build ipk packages at all...part of it is laziness, and part of it is quite deliberate. As I indicate in my README, I think you learn a _lot_ more from a hand-install than you do from a one-shot, hands-off packaged solution. Don't get me wrong, I have total respect to those who build them for the community, I've just made a conscious decision not to do so myself. As mentioned above, Entware is a _great_ way to go for pre-packaged solutions.

    Of the CFLAGS you provided, SHADOWPW is already present (and should be handled by a proper configure, I don't like the debug flag hack), /opt/usr/lib is the preferred RPATH I use (since /opt/lib should really be for "system" libraries, though I support it as well), prefix is /opt, the ssl-dir and bdb options are flat wrong for a build. I've rebuilt with the CNID cdb backend and I believe afp3 is now builtin without configuration in 3.0.

    I've pulled the static binary from PRECOMPILED-static since I don't know exactly what the default installation tree looks like, but *all* of the binaries can be found in their post-compile locations in the netatalk-3.0 directory (and subdirectories). If you're able to get something going with a direct-binary-replacement solution as above, I'd love to hear about it.

    By the way, from my limited build experience with netatalk for Tomato, I'd strongly recommend use of the --without-kerberos configuration option as well.

  38. andg

    andg Serious Server Member

    Seems like the site is down again...? I'm really interested in getting the netatalk 3.0 package with shadow passwords.
  39. rhester72

    rhester72 Network Guru Member

    This is the second report I've seen of this - it shouldn't be (and didn't appear to be), but just to be on the safe side, I bounced (and then upgraded) Hiawatha. Give it another try.

  40. Dark_Shadow

    Dark_Shadow LI Guru Member

    I get a blank page
  41. rhester72

    rhester72 Network Guru Member

    Still looks OK here, clear your cache maybe?

  42. Dark_Shadow

    Dark_Shadow LI Guru Member

  43. rhester72

    rhester72 Network Guru Member

    Oh, I went away from mooo after the whole FBI/porn thing about two years ago. *laughs* Their reputation certainly isn't what it was, plus I get *REALLY* low TTL and IPv6 with these guys. =)

  44. Morac

    Morac Network Guru Member

    I recently upgraded from Toastman's tomato-E3000USB-NVRAM60K-1.28.7498.1MIPSR2-Toastman-RT-VPN to tomato-E3000USB-NVRAM60K-1.28.7500.4MIPSR2-Toastman-RT-VPN and now all the pre-compiled executables (curl, unrar, openssl, and tcpdump) I downloaded off your site back in June 2012 are throwing segmentation faults.

    I tried to go to your site to see if there's an updated version, but the site is down.

    All I really want is curl, I don't need Optware. Any ideas on how to fix this?
  45. Morac

    Morac Network Guru Member

    I double checked and the original static compiled curl returns "illegal instruction", not "segmentation fault".
  46. koitsu

    koitsu Network Guru Member

    Regarding Morac's problem, I had written up an explanation in a different thread -- and people should read his follow-up post for more information:

    There are a multitude of possibilities here:

    1. Binaries were built on/for a different CPU architecture than what your router is
    2. ELF format change (meaning the actual binary file format itself)
    3. If /opt is on USB, possibly a USB layer bug/problem that manifests itself in a nefarious way
    4. If /opt is on a flash drive, possibly the flash drive is going bad (bad blocks/sectors)
    5. Possible filesystem corruption on the underlying /opt filesystem itself
    6. A very strange, but catastrophic, hardware failure

    I would do some analysis of the binaries rhester72 has up on his site, except his site seems down again.

    $ date
    Fri Sep 28 09:06:28 PDT 2012
    $ telnet 80
    I can assure you that Entware (NOT the same thing as Optware) works fine on my Asus RT-N16 running tomato-K26USB-1.28.0500.5MIPSR2Toastman-RT-N-Ext.trx. My /opt filesystem is a CIFS mount.

    root@gw:/tmp/home/root# which curl
    root@gw:/tmp/home/root# curl --help | head -2
    Usage: curl [options...] <url>
    Options: (H) means HTTP/HTTPS only, (F) means FTP only
    root@gw:/tmp/home/root# opkg list-installed | grep curl
    curl - 7.23.1-1
    libcurl - 7.23.1-1
    root@gw:/tmp/home/root# file /opt/bin/curl
    /opt/bin/curl: ELF 32-bit LSB executable, MIPS, MIPS32 version 1 (SYSV), dynamically linked (uses shared libs), with unknown capability 0xf41 = 0x756e6700, with unknown capability 0x70100 = 0x1040000, stripped
    I've also (historically) used rhester72's binaries without issue, but that was many firmware versions ago.
  47. Morac

    Morac Network Guru Member

    Unfortunately my USB stick is formatted FAT32, so I can't install optware. In fact I never installed Optware and never used /opt. That's the reason I was using static compiled versions since they are stand alone. That worked fine until I updated to a newer version of Toastman. I just want to point out that I never updated the binary applications. They are the same ones that were working prior to updating Toastman.

    I did install Entware underneath the /tmp/opt folder just to see if it would work and it does. That's on my Linksys E3000. I can then copy the curl executable to my USB stick and run it from there and it works (since the libraries are in /tmp/opt), so I can run executables off my USB stick.

    In any case I can eliminate a number of the possibilities above:
    1. Not likely since the binaries worked prior to upgrading to the latest Toastman.
    2. I didn't update the binaries, so this would have to be do to updating the firmware
    3. No, since I'm not using /opt
    4. No, see #3
    5. Again no, see #3
    6. Not likely.
  48. koitsu

    koitsu Network Guru Member

    ELF format changes can happen in either the underlying loader, the kernel, or the binaries themselves (rebuild on a different system, etc.).

    As I see it, since Entware works for you this rules out the above possibilities (meaning I do not see this as a Toastman build problem, nor do I see it as a issue, kernel issue, or hardware issue).

    Debugging this kind of problem is going to be extremely difficult given the lack of decent debugging environment on embedded hardware. You're going to need a lot of tools installed to debug this, and quite possibly a serial port.

    Welcome to why there is no such thing as an "all-in-one binary that will work on every OS/kernel/build/environment"; people think that just because something is statically linked that somehow it magically will work everywhere forever. That is not the case.

    I would recommend just sticking with Entware. You might file a bug/ticket with them asking if there is some way they might be able to make their suite work better (natively) on a FAT32 filesystem. I can assure you the response will very likely be "why are you using a non-extX filesystem USB stick for something that you use for Linux libraries and binaries?" (It's a legitimate question too, since it implies you'd be plugging/unplugging the stick to move it to a Windows machine on a regular basis)
  49. Morac

    Morac Network Guru Member

    I wouldn't mind formatting to extX if it was simply, but I don't want to go through the hassle of booting into Linux on my machine. Can this be done on the E3000 itself in TomatoUSB?

    I'll mention someone statically compiled curl 7.27.0 for me and it works. Unfortunately it's buggy and won't validate a valid certificate for some reason.
  50. rhester72

    rhester72 Network Guru Member

    Sadly, the site is indeed down...likely for quite some time. I've had more than my fair share of bad luck with drive failures and the last one was critical. Maybe again someday. :/

  51. koitsu

    koitsu Network Guru Member

    Yes, this can be done easily through the CLI (the GUI may offer a way to do it but others would be able to help with that). The commands you want to use are mkfs.ext2 or mkfs.ext3, depending on if you want to use ext2 or ext3, to make a new filesystem. However, I'm not sure how on Tomato to go about changing the partition type (from NTFS to Linux (and not Linux swap)), since the "fdisk" utility that comes with Busybox is a badly-documented pile of junk. You shouldn't need a Linux machine to accomplish this though, it should be 100% doable from the router, I just don't know the user-friendly commands to do it.
  52. koitsu

    koitsu Network Guru Member

    Rodney, I am happy to donate one of these disks to you for your own use, free of charge. I can assure you the drives are all in good/working condition (I know quite a lot about storage subsystems in general and have done data recovery for more people than I can count), sans the 2 brand new ones which I haven't tested (and don't plan to -- rather keep them new). Let me know if that'd help you. Thanks.

    P.S. -- I should note I won't have the 2TB disk available until late next week. (I'm replacing it with something that runs slower but cooler, since temperature is important for where I live)
  53. rhester72

    rhester72 Network Guru Member

    That's very much appreciated, honestly. So far, in a period of 7 weeks, I have lost 2x2TB drives and 1 3TB drive, including pretty much the entire /opt tree (I was indeed backing it up indirectly, via tar then copy to another host that is backed up via cloud...unfortunately, I didn't notice that an error in the tar script made it recursive and thus it corrupted, so the backup is present but unusuable). I have a tertiary cache of all the Tomato-related content that was on the drives, and a 60GB microdrive I was thinking of leveraging when I have the time and nerve to rebuild everything (the web server infrastructure, rsync, the init scripts, basically the whole router config) from scratch. It's really more of an issue of free time than of hardware, unfortunately.

    If someone wants to interim mirror what _was_ there, I'm happy to upload a tarball of the entire site as it was for temporary hosting until I can get things back online.

  54. Monk E. Boy

    Monk E. Boy Network Guru Member

    Actually I use the CLI on Tomato to partition & create ext3 USB flashdrives regularly. It's not all that complicated, but I do sit around googling each time to figure it out (months go by between partitioning & formatting, so my grey matter gets a bit fuzzy).

    I could probably write up some fairly basic instructions if someone needs to know how. FWIW, fdisk on Tomato is nigh-identical to the Redhat boxes I used to administer a few years ago.

    A word of warning: You will lose all data on the flash drive, and I'm not aware of any good way to mount ext3 on Windows, so don't expect to be moving the flash drive between Tomato & Windows anytime soon.
  55. koitsu

    koitsu Network Guru Member

    Re: ext2/ext3 on windows -- this might help (just briefly Googled this and pasted it, so no idea how reliable this software is):

    I'd love to read the little guide if you ever have the time to write it up, Monk E. Boy. The last time I had to use fdisk on Linux was back when it was interactive (and I'm not thinking of cfdisk; I'm thinking of fdisk as part of Slackware back in 1996 or so).
  56. Monk E. Boy

    Monk E. Boy Network Guru Member

    I should be setting up a RT-N16 this week, including its flash drive, so I'll try to remember to write up some howto steps.

    I've started setting up swap partitions too, though I don't run swapon to enable it, but I like the theory of having a partition there in case I momentarily need to use it in the future (for entware, optware, etc.). certainly looks promising, the options before it didn't work in a Windows 7 x64 environment but since that was updated to work with Vista x64 I suspect it will work. I think the IFS system got a security update fairly recently though, and with Microsoft updates usually mean compatibility quirks. I'll run some tests when I get home...
  57. Monk E. Boy

    Monk E. Boy Network Guru Member

  58. kasteleman

    kasteleman Serious Server Member

    where can i get the mdnsresponder? Can i also use it on my asus 66U with shibby 64k 102 build?
  59. rhester72

    rhester72 Network Guru Member

    The good news: I've now gotten screen and the IRC II client working, in addition to updating pixelserv to v30.

    The bad news: I still have no online FTP or web presence whatsoever.

    I'm going to try to do something about that over the next couple of weeks.

  60. rhester72

    rhester72 Network Guru Member

    Noticed a few dozen hits over the past few minutes - I guess somebody's still out there looking for the site.

    You might notice that something actually answers the phone now. This is not an accident. :)

    Stay tuned.

  61. rhester72

    rhester72 Network Guru Member

    I'm baaaaack. :)

    Directory path for HTTP is back to the original (see signature), I have no record of the blog article I wrote introducing the site and no time to recreate it now. FTP is also back online, and all utilities updated to current.

    kasteleman, mDNSResponder should be available to you now, though I can't say whether it will work on your unit - feel free to test.

    For some reason, I can't edit the OP anymore, but I'll keep the URLs current in my sig.

  62. kasteleman

    kasteleman Serious Server Member

    Resther72. Thx. Got the mdnsresponder running. Only get a error if my sleepproxyclient on my linuxbox tries to register itself:
    Failed to register with .......i'm only running the mdnsresponder and the debug says failed to send goodbye for....Am i missing something? Do i have to run something else like mdnsd?
  63. rhester72

    rhester72 Network Guru Member

    Unfortunately, I don't know a great deal about was compiled at a user's request and never tested (to the best of my knowledge). Hopefully someone with some experience will reply with some insight, but if by some chance you crack the nut yourself, please let us know so I can document any anomalies in the NOTES file.

  64. kasteleman

    kasteleman Serious Server Member

    Ok. Did some reading. Need mdnsd, dnsextd and mDNSResponder running. The last one can promote itself on the network as sleep proxy by the command: mDNSResponder -b -v 2 -n RTN66U -t _sleep-proxy._udp -x
    But the client then wants to register itself at the dnsserver with extended parameters indicating it is going to sleep. But therefore you need dnsextd which can extend the records in the dns. Hope i got it right. But now i need to get mdnsd and dnsextd running. First of all i did a ldd check for the components and downloaded some aditional files from you're site. Now figuring out the config files for mdnsd and dnsextd.....
  65. rs232

    rs232 Network Guru Member

    Hi Rodney
    can we please have portproxy statically compiled?

    This is a priceless piece of software if you have scenarios with thevices connected to Internet via two routers both performing NAT.

  66. rhester72

    rhester72 Network Guru Member

    Should be up in a few moments.

  67. rs232

    rs232 Network Guru Member

    Fantastic. I'm already running my tests. The latest version for some reason forces IPv6 which I don't have.
    Is there any chance you can compile the previous version?

    2010-03-02 Portproxy update. It now supports --persistent parameter. It is possible to specify to bind IPv6 address or interface, even when this interface or address is not available yet. If socket binding fails, portproxy will sleep for 10 seconds and then try again. Also, if IPv6 address is not specified, portproxy will try to read it from specified WAN interface (--wan-interface).
    2009-12-29 Complete code drop, with firmware images, documentation and source code. Extended NAT-PMP support added. First code drop with GUI supporting manual port forwarding and portproxy interface.

    If successful I'll publish an howto on this.

  68. rs232

    rs232 Network Guru Member

    Oh never mind, there's a typo in the options, I've found my way, leave it with me!
  69. mstombs

    mstombs Network Guru Member

  70. rhester72

    rhester72 Network Guru Member

    It comes from here:

    There does not appear to be very much documentation, agreed. I'm assuming you're hoping one of the Optware or Entware folks figures something out and makes an ipk out of it? (That isn't really something I do much of...)

  71. mstombs

    mstombs Network Guru Member

    I'm not personally interested in an ipk for crowdcontrol - I assume it would need an adblock type script to invoke - but I have noticed that D-Link have it in their AthSDK GL src (for example for DIR-825) and wouldn't want tomato users to not have the option to investigate... and I do know how good you are at compiling things that work!
  72. rhester72

    rhester72 Network Guru Member

    Oh, OK, I interpreted "compiles very easily" as it being commonly available in binary form already.


  73. rhester72

    rhester72 Network Guru Member

    Well, I wasn't first to the party by a mile, but Tomato now has native parted for those with drives > 2TB. No static build yet, hopefully soon.

  74. rhester72

    rhester72 Network Guru Member

    Why *not* a ncurses-based multiprotocol chat client directly on the router? ;) Solves first-world problems!

    centerim added.

  75. i1135t

    i1135t Network Guru Member

    Ok, I know this is an old thread but I have been able to load ipset using Rodney's modules with entware ipset install. Problem is when I try to issue an IPTABLES command to test the set, I get this error:

    iptables v1.3.8: Couldn't load match `--set':File not found
    I think the iptables (Toastman 1.28.7501.3 STD mod) don't understand the "set" option. I don't know if I just point to the .so files through a command? If not how is the best way to get this working since I am almost certain it would work once iptables can decipher this option.
  76. koitsu

    koitsu Network Guru Member

    The syntax of the iptables command you're using wrong. Is there some reason you didn't provide the full command you're using?
  77. i1135t

    i1135t Network Guru Member

    Sorry forgot to include that. This is what I used:

    iptables -A OUTPUT -m set --set NoMoreGoogleDNS dst -j DROP
    Got it from here
  78. koitsu

    koitsu Network Guru Member

    Thank you. The issue is that you need to insmod ip_set.ko and ipt_set.ko and other related modules. That's made quite clear in the below post, including the full list of modules that needs to be loaded:

    They need to be loaded in the order shown, but they can be placed in alternate locations (such as /tmp or on a USB flash drive, etc.). You probably need to load all 4; I would have to examine the module symbols to know if they depend upon one another. Don't take any chances, just load all 4.

    If insmod complains/spews an error, then the issue is with the modules you've downloaded, i.e. they will not work with the firmware you're using due to kernel differences. They cannot be "hacked" to work. You will need to compile them yourself for Tomato/TomatoUSB, or you will need someone to build them for you. ryzhov_al does hang out on this forum as well, but generally speaking he builds stuff for a different firmware, and that firmware/kernel is slightly different than TomatoUSB.

    These probably will not work on Tomato/TomatoUSB; these are for a different firmware/kernel.

    You can verify the modules loaded by using lsmod. All 4 of them should show up.

    The iptables -m set and --set commands (particularly the 2nd one) will not work until those modules are loaded; the argument parsing/support is actually within the module itself (yes really! That's how iptables modules work).

    Normally iptables would load these modules dynamically when encountering -m xxx, however because they're located in an alternate path outside of the modprobe path (which for IPv4 modules of this sort would be /lib/modules/ for most recent TomatoUSB builds), you have to insmod them manually first.

    Finally: can I ask what this has to do with rhester42's utilities web site? I don't understand the relevance of your request to this thread. I probably overlooked something, so if you could explain it to me, that'd be great. Thanks.
  79. rhester72

    rhester72 Network Guru Member

  80. i1135t

    i1135t Network Guru Member

    Yes, the modules load and work. I can insmod the (Rodney's) modules and see the modules loaded using lsmod. Ipset commands work and I can create sets this time (but using the one built with entware - the one on Rodney's site didn't work the last time I tried). As stated before, it does NOT work when trying to apply with iptables and errors out probably because it cannot find the modules in the kernel path. With Koitsu's explanation it, I do not see them loaded there nor in the modules.dep file meaning it really isn't loading fully, I guess.

    Aw well, was worth a shot and would have been a nice feature to exclude external countries like China or Nigeria (known phishing countries) from being reachable from my network. A good security tool nonetheless.
  81. koitsu

    koitsu Network Guru Member

    You can block netblocks of those countries without using any of those modules. The ipt_set.ko module / details in the linked thread just make extremely large quantities of rules not affect CPU load as much.

    The error shown:

    iptables v1.3.8: Couldn't load match `--set':File not found
    Should indicate you ran iptables like so: iptables -A {whateverchain} -m --set ...

    Note the missing word set after -m, but before --set.

    You can try iptables -m set -h or iptables -m set --help and see if that gives some additional output (would be at the very very bottom of the output -- you have to pay very close attention to the difference in output between iptables -h and iptables -m xxx -h).

    If you've already insmod'd the modules and they loaded fine, then they are active/working, and iptables has nothing to load. You still have to use the -m xxx argument to indicate what module (a.k.a. "match") you want to refer to. ipt_set.ko supposedly uses a match name of set, hence -m set.

    iptables will try to automatically load a kernel/netfilter module if it isn't already loaded when using -m xxx. You said insmod worked fine, and all 4 modules show up when using lsmod, so the problem seems to lie in the command syntax that's being used. At least that's my impression.

    It would help if you could provide full output, verbatim, of what you're entering into the shell. Please don't hide anything, please don't remove lines, etc. -- show everything you're doing. Thanks.

    Footnote: I do not understand why there are two modules, one called ipt_SET.ko and one called ipt_set.ko; note the case sensitivity. I hope someone can explain this.
  82. i1135t

    i1135t Network Guru Member

    So basically these are the steps I did:

    #install ipset from entware
    opkg install ipset4

    #scp modules from Rodney's website to dir he mentioned on the router /opt/lib/modules
    #Note the other .so files already exist from entware package under /opt/lib/ipset so I didn't scp the .so files that Rodney mentioned to the path he directed (iptables rule still fails with or without them under /opt/usr/lib/ipset as instructed by Rodney here.

    #insmod all 14 modules even though only 4 or so are needed (doesn't hurt)
    insmod ipt_set
    insmod ipt_SET
    insmod ip_set_setlist
    insmod ip_set_portmap
    insmod ip_set_nethash
    insmod ip_set_macipmap
    insmod ip_set_iptreemap
    insmod ip_set_iptree
    insmod ip_set_ipportnethash
    insmod ip_set_ipportiphash
    insmod ip_set_ipporthash
    insmod ip_set_ipmap
    insmod ip_set_iphash
    insmod ip_set

    #lsmod to show modules have been loaded successfully
    root@T1:/opt/lib/ipset# lsmod
    Module                  Size  Used by    Tainted: P 
    ipt_set                  992  0
    ipt_SET                1216  0
    ip_set_setlist          3536  0
    ip_set_portmap          3504  0
    ip_set_nethash          8144  0
    ip_set_macipmap        2608  0
    ip_set_iptreemap        9888  0
    ip_set_iptree          4976  0
    ip_set_ipportnethash    9936  0
    ip_set_ipportiphash    7824  0
    ip_set_ipporthash      7152  0
    ip_set_ipmap            3152  0
    ip_set_iphash          6000  1
    ip_set                14528 24 ipt_set,ipt_SET,ip_set_setlist,ip_set_portmap,ip_set_nethash,ip_set_macipmap,ip_set_iptreemap,ip_set_iptree,ip_set_ipportnethash,ip_set_ipportiphash,ip_set_ipporthash,ip_set_ipmap,ip_set_iphash
    tun                    6464  2
    tcp_vegas              1664  1
    ip6table_mangle          992  0
    ip6table_filter          704  0
    xt_webmon              16320  1
    xt_recent              6800  4
    xt_IMQ                  736  0
    imq                    2320  0
    usblp                  11312  0
    ohci_hcd              17232  0
    ehci_hcd              34640  0
    hfsplus                79216  0
    hfs                    49008  0
    vfat                    9216  0
    fat                    46000  1 vfat
    ext2                  55520  0
    ext3                  113440  1
    jbd                    48352  1 ext3
    mbcache                4528  2 ext2,ext3
    usb_storage            33120  1
    sd_mod                21376  2
    scsi_wait_scan          384  0
    scsi_mod              75488  3 usb_storage,sd_mod,scsi_wait_scan
    leds_usb                2128  0
    led_class              1552  1 leds_usb
    ledtrig_usbdev          2464  1 leds_usb
    usbcore              114736  6 usblp,ohci_hcd,ehci_hcd,usb_storage,ledtrig_usbdev
    nf_nat_pptp            1440  0
    nf_conntrack_pptp      3808  1 nf_nat_pptp
    nf_nat_proto_gre        1072  1 nf_nat_pptp
    nf_conntrack_proto_gre    2464  1 nf_conntrack_pptp
    nf_nat_ftp              1568  0
    nf_conntrack_ftp        5792  1 nf_nat_ftp
    nf_nat_sip              5920  0
    nf_conntrack_sip      19008  1 nf_nat_sip
    nf_nat_h323            5504  0
    nf_conntrack_h323      37152  1 nf_nat_h323
    nf_nat_rtsp            3600  0
    nf_conntrack_rtsp      4528  1 nf_nat_rtsp
    wl                  1781264  0
    et                    49280  0
    igs                    13680  1 wl
    emf                    17376  2 wl,igs
    #run ipset commands to build test set
    ipset -N NoMoreGoogleDNS iphash
    ipset -A NoMoreGoogleDNS
    ipset -A NoMoreGoogleDNS

    #list available sets to verify they exist
    ipset -L
    Name: NoMoreGoogleDNS
    Type: iphash
    References: 0
    Header: hashsize: 1024 probes: 8 resize: 50

    #build iptables rules to enforce built set
    Error occurs here!

    root@T1:/opt/lib/ipset# iptables -A OUTPUT -m set --set NoMoreGoogleDNS dst -j DROP
    iptables v1.3.8: Couldn't load match `set':File not found
    Try `iptables -h' or 'iptables --help' for more information.
    By the way, iptables -m set or iptables -m set --help give same output
    root@T1:/opt/lib/ipset# iptables -m set -h
    iptables v1.3.8: Couldn't load match `set':File not found
    Try `iptables -h' or 'iptables --help' for more information.
    root@T1:/opt/lib/ipset# iptables -m set --help
    iptables v1.3.8: Couldn't load match `set':File not found
    Try `iptables -h' or 'iptables --help' for more information.
  83. RMerlin

    RMerlin Network Guru Member

    I suspect your issue is the shared libraries aren't getting loaded when iptables (the userspace tool) tries to use the module.

    Try copying them to /opt/lib/iptables/ just to see what happens (instead of /opt/lib/ipset).
  84. i1135t

    i1135t Network Guru Member

    Nope, created /opt/lib/iptables dir and dropped the .so and .ko files into that dir and still no luck. :(
  85. koitsu

    koitsu Network Guru Member

    Thanks for the verbose explanation, i1135t. I appreciate it (honest/truly!). The error you've shown is different from the previous error you reported -- the previous error said "Couldn't load match `--set'", while the new error says "Couldn't load match `set'" (which makes a lot more sense).

    RMerlin's theory sounds "mostly" correct to me. The tricky part of understanding iptables/netfilter is that there is the kernel (netfilter) module pieces (.ko files) which within the kernel do the actual packet matching and "heavy lifting", but also (more often than not) there's a userland shared library also used (.so and .so.* files) which is used as a stepping stone between iptables (the program) and the related kernel bits.

    I believe -- meaning boy I sure hope it's done this way -- iptables (userland program) attempts to dynamically load the shared library bits (probably via dlopen() and friends), which then in turn refer to/call the related kernel module (netfilter module) bits. I forget exactly where the command-line parsing portions happen (I'd need to go review the xt_string stuff I've poked at in the past).

    What this means is that both userland and kernel have to be in sync.

    The URL rhester42 provided does not contain any .so files. All that's there are kernel modules.

    So either the .so files are part of the Entware/Optware package you've installed (and there is absolutely zero guarantee that package/those libraries will work with the ones rhester42 built -- see what I said above how everything needs to be in sync) and they're not being loaded (without any errors being shown, e.g. missing symbols, etc.) or possibly not found (e.g. they're not in the search path, which is very likely, and possibly this can be worked around by setting LD_LIBRARY_PATH (sigh, I really do not recommend this, it's a horrible hack)), OR, they ARE being found and the kernel/netfilter modules being out of sync with userland (or vice-versa) is causing grief.

    Is there no Entware/Optware package for the kernel modules that ipset4 refers to? There would need to be. If there isn't, that package sounds mostly worthless.

    And this doesn't even begin to touch on the possibility of kernel/netfilter modules being built from source with related kernel configuration bits that have certain features/symbols in use, where on a different kernel source/firmware those symbols would be missing (or possibly the ABI semantics changing).

    When it comes to netfilter stuff of this sort, there really can't be a "generic" package that works across all routers, even if the kernel version number is the same. Honestly to get this to work, as I see it, you'd need to build your own firmware for TomatoUSB + work ipt_set and friends into the netfilter code source (probably via some patch) and get it all to build -- and then, using the same kernel source (for TomatoUSB), build the userland portions for ipset, that way ensuring everything is in sync.

    I'm sorry this post of mine is so long, but there's really no easy way to explain all of this. This is all stuff developers/programmers/sysadmins (which I am one) end up dealing with/understanding, and is often not something end-users get. I've also never been very fond of the iptables "match/module" concept, solely for this exact reason (this approach on the BSDs would be shunned big time, solely because of the possible screw-ups like this)
  86. i1135t

    i1135t Network Guru Member

    It's on the website under /tomato/PRECOMPILED/lib/ipset/ but I tried it with those too and it doesn't work. RMerlin may be right where the iptables are trying to call these modules and cannot find them under their normal kernel path and bombs out. :( Ah well... maybe someone will build these into the firmware mods later on. Was worth a good try...
  87. rhester72

    rhester72 Network Guru Member

    If you do "ln -s /opt/lib/iptables /etc/iptext" and try it again (after loading the kernel modules, of course), does the behavior change at all?

  88. rhester72

    rhester72 Network Guru Member


    "This module matches IP sets which can be defined by ipset(8), where flags are and/or and there can be no more than six of them. Hence the command

    iptables -A FORWARD -m set --set test src,dst

    will match patchets, for which (depending on the type of the set) the source address or port number of the packet can be found in the specified set. If there is a binding belonging to the mached set element or there is a default binding for the given set, then the rule will match the packet only if additionally (depending on the type of the set) the destination address or port number of the packet can be found in the set according to the binding."


    "This modules adds and/or deletes entries from IP sets which can be defined by ipset(8). add the address(es)/port(s) of the packet to the sets delete the address(es)/port(s) of the packet from the sets, where flags are and/or and there can be no more than six of them. The bindings to follow must previously be defined in order to use multilevel adding/deleting by the SET target."

    Translation: They do pretty much the same thing, and I'm guessing one was deprecated in favor of the to which came first, your guess is as good as mine. Both are still present in the kernel source and neither provides a clue as to their provenance.

  89. i1135t

    i1135t Network Guru Member

    Sorry, same error.
  90. raphaelmg

    raphaelmg Networkin' Nut Member

    Hi people!
    I was wondering if anyone knows something about Rodney´s page (my utilities web site revived) because i´ve been trying to access it couple months and nothing...
    Plz if anyone knows some information about Rodney´s great stuff let me know.

  91. Toastman

    Toastman Super Moderator Staff Member Member

    According to the forum records, Rodney has not been seen on here since Nov 22 2014. Perhaps he has lost interest in Tomato or something else has happened.
  92. koitsu

    koitsu Network Guru Member

    What utility are you wanting? Entware-ng is actively maintained and offers quite a lot.
  93. raphaelmg

    raphaelmg Networkin' Nut Member

    Thank you guys!

    I will follow koitsu advice and search on Entware-ng for my utility.
    It´s called fwknop a port knocking utility.

    Thanks again people!

  94. koitsu

    koitsu Network Guru Member

    There are two Entware-ng fwknop packages: the first is the client, the other is the server/daemon:

    root@gw:/tmp/home/root# opkg list | grep fwknop
    fwknop - 2.6.9-1 - Fwknop implements an authorization scheme known as Single Packet Authorization (SPA) for Linux systems running iptables.  This mechanism requires only a single encrypted and non-replayed packet to communicate various pieces of information including desired access through an iptables policy. The main application of this program is to use iptables in a default-drop stance to protect services such as SSH with an additional layer of security in order to make the exploitation of vulnerabilities (both 0-day and unpatched code) much more difficult. This package contains the fwknop client.
    fwknopd - 2.6.9-1 - Fwknop implements an authorization scheme known as Single Packet Authorization (SPA) for Linux systems running iptables.  This mechanism requires only a single encrypted and non-replayed packet to communicate various pieces of information including desired access through an iptables policy. The main application of this program is to use iptables in a default-drop stance to protect services such as SSH with an additional layer of security in order to make the exploitation of vulnerabilities (both 0-day and unpatched code) much more difficult. This package contains the fwknop daemon.
    I determined which was for what using opkg info {packagename} and read the output very very carefully.

    Good luck!
  95. fonos

    fonos Serious Server Member

    According to the Wayback Machine, the last good record for the Web address in the first post of this thread is August 23, 2012. There are later records but they all show a 404 error.*/

    The newest entry in the Tomato Utilities section seems to be for October 18, 2011 (NB: The page archive date is November 13, 2012, suggesting inactivity for over a year.)

    I also found a blog entry, dated November 7, 2012, titled "Rumors of my death are greatly exaggerated" There is an email address on this page.

    I can find nothing beyond December 2012.
  96. raphaelmg

    raphaelmg Networkin' Nut Member

    Hi guys!

    I really appreciate your help!
    Koitsu, i will check and install these two packages (in fact i only need the daemon) thank you for your support!
    And Fonos i will also check the Wayback Machine to see if i can access some files. I´m trying to learn how to compile things to tomato, that´s why i was looking for Rodney work but now i will keep "googling" on this. :)

    Thank you again people for your time!

  97. koitsu

    koitsu Network Guru Member

    You might consider looking into Tomatoware,, which @lancethepants maintains. If you want to make packages for Entware-ng, there is documentation on their site for how to contribute.
    raphaelmg and Toastman like this.
  98. raphaelmg

    raphaelmg Networkin' Nut Member

    Sure! so I wil certain check this! Thank you again koitsu!
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice