1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

My utilities web site revived

Discussion in 'Tomato Firmware' started by rhester72, Apr 10, 2010.

  1. Cefur

    Cefur Networkin' Nut Member

    No hurry. Like I said, when/if you have time.

    Yes I understand that static is a no-go.

    Well, the results are unfortunately even worse than mine. ;-)

    In your case I get immediate "Segmentation fault" error. If I try my compiled version, I get "MJPG Streamer Version.: 2.0" followed by "Segmentation fault" (on loading *.so).

    For libjpeg.so ... I installed through ipkg install libjpeg ... so I have the latest installed. I hope this is not a problem.

    When I do "ldd mjpg_streamer" on my version I get the following.
    "ldd mjpg_streamer
    libpthread.so.0 => /opt/lib/libpthread.so.0 (0x2aabf000)
    libdl.so.0 => /opt/lib/libdl.so.0 (0x2aae2000)
    libgcc_s.so.1 => /opt/lib/libgcc_s.so.1 (0x2aaf5000)
    libc.so.0 => /opt/lib/libc.so.0 (0x2ab14000)
    libc.so.0 => /lib/libc.so.0 (0x2abc9000)
    ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x2aaa8000)
    "
    So it seems all links are ok. So my feeling is (but I might be wrong) is that my plugins (*.so files) are wrongly compiled. But I tried yours plugins (with my mjpg_streamer) but the same error happened (segmentation fault on load).

    If I do the same "ldd mjpg_streamer" on your version ... immediate error "Segmentation fault".

    And "ldd xyz.so" on any of the libraries mine or yours it returns "segmentation fault" error.

    Just for info. I loaded the following modules:
    input-core.ko
    videodev.ko
    uvcvideo.ko
    v4l2-common.ko
    v4l2-int-device.ko
    I hope I did not miss anyone important.

    If you have any question for me ... shoot.
     
  2. rhester72

    rhester72 Network Guru Member

    Interesting.

    I suspect this is the infamous MIPS-misalignment-on-4-byte-boundaries, piss-poor coding issue (a la Transmission *sigh*), but I'll need more time to look into it.

    Rodney
     
  3. rhester72

    rhester72 Network Guru Member

    Now I know why.

    This is because my first attempt ended up trying to compile mjpg_streamer as a dynamic library. That probably isn't good. :)

    Fixed that by switching to the 'old-school' compiler model where I let it "pretend" it's not being cross-compiled, using plain old gcc (or so it thinks). See the LOCAL_BIN declaration in tomato-include.sh for details.

    As long as it's in your library path, it should work fine. It's easy enough to test by pulling up help for input_uvc.so. If it's not in the path:

    Code:
    # mjpg_streamer -i "input_uvc.so --help"
    MJPG Streamer Version.: 2.0
    ERROR: could not find input plugin
           Perhaps you want to adjust the search path with:
           # export LD_LIBRARY_PATH=/path/to/plugin/folder
           dlopen: File not found
    
    If it is:

    Code:
    # mjpg_streamer -i "input_uvc.so --help"
    MJPG Streamer Version.: 2.0
     ---------------------------------------------------------------
     Help for input plugin..: UVC webcam grabber
     ---------------------------------------------------------------
     The following parameters can be passed to this plugin:
    
     [-d | --device ].......: video device to open (your camera)
     [-r | --resolution ]...: the resolution of the video device,
                              can be one of the following strings:
                              QSIF QCIF CGA QVGA CIF VGA
                              SVGA XGA SXGA
                              or a custom value like the following
                              example: 640x480
     [-f | --fps ]..........: frames per second
     [-y | --yuv ]..........: enable YUYV format and disable MJPEG mode
     [-q | --quality ]......: JPEG compression quality in percent
                              (activates YUYV format, disables MJPEG)
     [-m | --minimum_size ].: drop frames smaller then this limit, useful
                              if the webcam produces small-sized garbage frames
                              may happen under low light conditions
     [-n | --no_dynctrl ]...: do not initalize dynctrls of Linux-UVC driver
     [-l | --led ]..........: switch the LED "on", "off", let it "blink" or leave
                              it up to the driver using the value "auto"
     ---------------------------------------------------------------
    
    input_init() return value signals to exit
    
    The real problem is that the source assumes certain gcc-isms that are no longer true...such as assume-private-function-declarations-are-static. This hasn't been true in *ages*...but the fix was simple, just make the help() function static in each plugin so as not to conflict with the same in mjpg_streamer.c.

    Ironically, the plugins themselves should have worked - only help crashed.

    I also fixed up carriage returns on some of the logging and turned off debugging on output_autofocus.so (it was the only module where it was hard-coded to be on). Patches can be found in PATCHES/mjpg-streamer-r63/100-static.patch. (Feel free to submit them upstream.)

    That's more of an implementation issue with the busybox ldd. Using mipsel-uclibc-ldd from the SDK produces:

    Code:
    $ mipsel-uclibc-ldd *.so
    input_gspcav1.so:
    checking sub-depends for '/lib/libgcc_s.so.1'
    /lib/libgcc_s.so.1: not an ELF file.
    checking sub-depends for 'not found'
     libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00000000)
     libc.so.0 => not found (0x00000000)
     not a dynamic executable
    input_testpicture.so:
    checking sub-depends for '/lib/libgcc_s.so.1'
    /lib/libgcc_s.so.1: not an ELF file.
    checking sub-depends for 'not found'
     libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00000000)
     libc.so.0 => not found (0x00000000)
     not a dynamic executable
    input_uvc.so:
    checking sub-depends for 'not found'
    checking sub-depends for '/lib/libgcc_s.so.1'
    /lib/libgcc_s.so.1: not an ELF file.
    checking sub-depends for 'not found'
     libjpeg.so.8 => not found (0x00000000)
     libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00000000)
     libc.so.0 => not found (0x00000000)
     not a dynamic executable
    output_autofocus.so:
    checking sub-depends for 'not found'
    checking sub-depends for '/lib/libgcc_s.so.1'
    /lib/libgcc_s.so.1: not an ELF file.
    checking sub-depends for 'not found'
     libm.so.0 => not found (0x00000000)
     libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00000000)
     libc.so.0 => not found (0x00000000)
     not a dynamic executable
    output_file.so:
    checking sub-depends for '/lib/libgcc_s.so.1'
    /lib/libgcc_s.so.1: not an ELF file.
    checking sub-depends for 'not found'
     libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00000000)
     libc.so.0 => not found (0x00000000)
     not a dynamic executable
    output_http.so:
    checking sub-depends for '/lib/libgcc_s.so.1'
    /lib/libgcc_s.so.1: not an ELF file.
    checking sub-depends for 'not found'
     libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00000000)
     libc.so.0 => not found (0x00000000)
     not a dynamic executable
    
    which is correct.

    Unfortunately, I don't have any sort of camera hooked up to my router, so I'm assuming what you're loading is correct. :) Only one way to find out! ;)

    Don't forget to put the plugins in your library search path or set LD_LIBRARY_PATH - you don't need to do both, just one or the other.

    I've been able to successfully with through a browser (on port 8080) with:

    # mjpg_streamer -i input_testpicture.so -o "output_http.so -w /tmp/www"

    where I copied the contents of the mjpg-streamer-r63/www directory into /tmp/www on Tomato. It worked in both static and streamed mode.

    OK...does it work? :)

    Rodney
     
  4. Cefur

    Cefur Networkin' Nut Member

    Hmmm no. :frown:

    But now I am pretty sure the problem is with me (since it works ok for you). Now your mjpg_streamer works as mine. But I still get the same error.

    Also "mipsel-uclibc-ldd" shows ok links (I checked the input_uvc.so, which was giving "headaches") for my plugin.

    Since I am lazy ... I don't do a clear NVRAM on upgrades. So far everything was working ok (I don't us anything special) since ... long time ago. Could this be the reason for my problems?

    This was what I was about to submit, but decided to do one more thing. The one that solves 99.99% of all software problems. DO A RESET!

    Suddenly it was working! :wall::wall::wall::wall::wall::wall::wall::wall:

    Even my version. :flabbergasted::flabbergasted::flabbergasted:

    Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...... :knock::knock::knock:

    Sorry for all the troubles I put you through. :blush:

    Current version (or default settings) is actually working quite poorly. There is a lot of flickering etc., but I have a working starting point. The CPU usage is below 2%, so this is not a problem. Thank you. :rockon:

    I have a couple of questions for you:
    - did you use the latest version (svn) or 0.63? It has more plugins (with improvements).
    - did you compiled using "USE_LIBV4L2"? I believe newer cameras would benefit from it. But it requires lv4l2 library.

    In any case, you are :cool1:!
     
  5. rhester72

    rhester72 Network Guru Member

    I used 0.63, since svn is usually a works-today-broken-tomorrow affair. I can try svn.

    I didn't, because I didn't know what it was. I'll try to find that library.

    Rodney
     
  6. rhester72

    rhester72 Network Guru Member

    Tried - found and fixed the same bugs as in 0.63. Ah, well, at least now I'm getting good at it. *LOL*

    Found, compiled, and integrated into input_uvc.so, but unable to test.

    NOTE: You'll need both libv4l2.so.0 and libv4lconvert.so.0 from PRECOMPILED/lib.

    Rodney
     
  7. Cefur

    Cefur Networkin' Nut Member

    WAW you are fast.

    Thank you.

    Well flickering is still here (so it must be settings wise - I hope).
    The added value of v4l2 (first impression) is in control that you obtain over camera. Like picture manipulation etc. which was not there before.
     
  8. Toastman

    Toastman Super Moderator Staff Member Member

    Nice one, guys. Hope it works out!
     
  9. Cefur

    Cefur Networkin' Nut Member

    Yep it works nicely, now.

    I had to set the resolution to max. :confused:
    So at 25 fps the CPU load is at about 7%. (only http output was tested)

    With nice scripts/tricks provided by Rodney, compilation of new versions will be a piece of cake.

    Now only the sound (which is also on webcam) is missing. If anyone know good software that works with web mic ...
     
  10. rhester72

    rhester72 Network Guru Member

    I'm guessing (purely) that the downscaling was being done in mjpg-streamer and not in the hardware - the hardware was probably still sending full frames and you were resizing in software on the router (which is horrible). May be some way to tune that in the video driver itself (upstream), I dunno.

    Glad to hear it works. I was wondering about the audio thing myself...and how you'd ever integrate the streams.

    Rodney
     
  11. rhester72

    rhester72 Network Guru Member

    Minor issue forwarding the legacy domains via Pound corrected.

    bandwidthd (I first read about it at the TomatoUSB forum, rather similar to darkstat) seemed interesting enough. Ported. Read NOTES!

    transmission 2.10 also makes an appearance - recompiled using a very different strategy than before (significant change to libcurl, different inline handler, IPv6 patch/hack), seems possibly more stable than 1.77->2.04. If it starts crashing regularly, it will be pulled.

    Rodney
     
  12. Cefur

    Cefur Networkin' Nut Member

    Well I dont have any good idea. There is really not a lot of recent work done one "web audio". Some stuff is old/unsupported and dont work for me (like esound).

    I found something called p... something and it supported video+audio, but unfortunately the video did not fully support MJPEG, and therefore used YUYV, which was horrible on CPU load even at small resolution. Otherwise video+audio worked ok. The real problem is that the last beta snapshot was created almost a year ago. So I do not know if it is still properly supported.
     
  13. onehomelist

    onehomelist Addicted to LI Member

    I got bandwidthd successfully installed. How can I access the graphs over http? What port does its webserver listen on?
     
  14. rhester72

    rhester72 Network Guru Member

    It doesn't have a built-in webserver. It outputs HTML that should be served by a web server of your choice (the built-in Tomato httpd works fine in my testing).

    Rodney
     
  15. onehomelist

    onehomelist Addicted to LI Member

    I tried by creating a new directory 'band' in /tmp/var/wwwext/.
    And I did:
    Code:
    mount -o bind /opt/var/local/bandwidthd/htdocs/ /tmp/var/wwwext/band/
    When i tried to access it on the browses with
    Code:
    http://192.168.0.1/wwwext/band
    It gave page not found error.
     
  16. rhester72

    rhester72 Network Guru Member

    Is bandwidthd building index files in that directory? Anything interesting in /var/log/messages?

    Rodney
     
  17. onehomelist

    onehomelist Addicted to LI Member

    Yes there were a few index.html files in the mounted 'band' directory. I even tried making index.html executable. I will soon provide you information about sys logs. Thank You.
     
  18. rhester72

    rhester72 Network Guru Member

  19. ironman07882

    ironman07882 Networkin' Nut Member

    Build Request - IP Tables State

    Hi, Rodney,

    I like your website; it provides valuable utilities for the Tomato router..

    I'm interested in how iptables works and understanding how the linux 2.4 kernel filters packets. I was wondering if you would consider building IP Tables State? It dynamically tracks the ip_conntrack kernel table. It's output presents 'netstat' like information in an analogous way to 'top' presenting cpu load.

    The binary seems fairly simple to build (one C++ source file). There are two library dependencies: ncurses which you have and libnetstat_ipconntrack which I believe you do not currently build.

    In any event, I appreciate your consideration!

    http://www.phildev.net/iptstate/

    Cheers!
     
  20. i1135t

    i1135t Network Guru Member

    Build Request

    Hi Rhester, I love your site since it has many tools that I use personally with OpenVPN and TB's additions. One thing I am lacking is PPTP support, which I know isn't as secure as OpenVPN, but because it is readily available on most Windows boxes, I find it very useful. Is it possible for you to compile a static build for "PopTop" for K2.4. The lastest stable build is 1.3.4 and and appears to work somewhat... I say that because I have been testing the version distributed through IPKG, but cannot get it working with the MPPE encryption and compression that TB implemented into his build. Would appreciate any help you can give..

    http://sourceforge.net/projects/poptop/files/pptpd/pptpd-1.3.4/
     
  21. rhester72

    rhester72 Network Guru Member

    I appreciate your interest in netfilter, but bear in mind that iptstate is dependent on libnetfilter_conntrack, which in turn requires a 2.6 kernel (>= 2.6.18). The K26 builds use kernel 2.6.22, but do not currently have nfnetlink_conntrack enabled.

    The practical upshot? Dynamic and static builds are up, but they won't work until you a) get someone to compile a custom firmware for you enabling nfnetlink_conntrack (teddy_bear would be a useful person to poke about this, hopefully becoming a part of TomatoUSB mainline) and b) you upgrade your device to K26 (I'm _assuming_ that's possible, I don't know which router you're using).

    The tool seems pretty neat...I'm interested in seeing it running too!

    Rodney
     
  22. rhester72

    rhester72 Network Guru Member

    I've compiled, but a static build is not possible, due to the plugin system used by pptpd. However, the only required libraries are the ones included with Tomato, so you shouldn't have any linkage problems.

    The build was done with the K26 SDK for consistency's sake - I tried with both the K24 and K26 SDKs and the resulting linkage dependencies were identical, so it should work on K24 with no issue. If not, let me know and I'll rebuild.

    Rodney
     
  23. rhester72

    rhester72 Network Guru Member

    Fair warning:

    I'm trying to track down a K26 iptables regression that will require testing of a likely significant number of different builds on my router...which also houses my web and FTP sites. You *will* notice disruption from time to time. Sorry about that - all in the name of science! =)

    Rodney
     
  24. rhester72

    rhester72 Network Guru Member

    Hrm.

    Welll, I got a loadable kernel module for K26 (nf_conntrack_netlink.ko) compiled, and the app now loads, but all of the IPs show up as zeroes and the TTLs are unreported. Very odd. Will keep looking at it.

    Rodney
     
  25. i1135t

    i1135t Network Guru Member

    Ok, I have tested this but got this error when trying to connect:
    Code:
    Oct 20 20:55:00 T1 daemon.err pptpd[622]: MGR: Failed to exec /opt/sbin/pptpctrl!
    Looks like it's still pointing to /opt directory.
     
  26. rhester72

    rhester72 Network Guru Member

    re: pptpctrl - sorry about that, I didn't realize that pptpd had external binary dependencies. I've put pptpctrl and bcrelay (if needed) in PRECOMPILED/pptpd.

    As far as pointing to /opt/sbin, well...it has to point _somewhere_ you can write to, and /opt seems to be the established standard across various Linux router distributions. If you don't have any storage mounted on /opt, I'd recommend doing something like the following in Init:

    mkdir -p /opt/sbin
    ln -s /<your_path>/pptpd /opt/sbin
    ln -s /<your_path>/pptpctrl /opt/sbin

    Rodney
     
  27. rhester72

    rhester72 Network Guru Member

    Something's up with that kernel module for sure - but no matter.

    [​IMG]

    No kernel module required, static build available (but it's half a meg!) for those who don't want to fool with ncurses and c++ libraries.

    It's a very, very cool tool. Thanks for bringing it to my attention!

    Rodney
     
  28. i1135t

    i1135t Network Guru Member

    Thank you, finally got PPTP server running with encryption. Phew... that took me a long time, but with your help and tons of tons of googling, got it working! Now all I need is to setup route statements for the traffic to work correctly since I cannot ping the router or the internet from the outside nor assigned a default gateway on the VPN connection.. hmmm.

    On goes the battle... =)
     
  29. rhester72

    rhester72 Network Guru Member

    A few minor changes. ;)

    All apps have been recompiled for consistent directory standards - please see the updated 00README for details.

    ncurses has been updated to support terminfo and termcaps in addition to a new fallback of xterm-color. This should ease the burden for anyone not using a sane (read: xterm) ssh client. ;)

    A recent additions is psmisc (primarily for pstree), very useful for easily seeing parent/child and pthread relationships.

    One was just for fun (frotz):

    [​IMG]

    And one was both fun and potentially useful (upx):

    [​IMG]

    (You read that right - 70% disk savings!)

    A lot of people don't realize that UPX, even running on Windows or Linux i386/x64, can compress mipsel/MIPS v1 binaries. Now MIPS can also compress natively. For those running VERY low on disk space for binaries (jffs, for instance), this could be genuinely useful, with no additional memory overhead. I've also provided a upx-compressed-with-upx binary called upx-tiny for those truly out of space. <G>

    Please note that ldd will *NOT* work on packed binaries, so please ensure you have all required libraries for an executable before you compress it! If not, you can always uncompress it via upx with the -d switch.

    Enjoy!

    Rodney
     
  30. teddy_bear

    teddy_bear Network Guru Member

    Poptop is an old, outdated, and no longer supported application.

    There's however a newer alternative/replacement - accel-pptpd - this project is under active development, and includes PPTP, L2TP and PPPoE servers (TomatoUSB currently uses the PPTP client from this project). It does list kernel 2.6.25 or higher as a requirement - but most probably this can be worked out either with some kernel patches, or accel-pptpd patches (well, first we need to figure out what is the reason for this requirement of course).

    I would like to eventually get some PPTP server built-in, but reluctant to work on Poptop integration - this one looks a lot more promising...

    Rodney, are you up to challenge to compile and make it to work under Tomato as the first step :)?
     
  31. rhester72

    rhester72 Network Guru Member

    This is why 2.6.25 (taken from http://kernelnewbies.org/Linux_2_6_25):

    ray123, this looks like your territory. Want to take a swing at the kernel side? =)

    The compile will go nowhere without it, of course.

    Rodney
     
  32. teddy_bear

    teddy_bear Network Guru Member

    Should not be too hard backporting it to 2.6.22 - lots of following commits after the first one - but neither is too complex. Is this the only requirement?
     
  33. rhester72

    rhester72 Network Guru Member

    That's all that was obvious based on the CMake failures, and they _appeared_ to be exhaustive.

    Rodney
     
  34. teddy_bear

    teddy_bear Network Guru Member

    I did the timerfd backport. Will test it a bit, and if everything looks ok, I'll check it in to the git repo.
     
  35. onehomelist

    onehomelist Addicted to LI Member

  36. onehomelist

    onehomelist Addicted to LI Member

    Your http page gives 'internal server error".
     
  37. rhester72

    rhester72 Network Guru Member

    It looks like it's up fine from here.

    I did reboot the router this morning, you may have hit it while it was still coming up.

    Rodney
     
  38. teddy_bear

    teddy_bear Network Guru Member

    Rodney - I checked in timerfd syscall backport (briefly tested that it didn't break anything important) - hopefully that will help you to continue with accel-pptpd...
     
  39. rhester72

    rhester72 Network Guru Member

    Thanks! I'll take a look tomorrow morning - a bit under the weather here today. Seems like this is going to be the winter of illness... :/

    Rodney
     
  40. rhester72

    rhester72 Network Guru Member

    This is not enjoyable. :)

    The CMake test for timerfd_create() is trying to compile directly against the kernel's system.map - i.e. it expects it to be a built-in function. It's using the correct SDK tree, but as a result, can't find an actual binary to include at link time to resolve the external linkage...and I can't see a way to "help" it find it, particularly since there's no such linkage in the SDK itself.

    I'm not as familiar with how to hack CMake as I am regular makefiles - any ideas on this one? (Even if I could get past the CMake check, which is rather trivial, the actual compile-time linkage will fail.)

    Rodney
     
  41. Int15

    Int15 Network Guru Member

    I'm getting the same message.
     
  42. rhester72

    rhester72 Network Guru Member

    Weird. Bounced hiawatha, seems OK now.

    Rodney
     
  43. rhester72

    rhester72 Network Guru Member

    Ouch. The site is definitely down now - just loaded a bad kernel module. :/

    Back in about 12 hours. *sigh*

    Rodney
     
  44. rhester72

    rhester72 Network Guru Member

    Fascinating. The router's back up, having rebooted itself due to watchdog (nice!).

    Will leave it alone for now and test later tonight. =)

    Rodney
     
  45. teddy_bear

    teddy_bear Network Guru Member

    Who said it's gonna be easy :wink:?
    I'm not familiar with CMake either - but I suppose if you just remove the timerfd check from one of the CMakeLists.txt files, you should at least get past the CMake stage...

    However, you're right that it will fail at the compile time anyway - the toolchain headers must be updated to include the new backported syscalls. Rebuilding toolchain should fix it since it auto-generates some headers based on the kernel sources. Or maybe you can just manually change the "tomato/tools/brcm/K26/hndtools-mipsel-uclibc-4.2.4/include/bits/sysnum.h" file to include the missing timerfd syscalls definitions. That might even fix CMake too...
     
  46. i1135t

    i1135t Network Guru Member

    Cool, I'd like to see accel-pptpd working as well. Poptop works great, but speed is slow compared to OpenVPN.

    Side note, Rodney, any chance of compiling Moblock for K2.4? I'd like to try it out since it looks very promising.

    Keep up the good work guys...
     
  47. rhester72

    rhester72 Network Guru Member

    Workin' on that. :)

    The IPQ version should work on K24 but has not been tested - give it a shot!

    Rodney
     
  48. rhester72

    rhester72 Network Guru Member

    I don't think the include is the problem, ironically.

    This is what I'm getting out of CMake:


    Linking C executable cmTryCompileExec
    /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTryCompileExec.dir/link.txt --verbose=1
    /home/rhester/tomato/tools/brcm/K26/hndtools-mipsel-uclibc-4.2.4/bin/mipsel-uclibc-gcc -Os -s -ffunction-sections -fdata-sections -I/home/rhester/tomato_local/include -L/home/rhester/tomato_local/lib -Wall -fvisibility=hidden -D_GNU_SOURCE -DGCC_SPINLOCK -DMEMDEBUG -DCHECK_FUNCTION_EXISTS=timerfd_create -s -Wl,--gc-sections,-rpath=/opt/lib:/opt/usr/lib,-rpath-link=/home/rhester/tomato_local/lib -L/home/rhester/tomato_local/lib CMakeFiles/cmTryCompileExec.dir/CheckFunctionExists.c.o -o cmTryCompileExec -rdynamic
    CMakeFiles/cmTryCompileExec.dir/CheckFunctionExists.c.o: In function `main':
    CheckFunctionExists.c:(.text.main+0x20): undefined reference to `timerfd_create'
    collect2: ld returned 1 exit status

    I'm not entirely sure how gcc handles linkage against kernel functions - there is no .so that I know of. I assumed it used System.map but that doesn't appear to be correct either.

    Any ideas?

    Rodney
     
  49. rhester72

    rhester72 Network Guru Member

    For those who can't reach the site via web, bear with me - it seems there is a negative interaction between Pound and hiawatha that's causing the problem. Working on it. :)

    UPDATE: I think the alive setting I was using with Pound was actually eating up all the connections to Hiawatha. I've returned it to the default, will see.

    UPDATE 2: Ugh. It would seem that Pound's notion of "consistent sessions" is causing hiawatha to believe *every* request is a new (and persistent!) client. Anyone have experience with this? I've disabled Pound for now as a result.

    UPDATE 3: Pound is back, for now. The workaround is not to use the HideProxy directive in Hiawatha. The reason appears to be a bug in Hiawatha. :/

    Rodney
     
  50. teddy_bear

    teddy_bear Network Guru Member

    So does the CMake error out and exit right after the failed check for built-in timerfd_create? Looking at the accel-pptpd/triton/CMakeLists.txt, it should not - instead it should then move on to syscall() check:
    Code:
    CHECK_FUNCTION_EXISTS(timerfd_create HAVE_TIMERFD)
    
    IF (HAVE_TIMERFD)
    	ADD_DEFINITIONS(-DHAVE_TIMERFD)
    ELSE (HAVE_TIMERFD)
    	INCLUDE (CheckCSourceCompiles)
    	CHECK_C_SOURCE_COMPILES("
    	#include <sys/syscall.h>
    	int main()
    	{
    		syscall(SYS_timerfd_create);
    	}" HAVE_SYSCALL)
    	IF (NOT HAVE_SYSCALL)
    		MESSAGE(FATAL_ERROR "Your system is too old and is not supported by accel-pptp, sorry...")
    	ENDIF (NOT HAVE_SYSCALL)
    	SET(sources_c ${sources_c} timerfd.c)
    ENDIF (HAVE_TIMERFD)
    
    The first check - CHECK_FUNCTION_EXISTS(timerfd_create HAVE_TIMERFD) - should fail, and that's probably where your CMake output comes from... However, the following check should be successful as long as you have the correct header file (bits/sysnum.h) in the toolchain "include" directory.
     
  51. i1135t

    i1135t Network Guru Member

    Actually, I tried that version, and looking at the sh file, it looks like I must change the path of ip_queue.ko to where it resides, which isn't available on K2.4, I believe. Same with the NFQ script. I tried loading the K2.6 ip_queue.ko module as ip_queue.o, which is what it's asking for, but it's not taking it. :( Is this module available for K2.4?
     
  52. rhester72

    rhester72 Network Guru Member

    Ah, I forgot it requires the ipq module.

    I don't have one precompiled for K24 as for whatever reason 2.4 stopped compiling for me many moons ago.

    If anyone wants to volunteer one up, I'll gladly put it on my site!

    Rodney
     
  53. rhester72

    rhester72 Network Guru Member

    You're right - I didn't realize the earlier fail was a 'safe' fail.

    I've looked at sysnum.h, and I follow the format, but I can't figure out what needs to be added here. (Not that big of a kernel hacker, unfortunately ;)

    I noticed you mentioned rebuilding the toolchain before - is there documentation on how to do that? Sounds like it might be a bit more complete/sane than me hacking a single kernel header.

    Rodney
     
  54. rhester72

    rhester72 Network Guru Member

    I guess this is where some of my confusion comes from.

    This:

    http://repo.or.cz/w/tomato.git/blob...-rt/linux/linux-2.6/include/asm-mips/unistd.h

    suggests that the highest syscall in sysnum.h should be 319. However, sysnum.h already has definitions up to 323:

    Code:
    #undef __NR_eventfd
    #define __NR_eventfd (4000 + 319)
    #define SYS_eventfd __NR_eventfd
    #undef __NR_track_flag
    #define __NR_track_flag (4000 +320)
    #define SYS_track_flag __NR_track_flag
    #undef __NR_set_mbss_nolan
    #define __NR_set_mbss_nolan (4000 + 321)
    #define SYS_set_mbss_nolan __NR_set_mbss_nolan
    #undef __NR_set_mbss_nowan
    #define __NR_set_mbss_nowan (4000 + 322)
    #define SYS_set_mbss_nowan __NR_set_mbss_nowan
    #undef __NR_set_mbss_prio
    #define __NR_set_mbss_prio (4000 + 323)
    #define SYS_set_mbss_prio __NR_set_mbss_prio
    #undef __NR_Linux_syscalls
    #define __NR_Linux_syscalls 323
    #define SYS_Linux_syscalls __NR_Linux_syscall
    
    Help? =)

    Rodney
     
  55. teddy_bear

    teddy_bear Network Guru Member

    Don't worry about that - the toolchain has been rebuilt a while ago, it has old syscall definitions that have been removed from kernel later.
    But as you said - the cleaner way would be to just rebuild the toolchain:
    Supposedly you can just run the "build26.sh" script in the tomato/toolchain directory, and it will do the work...
     
  56. rhester72

    rhester72 Network Guru Member

    Web server down for a bit. Working with the Hiawatha author on a fix.

    Rodney
     
  57. rhester72

    rhester72 Network Guru Member

    Got a _lot_ further now. Still working (on new issues).

    Rodney
     
  58. rhester72

    rhester72 Network Guru Member

    Hrm.

    [ 58%] Building C object accel-pptpd/CMakeFiles/accel-pptpd.dir/log.c.o
    /home/rhester/tomato_local/accel-pptp-1.1.2/accel-pptpd/log.c:44: error: thread-local storage not supported for this target

    Is this not available in MIPS gcc? The code in question is:

    Code:
    static __thread struct ppp_t *cur_ppp;
    static __thread struct _log_msg_t *cur_msg;
    static __thread char *stat_buf;
    
    Rodney
     
  59. rhester72

    rhester72 Network Guru Member

    I also tried forcing -pthread as input to gcc for this object, no change...saw it suggested for a similar problem elsewhere (didn't help there either *laughs*).

    Rodney
     
  60. teddy_bear

    teddy_bear Network Guru Member

    Argh... Yep, the gcc is compiled in the toolchain without TLS support... To include TLS, uClibc needs updating to 0.9.32.

    Do you think it would be possible to get rid of TLS requirement by patching log.c?
     
  61. rhester72

    rhester72 Network Guru Member

    re: my web site stability: I was able to work with the Hiawatha author to identify and correct a long-standing bug when sitting behind a reverse proxy (Pound, in my case), the fix for which will appear in Hiawatha 7.4 (and is already applied on my router)...so the "internal server error" messages should be gone for good. Thanks to those who let me know - it was quite challenging to pin down!

    re: my work on accel-pptpd, it's going slowly/not at all, since I've been dealing with an illness severe enough that I've only been awake 5 of the past 27 hours. I'm slowly recovering and will be working on it full-force ASAP, but I strongly suspect we're going to need TLS to make it work properly (since the thread-safing on logging is very much required without a serious rewrite of the code).

    Rodney
     
  62. Toastman

    Toastman Super Moderator Staff Member Member

    Rodney, sorry to hear you've been ill, hope you're OK soon.
     
  63. rhester72

    rhester72 Network Guru Member

    Toastman,

    Thanks - just a rote case of the flu, I think. Just haven't had one put me on my butt like this in years. :)

    Rodney
     
  64. rhester72

    rhester72 Network Guru Member

    re: accel-pptp...

    Could they have made this any more difficult?

    Thread-safing aside (my hack leaves us pretty screwed there, but at least I can compile around it)...now we're missing a bunch of gcc atomics:

    Linking C executable accel-pptpd
    CMakeFiles/accel-pptpd.dir/log.c.o: In function `_log_free_msg':
    log.c:(.text._log_free_msg+0x20): undefined reference to `__sync_sub_and_fetch_1'
    CMakeFiles/accel-pptpd.dir/log.c.o: In function `do_log':
    log.c:(.text.do_log+0x330): undefined reference to `__sync_add_and_fetch_1'
    CMakeFiles/accel-pptpd.dir/memdebug.c.o: In function `md_free':
    memdebug.c:(.text.md_free+0x10c): undefined reference to `__sync_lock_test_and_set_4'
    CMakeFiles/accel-pptpd.dir/memdebug.c.o: In function `_md_malloc':
    memdebug.c:(.text._md_malloc+0xd0): undefined reference to `__sync_lock_test_and_set_4'
    CMakeFiles/accel-pptpd.dir/memdebug.c.o: In function `siginfo':
    memdebug.c:(.text.siginfo+0x34): undefined reference to `__sync_lock_test_and_set_4'
    CMakeFiles/accel-pptpd.dir/memdebug.c.o: In function `siginfo2':
    memdebug.c:(.text.siginfo2+0x38): undefined reference to `__sync_lock_test_and_set_4'
    triton/libtriton.so: undefined reference to `swapcontext'
    triton/libtriton.so: undefined reference to `getcontext'
    triton/libtriton.so: undefined reference to `makecontext'
    triton/libtriton.so: undefined reference to `__sync_fetch_and_sub_4'
    triton/libtriton.so: undefined reference to `__sync_fetch_and_add_4'

    *rips hair out*

    Remind me what's so wrong with the older daemon again? =)

    Rodney
     
  65. Amuro

    Amuro Addicted to LI Member

    Still trying to run mediainfo

    put both libs to /opt/usr/lib and the mediainfo in /opt/bin

    root@Yuki:/opt/bin# mediainfo
    Segmentation fault


    both libs and mediainfo have been a+x. Not sure what to do.
     
  66. rhester72

    rhester72 Network Guru Member

    Which Tomato release are you running?

    Rodney
     
  67. Amuro

    Amuro Addicted to LI Member

    Tomato v1.28.9050 MIPSR2-beta20 K26 USB vpn3.6
     
  68. teddy_bear

    teddy_bear Network Guru Member

    Argh :mad:. Oh well... CMake is bad enough already. Then TLS (can be solved by upgrading uClibc though). Now this - not sure yet how to deal with it...
    The old daemon... old, compromised, unsupported - doesn't sound that bad anymore ;).

    Hope you get well soon - flu can be nasty :(...
     
  69. rhester72

    rhester72 Network Guru Member

    How are you getting the binaries onto the router? You're running effectively the very same release I am.

    Rodney
     
  70. Amuro

    Amuro Addicted to LI Member

    Thank you for helping me with this.

    Initially, downloaded via Firefox mac and moved to smb share via Finder. Then move and chmod via ssh to /opt - Got he previous error.

    Deleted them and curl'd all 3 files from your ftp via ssh, moved them to /opt.... and

    Code:
    root@Yuki:/opt/bin# mediainfo
    mediainfo: can't load library 'libintl.so.0'
    Actually, there is a "libintl.so.0 " under /opt/lib, from optware I guess,

    Code:
    -rw-r--r--    1 root     root        3.0K Aug  5 22:37 /opt/lib/libintl-0.9.28.so
    lrwxrwxrwx    1 root     root          12 Sep 30 19:40 /opt/lib/libintl.so -> libintl.so.0
    lrwxrwxrwx    1 root     root          17 Sep 30 19:40 /opt/lib/libintl.so.0 -> libintl-0.9.28.so
    making "libintl-0.9.28.so" executable. No change. Coping it to /opt/usr/lib and renaming it.

    Same error. FS is ext2
     
  71. rhester72

    rhester72 Network Guru Member

    What is the output of "ldd /opt/bin/mediainfo"?

    Rodney
     
  72. ~nephelim~

    ~nephelim~ LI Guru Member

    I recently came across Mawk

    There are mipsel binaries for debian but they won't run on tomato.

    An old and buggy version was compiled for openwrt (I cannot find the related sources) and it works on tomato.

    Though I lack the necessary knowhow I managed to setup a toolchain to the point and I was able to compile a working binary ( mstombs's pixelserv)

    I had no luck with mawk. I tampered the makefile enough to have mawk provide a proper error under tomato ( /tmp/mawk: can't resolve symbol 'setlocale' )

    But I'm basically stuck. :cry:

    Anybody willing to give this a look anytime in the future? :sad:


    EDIT: It looks like commenting "#define LOCALE 1" in config.h solved the setlocale issue.

    it should be also possible to add:

    Code:
    cf_cv_locale=no

    before the ./configure line:
    Code:
    echo "$as_me:5354: result: $cf_cv_locale" >&5
    I was not able to call ./configure with an option that set "cf_cv_locale=no"

    if possible please suggest better arguments for CPPFLAGS CFLAGS and LDFLAGS in order to reduce mawk size.

    After invoking ./configure --build=mipsel-uclibc-linux I edited the makefile like this.

    Makefile:
    Code:
    o               = .o
    
    prefix          = /usr/local
    exec_prefix     = ${prefix}
    
    srcdir		= .
    
    manext		= 1
    bindir		= ${exec_prefix}/bin
    mandir		= ${prefix}/man/man$(manext)
    
    CC              = mipsel-uclibc-gcc
    CPP             = mipsel-linux-cpp
    
    CPPFLAGS	= -I. -I$(srcdir) -DHAVE_CONFIG_H  -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 
    EXTRA_CFLAGS	= 
    CFLAGS          = -mips32 -g2 -Os -s -ffunction-sections -fdata-sections -DLOCAL_REGEXP $(EXTRA_CFLAGS)
    LDFLAGS         = -Wl,-Os,-s,-x,--gc-sections,--strip-discarded
    LIBS            = -lm 
    
    YACC            = bison -y
    
    CTAGS		= 
    ETAGS		= 
    
    LINT		= 
    LINTOPTS	= 
    
    INSTALL         = /usr/bin/install -c
    INSTALL_PROGRAM = ${INSTALL}
    INSTALL_DATA    = ${INSTALL} -m 644
    transform	= s,x,x,
    
    # where to put mawk
    BINDIR          = $(DESTDIR)$(bindir)
    
    # where to put the man pages
    MANDIR          = $(DESTDIR)$(mandir)
    
    #######################################
    
    actual_PROG  = `echo mawk|    sed '$(transform)'`
    binary_PROG  = $(actual_PROG)$x
    
    PROG  = mawk$x
    
    OBJS = parse$o scan$o memory$o main$o hash$o execute$o code$o \
      da$o error$o init$o bi_vars$o cast$o print$o bi_funct$o \
      kw$o jmp$o array$o field$o split$o re_cmpl$o regexp$o zmalloc$o \
      fin$o files$o scancode$o matherr$o fcall$o version$o \
      missing$o
    
    .SUFFIXES: .c .i $o
    
    .c.i :
    	
    	$(CPP) -C $(CPPFLAGS) $< >$@
    .c$o :
    	
    	$(CC) -c $(CPPFLAGS) $(CFLAGS) $<
    
    all :	$(PROG)
    
    check :  mawk_test fpe_test
    
    $(PROG) : $(OBJS)
    	$(CC) $(LDFLAGS) -o $@ $(OBJS) $(LIBS)
    
    CHECK	= MAWK=`pwd`/$(PROG); export MAWK; cd $(srcdir)/test; echo "** $@"
    
    mawk_test :  $(PROG)  # test that we have a sane mawk
    	@sh -c '$(CHECK); ./mawktest'
    
    fpe_test :  $(PROG) # test FPEs are handled OK
    	@sh -c '$(CHECK); ./fpe_test'
    
    parse.c  : parse.y
    	@echo  expect 4 shift/reduce conflicts
    	$(YACC) -d parse.y
    	mv y.tab.c parse.c
    	-if cmp -s y.tab.h parse.h ;\
    	   then rm y.tab.h ;\
    	   else mv y.tab.h parse.h ; fi
    
    NOIDENT = sed -e 's/[$$]MawkId\([^$$]*\)[$$]/@MawkId\1@/'
    
    array.c : array.w
    	-@rm -f $@
    	notangle -R'"array.c"' array.w | $(NOIDENT) | cpif array.c
    
    array.h : array.w
    	-@rm -f $@
    	notangle -R'"array.h"' array.w | $(NOIDENT) | cpif array.h
    
    scancode.c :  $(srcdir)/makescan.c  scancode.h nstd.h config.h
    	$(CC) $(CFLAGS) $(CPPFLAGS) -o makescan.exe $(srcdir)/makescan.c
    	rm -f scancode.c
    	./makescan.exe > scancode.c
    	rm makescan.exe
    
    MAWK_BIN = $(BINDIR)/$(binary_PROG)
    MAWK_MAN = $(MANDIR)/$(actual_PROG).$(manext)
    
    install :  $(BINDIR) $(MANDIR) $(PROG)
    	$(INSTALL_PROGRAM) $(PROG) $(MAWK_BIN)
    	$(INSTALL_DATA) man/mawk.1 $(MAWK_MAN)
    
    uninstall :
    	rm -f $(MAWK_BIN)
    	rm -f $(MAWK_MAN)
    
    clean :
    	rm -f y.tab.c y.tab.h makescan.exe
    	rm -f *$o test/$(PROG) *core* test/*core* $(PROG)
    
    distclean :  clean
    	rm -f config.h Makefile \
    	    config.status config.log config.cache
    	rm -f scancode.c
    	rm -f defines.out maxint.out fpe_check$x tags makedeps.awk
    
    maintainer-clean :  distclean
    	rm -f parse.c parse.h
    	rm -f array.c array.h
    	rm -f configure config.guess config.sub install-sh
    	rm -fr autom4te.cache
    
    lint :
    	$(LINT) $(LINTOPTS) $(CPPFLAGS) $(OBJS:$o=.c)
    
    tags :
    	$(CTAGS) *.[ch] */*.[ch]
    
    #TAGS :
    #	$(ETAGS) *.[ch] */*.[ch]
    
    $(BINDIR) \
    $(MANDIR) :
    	mkdir -p "$@"
    
    # output from makedeps.sh
    array.o : array.h bi_vars.h config.h field.h mawk.h memory.h nstd.h sizes.h symtype.h types.h zmalloc.h
    bi_funct.o : array.h bi_funct.h bi_vars.h config.h field.h files.h fin.h init.h mawk.h memory.h nstd.h regexp.h repl.h sizes.h symtype.h types.h zmalloc.h
    bi_vars.o : array.h bi_vars.h config.h field.h init.h mawk.h memory.h nstd.h sizes.h symtype.h types.h zmalloc.h
    cast.o : array.h config.h field.h mawk.h memory.h nstd.h parse.h repl.h scan.h scancode.h sizes.h symtype.h types.h zmalloc.h
    code.o : array.h code.h config.h field.h init.h jmp.h mawk.h memory.h nstd.h sizes.h symtype.h types.h zmalloc.h
    da.o : array.h bi_funct.h code.h config.h field.h mawk.h memory.h nstd.h repl.h sizes.h symtype.h types.h zmalloc.h
    error.o : array.h bi_vars.h config.h mawk.h nstd.h parse.h scan.h scancode.h sizes.h symtype.h types.h
    execute.o : array.h bi_funct.h bi_vars.h code.h config.h field.h fin.h mawk.h memory.h nstd.h regexp.h repl.h sizes.h symtype.h types.h zmalloc.h
    fcall.o : array.h code.h config.h mawk.h memory.h nstd.h sizes.h symtype.h types.h zmalloc.h
    field.o : array.h bi_vars.h config.h field.h init.h mawk.h memory.h nstd.h parse.h regexp.h repl.h scan.h scancode.h sizes.h symtype.h types.h zmalloc.h
    files.o : array.h config.h files.h fin.h init.h mawk.h memory.h nstd.h sizes.h symtype.h types.h zmalloc.h
    fin.o : array.h bi_vars.h config.h field.h fin.h mawk.h memory.h nstd.h parse.h scan.h scancode.h sizes.h symtype.h types.h zmalloc.h
    hash.o : array.h config.h mawk.h memory.h nstd.h sizes.h symtype.h types.h zmalloc.h
    init.o : array.h bi_vars.h code.h config.h field.h init.h mawk.h memory.h nstd.h sizes.h symtype.h types.h zmalloc.h
    jmp.o : array.h code.h config.h init.h jmp.h mawk.h memory.h nstd.h sizes.h symtype.h types.h zmalloc.h
    kw.o : array.h config.h init.h mawk.h nstd.h parse.h sizes.h symtype.h types.h
    main.o : array.h code.h config.h files.h init.h mawk.h memory.h nstd.h sizes.h symtype.h types.h zmalloc.h
    makescan.o : config.h nstd.h scancode.h
    matherr.o : array.h config.h init.h mawk.h nstd.h sizes.h symtype.h types.h
    memory.o : config.h mawk.h memory.h nstd.h sizes.h types.h zmalloc.h
    missing.o : config.h nstd.h scancode.h
    parse.o : array.h bi_funct.h bi_vars.h code.h config.h field.h files.h jmp.h mawk.h memory.h nstd.h sizes.h symtype.h types.h zmalloc.h
    print.o : array.h bi_funct.h bi_vars.h config.h field.h files.h mawk.h memory.h nstd.h parse.h scan.h scancode.h sizes.h symtype.h types.h zmalloc.h
    re_cmpl.o : array.h config.h mawk.h memory.h nstd.h parse.h regexp.h repl.h scan.h scancode.h sizes.h symtype.h types.h zmalloc.h
    scan.o : array.h code.h config.h field.h files.h fin.h init.h mawk.h memory.h nstd.h parse.h repl.h scan.h scancode.h sizes.h symtype.h types.h zmalloc.h
    split.o : array.h bi_funct.h bi_vars.h config.h field.h mawk.h memory.h nstd.h parse.h regexp.h repl.h scan.h scancode.h sizes.h symtype.h types.h zmalloc.h
    version.o : array.h config.h init.h mawk.h nstd.h patchlev.h sizes.h symtype.h types.h
    zmalloc.o : config.h mawk.h nstd.h sizes.h types.h zmalloc.h
    regexp.o : rexpdb.c rexp4.c rexp2.c regexp_system.c sizes.h mawk.h rexp0.c rexp1.c config.h rexp.h regexp.h nstd.h rexp3.c rexp.c field.h

    config.h

    Code:
    /* config.h.  Generated automatically by configure.  */
    /*
     * $MawkId: config_h.in,v 1.17 2010/05/06 23:36:28 tom Exp $
     * vile:cmode
     * template for config.h
     */
    
    /* #undef DECL_ENVIRON */
    /* #undef FPE_TRAPS_ON */
    /* #undef GCC_NORETURN */
    /* #undef GCC_PRINTF */
    /* #undef GCC_PRINTFLIKE */
    /* #undef GCC_SCANF */
    /* #undef GCC_SCANFLIKE */
    /* #undef GCC_UNUSED */
    #define HAVE_FORK 1
    #define HAVE_ISNAN 1
    #define HAVE_PIPE 1
    #define HAVE_REAL_PIPES 1
    /* #undef HAVE_REGEXPR_H_FUNCS */
    /* #undef HAVE_REGEXP_H_FUNCS */
    /* #undef HAVE_REGEX_H_FUNCS */
    #define HAVE_SIGACTION 1
    #define HAVE_SIGACTION_SA_SIGACTION 1
    /* #undef HAVE_SIGINFO_H */
    /* #undef HAVE_STRTOD_OVF_BUG */
    #define HAVE_SYS_WAIT_H 1
    #define HAVE_UNISTD_H 1
    #define HAVE_WAIT 1
    /* #undef LOCALE */
    /* #undef MAX__INT */
    /* #undef MAX__LONG */
    /* #undef MAX__UINT */
    /* #undef NOINFO_SIGFPE */
    /* #undef NO_ERRNO_H */
    /* #undef NO_FCNTL_H */
    /* #undef NO_FMOD */
    /* #undef NO_LEAKS */
    /* #undef NO_MATHERR */
    /* #undef NO_MATH_H */
    /* #undef NO_MEMCPY */
    /* #undef NO_PROTOS */
    /* #undef NO_STDARG_H */
    /* #undef NO_STDLIB_H */
    /* #undef NO_STRCHR */
    /* #undef NO_STRERROR */
    /* #undef NO_STRING_H */
    /* #undef NO_STRTOD */
    /* #undef NO_TIME_H */
    /* #undef NO_VFPRINTF */
    /* #undef NO_VOID_STAR */
    #define SIZE_T_STDDEF_H 1
    /* #undef SIZE_T_TYPES_H */
    #define SYSTEM_NAME "linux-gnu"
    /* #undef TURN_OFF_FPE_TRAPS */
    /* #undef TURN_ON_FPE_TRAPS */
    /* #undef USE_IEEEFP_H */
    /* #undef const */
    
    #ifndef GCC_NORETURN
    #define GCC_NORETURN /* nothing */
    #endif
    
    #ifndef GCC_UNUSED
    #define GCC_UNUSED /* nothing */
    #endif
    
    The binary works and its 137 KB in size. it appears to be slightly slower than the outdated version.

    Edit2: UPX(-9) compressed binary is 54KB

    Edit 3: using jenkins lookup3 in mawk hash.c is faster (just like the old openwrt version, though)
    Code:
    // Adapted from Tomato jhash.h
    
    #define __rot(x,k) (((x)<<(k)) | ((x)>>(32-(k))))
    
    /* __jhash_mix - mix 3 32-bit values reversibly. */
    #define __jhash_mix(a,b,c) \
    { \
      a -= c;  a ^= __rot(c, 4);  c += b; \
      b -= a;  b ^= __rot(a, 6);  a += c; \
      c -= b;  c ^= __rot(b, 8);  b += a; \
      a -= c;  a ^= __rot(c,16);  c += b; \
      b -= a;  b ^= __rot(a,19);  a += c; \
      c -= b;  c ^= __rot(b, 4);  b += a; \
    }
    
    /* __jhash_final - final mixing of 3 32-bit values (a,b,c) into c */
    #define __jhash_final(a,b,c) \
    { \
      c ^= b; c -= __rot(b,14); \
      a ^= c; a -= __rot(c,11); \
      b ^= a; b -= __rot(a,25); \
      c ^= b; c -= __rot(b,16); \
      a ^= c; a -= __rot(c,4);  \
      b ^= a; b -= __rot(a,14); \
      c ^= b; c -= __rot(b,24); \
    }
    
    /* An arbitrary initial parameter */
    #define JHASH_INIT_PARAM 0xdeadbeefUL
    
    unsigned 
    hash(const char *k)
    {
        size_t length = 0UL;
        const char *z = k;
        while (*z++) {
    		length++;
        }
    	unsigned a,b,c;
    
    	/* Set up the internal state */
    	a = b = c = JHASH_INIT_PARAM + length;
    	 
    	/* all but the last block: affect some 32 bits of (a,b,c) */
    	while (length > 12) {
    	a += (k[0] + ((unsigned)k[1]<<8) + ((unsigned)k[2]<<16) + ((unsigned)k[3]<<24));
    	b += (k[4] + ((unsigned)k[5]<<8) + ((unsigned)k[6]<<16) + ((unsigned)k[7]<<24));
    	c += (k[8] + ((unsigned)k[9]<<8) + ((unsigned)k[10]<<16) + ((unsigned)k[11]<<24));
    	__jhash_mix(a, b, c);
    	length -= 12;
    	k += 12;
    	}
    	/* last block: affect all 32 bits of (c) */
    	/* all the case statements fall through */
    	switch (length) {
    	case 12: c += (unsigned)k[11]<<24;
    	case 11: c += (unsigned)k[10]<<16;
    	case 10: c += (unsigned)k[9]<<8;
    	case 9 : c += k[8];
    	case 8 : b += (unsigned)k[7]<<24;
    	case 7 : b += (unsigned)k[6]<<16;
    	case 6 : b += (unsigned)k[5]<<8;
    	case 5 : b += k[4];
    	case 4 : a += (unsigned)k[3]<<24;
    	case 3 : a += (unsigned)k[2]<<16;
    	case 2 : a += (unsigned)k[1]<<8;
    	case 1 : a += k[0];
    	__jhash_final(a, b, c);
    	case 0 :
    	break;
    	}
    	return c;
        
    }
    
    unsigned 
    hash2(const char *k, size_t length)
    {
    	unsigned a,b,c;
    
    	/* Set up the internal state */
    	a = b = c = JHASH_INIT_PARAM + length;
    	 
    	/* all but the last block: affect some 32 bits of (a,b,c) */
    	while (length > 12) {
    	a += (k[0] + ((unsigned)k[1]<<8) + ((unsigned)k[2]<<16) + ((unsigned)k[3]<<24));
    	b += (k[4] + ((unsigned)k[5]<<8) + ((unsigned)k[6]<<16) + ((unsigned)k[7]<<24));
    	c += (k[8] + ((unsigned)k[9]<<8) + ((unsigned)k[10]<<16) + ((unsigned)k[11]<<24));
    	__jhash_mix(a, b, c);
    	length -= 12;
    	k += 12;
    	}
    	/* last block: affect all 32 bits of (c) */
    	/* all the case statements fall through */
    	switch (length) {
    	case 12: c += (unsigned)k[11]<<24;
    	case 11: c += (unsigned)k[10]<<16;
    	case 10: c += (unsigned)k[9]<<8;
    	case 9 : c += k[8];
    	case 8 : b += (unsigned)k[7]<<24;
    	case 7 : b += (unsigned)k[6]<<16;
    	case 6 : b += (unsigned)k[5]<<8;
    	case 5 : b += k[4];
    	case 4 : a += (unsigned)k[3]<<24;
    	case 3 : a += (unsigned)k[2]<<16;
    	case 2 : a += (unsigned)k[1]<<8;
    	case 1 : a += k[0];
    	__jhash_final(a, b, c);
    	case 0 :
    	break;
    	}
    	return c;
    }
     
  73. Amuro

    Amuro Addicted to LI Member

    Code:
    #ldd /opt/bin/mediainfo 
    	libz.so.1 => /opt/lib/libz.so.1 (0x2aabf000)
    	libstdc++.so.6 => /opt/lib/libstdc++.so.6 (0x2aae6000)
    	libpthread.so.0 => /opt/lib/libpthread.so.0 (0x2abd3000)
    	libm.so.0 => /opt/lib/libm.so.0 (0x2abf6000)
    	libgcc_s.so.1 => /opt/lib/libgcc_s.so.1 (0x2ac15000)
    	libc.so.0 => /opt/lib/libc.so.0 (0x2ac34000)
    	libm.so.0 => /lib/libm.so.0 (0x2ace9000)
    	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x2ad07000)
    	libc.so.0 => /lib/libc.so.0 (0x2ad26000)
    	libintl.so.0 => not found
    	libstdc++.so.6 => not found
    	ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x2aaa8000)
    But libintl.so.0 is there as a link in /opt/lib

    also

    Code:
    root@Yuki:/# l /opt/lib/libst*
    lrwxrwxrwx    1 root     root          18 Sep 30 19:40 /opt/lib/libstdc++.so -> libstdc++.so.6.0.8
    lrwxrwxrwx    1 root     root          18 Sep 30 19:40 /opt/lib/libstdc++.so.6 -> libstdc++.so.6.0.8
    -rwxr-xr-x    1 root     root      954.4K Aug  5 22:37 /opt/lib/libstdc++.so.6.0.8
    
     
  74. rhester72

    rhester72 Network Guru Member

    I think you may have grabbed a bad build while I was working out the best way to handle shared library paths.

    This is my ldd output (note the lack of libintl):

    Code:
    
    cerberus:~# ldd /tmp/mediainfo
            libz.so.1 => /opt/usr/lib/libz.so.1 (0x2aabf000)
            libstdc++.so.6 => /opt/usr/lib/libstdc++.so.6 (0x2aae3000)
            libpthread.so.0 => /opt/lib/libpthread.so.0 (0x2abce000)
            libm.so.0 => /opt/lib/libm.so.0 (0x2abf2000)
            libgcc_s.so.1 => /opt/lib/libgcc_s.so.1 (0x2ac10000)
            libc.so.0 => /opt/lib/libc.so.0 (0x2ac2f000)
            libm.so.0 => /lib/libm.so.0 (0x2aca5000)
            libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x2acc3000)
            libc.so.0 => /lib/libc.so.0 (0x2ace2000)
            ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x2aaa8000)
    
    Can you try redownloading the binary?

    Rodney
     
  75. ~nephelim~

    ~nephelim~ LI Guru Member

  76. rhester72

    rhester72 Network Guru Member

    PRECOMPILED or PRECOMPILED-static?

    I'm guessing the former, because the bus error is likely caused by some sort of library mismatch. May I see the output of "ldd upx"?

    Rodney
     
  77. mstombs

    mstombs Network Guru Member

    The mipsel_linux upx from sourceforge is 456KB and that runs on my Tomato router!

    Code:
    Tomato v1.28.9052 MIPSR1-beta23 K26 USB Ext
    root@unknown:/tmp/home/root# ls -laF upx
    -rwxr-xr-x    1 root     root        466996 Sep  8 07:07 upx*
    root@unknown:/tmp/home/root# upx --version
    upx 3.07
    NRV data compression library 0.84
    UCL data compression library 1.03
    zlib data compression library 1.2.3
    LZMA SDK version 4.43
    Copyright (C) 1996-2010 Markus Franz Xaver Johannes Oberhumer
    Copyright (C) 1996-2010 Laszlo Molnar
    Copyright (C) 2000-2010 John F. Reiser
    Copyright (C) 2002-2010 Jens Medoch
    Copyright (C) 1995-2005 Jean-loup Gailly and Mark Adler
    Copyright (C) 1999-2006 Igor Pavlov
    UPX comes with ABSOLUTELY NO WARRANTY; for details type 'upx -L'.
    ...
    root@unknown:/tmp/home/root# ldd upx
                           Ultimate Packer for eXecutables
                              Copyright (C) 1996 - 2010
    UPX 3.07        Markus Oberhumer, Laszlo Molnar & John Reiser   Sep 08th 2010
    
    Usage: upx [-123456789dlthVL] [-qvfk] [-o file] file..
    
    Commands:
      -1     compress faster                   -9    compress better
      -d     decompress                        -l    list compressed file
      -t     test compressed file              -V    display version number
      -h     give more help                    -L    display software license
    Options:
      -q     be quiet                          -v    be verbose
      -oFILE write output to 'FILE'
      -f     force compression of suspicious files
      -k     keep backup files
    file..   executables to (de)compress
    
    Type 'upx --help' for more detailed help.
    
    UPX comes with ABSOLUTELY NO WARRANTY; for details visit http://upx.sf.net
     
  78. ~nephelim~

    ~nephelim~ LI Guru Member

    I successfully used the static version. Thanks for making upx available.

    Sorry for such clumsy help-request. :blush:
    It was a truncated(corrupted) upx download that caused that bus error (ldd response was bus error as well) for the precompiled non-static version.

    It was my fault for not double-checking the actual binary size (1MB)

    The precompiled non-static version provide this ldd output.
    Code:
    ldd /tmp/upx
    	libucl.so.1 => not found 
    	libz.so.1 => not found 
    	libstdc++.so.6 => not found 
    	libm.so.0 => /lib/libm.so.0 (0x2aabf000) 
    	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x2aad3000) 
    	libc.so.0 => /lib/libc.so.0 (0x2aaf2000) 
    	ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x2aaa8000)
    upx-tiny (non precompiled-static)
    66c08df9d3f21785136bcc302561d858 /tmp/upx-tiny
    Code:
    ldd /tmp/upx-tiny
    	libucl.so.1 => not found 
    	libz.so.1 => not found 
    	libstdc++.so.6 => not found 
    	libm.so.0 => /lib/libm.so.0 (0x2aac1000) 
    	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x2aad5000) 
    	libc.so.0 => /lib/libc.so.0 (0x2aaf4000) 
    	ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x2aaa8000) 
     
  79. ~nephelim~

    ~nephelim~ LI Guru Member

    Thanks.

    It tested that version as well and now mawk size is 56.4kb :)

    Interestingly enough I found out that even upx for window support mipsel binaries.
     
  80. rhester72

    rhester72 Network Guru Member

    Sure does! I provided mipsel-native UPX mostly for the intellectual challenge. =)

    Take care with --ultra-brute and the like though, the decompression speed on slower CPUs can be pretty bad.

    Rodney
     
  81. Amuro

    Amuro Addicted to LI Member

    I downloaded it again, twice, using wget and curl, via ftp and http

    Both give me the same checksum

    Code:
    # cksfv mediainfo
    ; Generated by cksfv v1.3.14 on 2010-11-08 at 13:34.58
    ; Project web site: http://www.iki.fi/shd/foss/cksfv/
    ;
    ;      4964136  13:34.27 2010-11-08 mediainfo
    mediainfo 73289448
    and both still show the same error as before.

    Perhaps you can provide the sfv checksum of your binary. The one I am d/l comes from PRECOMPILED.

    Or, maybe, upload the binary from your router, perhaps it is a later version than the one in the site (?).
     
  82. ~nephelim~

    ~nephelim~ LI Guru Member

    It's alway a good thing to have a chance to experiment and learn something new. :)


    It's really a pity.

    Would it be possible to compile the embedded decoder portion of upx mipsel stub for a more specific mips architecture that perform faster on tomato routers CPU types (if it is not already)?

    Mine for example is BCM5352 so it should be -mips32, perhaps most routers will support such target code.
     
  83. rhester72

    rhester72 Network Guru Member

    The decompression libraries are already assembly-optimized for each target platform, for the most part. Can't squeeze juice out of a turnip. =)

    Rodney
     
  84. rhester72

    rhester72 Network Guru Member

    md5sum:

    be1ca1e050a7c79cf69826161a3211fb mediainfo

    I'm going to see if I can get a static build working to see if that helps, though there's still some issue with your libraries that I can't figure out. Any chance of seeing an ls -l of your /opt/usr/lib?

    Rodney
     
  85. rhester72

    rhester72 Network Guru Member

    I think that's as good as it gets, size-wise. If you disable the builtin regex engine, you can get down to 127796, but I don't think that's a good idea for an awk clone. ;)

    In fact, I learned something - the -mips32 CFLAG really does make a size difference, though I would never have expected it. Thanks for that tip! ;)

    Rodney
     
  86. ~nephelim~

    ~nephelim~ LI Guru Member

    I agree :biggrin:
    I was hoping for a chance to apply --ultra-brute without incurring in speed penalities.

    Overall mawk is faster but busybox's awk is smaller (compiled build 52 to check awk.o size. it's 51,5 kB)


    I was surprised as well. I was actually hoping to get mawk run faster at the same size but I noticed no improvement speedwise with the awk script I tested.
     
  87. rhester72

    rhester72 Network Guru Member

    *All* binaries recompiled with -mips32 architecture flag for significant size savings (and the -potential- for better performance, but this is NOT guaranteed!).

    To anyone who cares: MediaInfo has been updated to current and is at last available in a full static build (it's so big already that it actually doesn't make that much difference in size). The solution ultimately turned out to be setting an obscure environment variable properly, which is obvious now that I've fixed it. *sigh*

    Rodney
     
  88. Aiko

    Aiko Networkin' Nut Member

    Great. I'll make sure to try the fresh recompile of some packages.
     
  89. Amuro

    Amuro Addicted to LI Member

    Thank you, it is working great now.

    Here is my /opt/usr/lib

    Code:
    -rwxr-xr-x    1 root     root          3072 Nov  6 09:24 libintl.so.0
    -rwxr-xr-x    1 503      503        1156024 Nov  6 08:07 libstdc++.so.6
    -rwxr-xr-x    1 503      503          84552 Nov  6 08:06 libz.so.1
    
     
  90. rhester72

    rhester72 Network Guru Member

    I can't see exact file sizes with those options, but you may want to grab new copies of libstdc++ and libz. I _think_ libintl also has a subdependency (I just shut my Linux box down so I can't check :/), which may well have been the problem all along.

    Rodney
     
  91. mstombs

    mstombs Network Guru Member

    I see this is already the default in Tomatousb Makefile, that also often uses it in association with "-mtune=mips32" any idea what mtune means?
     
  92. ~nephelim~

    ~nephelim~ LI Guru Member


    As far I understood is meant to be coupled with -march=CPU1 when the binary should be finetuned for better performance on a -mtune=CPU2 that support the same or a superset of CPU1 instructions.

    It is probably a way to retain backward compatibility without penalizing perfromance (or force the proper optimizations)

    Code:
    -mips32                 generate MIPS32 ISA instructions
    -mips32 should be equivalent to -march=mips32 or -mcpu= in the above quote
     
  93. mstombs

    mstombs Network Guru Member

    Thanks, but I had assumed the Toolchain which is constructed specifically for building binaries for the Broadcom routers would already know about these! I know that the kernel may need different flags than apps (especially for mipsR2?) and I guess apps need compatible flags to their dynamically linked libraries.

    I guess it is all credit to GNU/Linux that binaries seem so transportable - I have found that sometimes the same router mipsel binaries work on old Ti AR7 (2.4.17 Montavista) as well as Tomato or dd-wrt 2.4 or 2.6 kernels. If they don't work they do usually shout loud quickly!
     
  94. ~nephelim~

    ~nephelim~ LI Guru Member

    True enough it appears that in addition to the instruction set it looks like that using too much different toolchains can introduce incompatibilities.

    I found out a codesourcery mips tooolchain pdf that points out that mtune might default to 4Kc CPUtype when -mips32 is used alone

    Since brcm toolchain lack -mtune broadcom cputype parameters it might not be specific for broadcom at all.

    -mips32r2 enable MIPS DSP ASE instruction set and I guess it could be useful for usemode applications as well if backward compatibility is not a concern
     
  95. rhester72

    rhester72 Network Guru Member

    WARNING: Do *NOT* use -mips32 if you require word-aligned data structure alignment!

    I just found this out the hard way with Transmission. :/ So far it's the only app (in my repository) that is subject to this, but there are definitely others (rtorrent comes to mind).

    Rodney
     
  96. rhester72

    rhester72 Network Guru Member

    Sure enough!

    libintl depends on libiconv - put that in /opt/usr/lib and I'll bet the dynamic mediainfo will work just fine as well.

    Rodney
     
  97. rhester72

    rhester72 Network Guru Member

    Hiawatha 7.4 compiled - this contains two *CRITICAL* bug fixes (one connection leak and one memory leak, both when using XSLT transforms). If you use it, upgrade! =)

    Rodney
     
  98. rhester72

    rhester72 Network Guru Member

    All FTP, all the time - ncftp and lftp compiled...

    ...and now including netkit-ftp-ssl (a la the "standard" FTP client in virtually all Linux distributions).

    Just because I could. *LOL*

    Rodney
     
  99. rhester72

    rhester72 Network Guru Member

    Neat - and useful!

    When you absolutely, positively MUST know what's happening under the hood, I bring you...atop!

    [​IMG]

    Rodney
     
  100. rhester72

    rhester72 Network Guru Member

    Latest ethtool compiled (and sort-of working, even!). The reason for this will hopefully become more obvious in the (near?) future. Hint: it involves the screenshot above and the strangeness with which Tomato handles VLAN interfaces.

    Rodney
     

Share This Page