Tomatoware

Discussion in 'Tomato Firmware' started by lancethepants, Mar 7, 2014.

  1. lancethepants

    lancethepants Network Guru Member

    Exactly correct, Tomatoware binaries are hard coded with Tomatoware's version of libc (uclibc).

    Here is a selection that used to be in the README, but I took out. (Maybe I should put it back in).
    There were too many problems trying to create a system that would build against the tomato libraries. Even more problematic, Tomato's libraries are pretty minimal. Need libssp? You won't find it in Tomato. (This actually caused some issues with compiling dnscrypt, to which the author so kindly removed its mandatory dependency on libspp, and made it optional).

    Getting a semi-modern compiler to work was going to be a no go as well. Tomato actually runs on some older technology. Not saying it's bad in the least bit, but moving to newer stuff is more work than it's worth (and probably impossible too because of binary blobs we have to work with).

    The choice was clear to me that using tomato's libraries was not a possibility.

    Because of this, binaries produced by tomatoware do not use the system's built in libraries, rather, the supplied libraries that come with tomatoware. If you did managed to hard code any binaries you produced with tomatoware to the system libs (possible through some LDFLAGS commands), the results would be unpredictable. Tomato's version of uclibc is older than tomatoware, and uclibc does not guarantee any kind of compatibility from release to release.

    My answer to binary portability has always been to compile static binaries. All the libraries in Tomatoware are available in their static (and dynamic) forms to allow for this.

    edit: If size is going to be restraining issue, you can strip and then compress the binary (with upx). I do this for all binaries I publish. This usually does pretty well.
     
    Last edited: Oct 27, 2014
  2. HunterZ

    HunterZ Network Guru Member

    Thanks for the info. Now I need to decide whether to produce dynamically linked binaries using Tomato cross compiler, and/or just do static builds. I may just go static since I'm already producing them for non-Tomato MIPS router users, and because I've already thrown binary size concerns out the window in other ways.
     
  3. lancethepants

    lancethepants Network Guru Member

    I've had mostly successes with just providing static binaries.

    I will admit one case in which the tomatoware/entware toolchain did not work. Tinc (and I would suspect OpenVPN) on K24 builds of tomato and dd-wrt would not run.
    Entware/Tomatoware uses a 2.6 kernel version. I suspect this was a result of kernel differences between 2.4 and 2.6, and the tun driver. For that I had to specifically get a K24 toolchain.

    I would not suspect you application would have any such issue though.
     
  4. HunterZ

    HunterZ Network Guru Member

    Any chance of support for backtrace() being added? execinfo.h doesn't seem to be included.
     
  5. HunterZ

    HunterZ Network Guru Member

  6. leandroong

    leandroong LI Guru Member

  7. lancethepants

    lancethepants Network Guru Member

    @HunterZ
    Hard to find much information on backtrace and uclibc. Looks like it started out exclusive to glibc. The more you play with c libraries that are NOT glibc, the more you see glibc does not follow posix standards very well. I've seen threads with patches to add it in uclibc, though I'm not sure if they made it to production. I do minimal modifications on the toolchain to adjust for various prefixes, but everything else I just leave to upstream.
    I also just dl the static upx binary as well. I think it might be appropriate to include though. I sometimes transfer my binaries to x86 to compress because it's a lot faster, but I do occasionaly use it on the router too.

    @leandroong
    I'm pretty happy with the entware toolchain. It's probably best to use a toolchain with the same kernel version as the device you're running (or earlier). libc builds against linux kernel headers, but having a later version that the router could potentially cause compatibility issues.
     
  8. HunterZ

    HunterZ Network Guru Member

    I have a script to build and package pixelserv on the router using Tomatoware, so it was nice to have a mips upx to add as a middle step in the script. Otherwise I would have had to turn it into a multi-stage process.
     
  9. Adrianlin

    Adrianlin Serious Server Member

    Hi,
    I would love to compile forked-daapd for entware. Having no experience with compiling, the list of all the dependencies required was pretty daunting. Would I be better off doing it on a Debian system than with Tomatoware?
     
  10. HunterZ

    HunterZ Network Guru Member

    It looks like Debian's repo has a mipsel package here: https://packages.debian.org/sid/forked-daapd

    You may be able to get pretty far with installing as many entware versions of the dependencies as possible and then grabbing Debian mipsel binaries from remaining dependency packages. You may run into trouble if any of them are dynamically linked against incompatible system libraries, though.
     
  11. koitsu

    koitsu Network Guru Member

    I don't think Debian mipsel uses uClibc, do they? If they don't, that'd be the #1 blocker.
     
    Last edited: Nov 21, 2014
  12. lancethepants

    lancethepants Network Guru Member

    You can install debian squeeze packages through this option on the router. (Don't know if a wheezy option exists). Once installed you chroot into the debian environment and can apt-get whatever you want. I haven't looked to see if what you're looking for is present, maybe?
    http://forums.smallnetbuilder.com/showthread.php?t=10558
    Do an "apt-get update" to make sure to get the latest list of packages.
    Before creating Tomatoware I tried compilng applications with this, with no success (hence Tomatoware), but the few packages I tried out seemed to work.

    Tomatoware can be quite capable, you might need the patience of a saint though. I've compiled HandBrakeCLI once (Just to say I could), which has a similarly disheartening list of prereqs (and plenty of other hacks). Being uclibc based too can mean some things need patching.

    Apps that have lots of prereqs, or that I use and upgrade frequently I cross compile, simlar to this fashion.
    https://github.com/lancethepants/aria2-mipsel-static/blob/master/aria2.sh

    If the debian method doesn't have what you're looking for, let me know, and I can take a stab at it.
    edit: Just took a look, don't think I see it.
    edit2: next post.
     
    Last edited: Nov 21, 2014
  13. lancethepants

    lancethepants Network Guru Member

    Here's what I've tried, maybe you can get some results from this.
    Once you've got debian installed. Add
    Code:
    deb http://ftp.de.debian.org/debian squeeze main
    
    to
    Code:
    /etc/apt/sources.list
    
    Run
    Code:
    apt-get update
    apt-get install forked-daapd
    
    From there it starts dbus-daemon, and an avahi-daemon. The dbus seems to need to be running.
    Running "forked-daapd -f" produces the following.
    Code:
    root@RT-BCAEC5C3073A:/# forked-daapd -f
        main: Forked Media Server Version 0.11 taking off
        main: mDNS init
        mdns: Avahi state change: Client connecting
        scan: Could not create eventfd: Invalid argument
        main: File scanner thread failed to start
        main: Database deinit
        main: mDNS deinit
        main: Exiting.
    
    So it does something, but stops. I don't know what's required beyond this point, maybe just a valid config, IDK. Up to you to see if it will actually work.
     
  14. Adrianlin

    Adrianlin Serious Server Member

    Ok, I'm going to take a stab at all the suggestions (starting from using debian packages) over the next couple of days time permitting. Thanks heaps for the help so far :).
     
  15. Adrianlin

    Adrianlin Serious Server Member

    I'm trying to get up to the same point as you Lance, but I'm getting:

    Code:
    mount: only root can do that (effective UID is 1000)
    when I try:

    Code:
    mount -o bind /dev/ /opt/debian/debian/dev/
     
    Last edited: Nov 23, 2014
  16. TeHashX

    TeHashX Serious Server Member

    Where did you untar debian folder? to /opt/debian or to /opt/debian/debian?
     
  17. lancethepants

    lancethepants Network Guru Member

    Make sure to run /bin/mount. I need to remove the mount binary from tomatoware that could be causing your issue.
     
  18. TeHashX

    TeHashX Serious Server Member

    I made a chrooted debian wheezy v7.7 in case someone need it, it's for mipsel devices with all necessary packages to compile transmission, libevent, minidlna... I'm using it on RT-AC66U loaded with Merlin's firmware
     
  19. HunterZ

    HunterZ Network Guru Member

    I wonder if a debian chroot would be the best approach to getting ddd working for graphically debugging programs that are running on the router (with the display variable set to point at a remote X11 server of course).
     
  20. Adrianlin

    Adrianlin Serious Server Member

    I restarted (unmounting Tomatoware in the process, although no idea if this was the problem) and was able to chroot into debian. From this page it seems that the router's kernel might be too dated. Using a newer version of debian wouldn't fix the problem would it?

    Edit: I'm using Shibby build on a Linksys e3200. Just to clarify, I'm asking if chroot debian uses the original linux kernel, or a different one.
     
    Last edited: Nov 24, 2014
  21. TeHashX

    TeHashX Serious Server Member

    Witch router and witch kernel?
     
  22. ceckin

    ceckin LI Guru Member

    I tried compiling the toolchain on Ubuntu and got the same error. After looking a bit it turns out its a regression, which explains why someone might've been able to successfully build it on 14.04, but else will fail.

    Bug report: #15923
     
  23. HunterZ

    HunterZ Network Guru Member

  24. lancethepants

    lancethepants Network Guru Member

    @ceckin @HunterZ See if my the latest commit fixes the issue you are experiencing. I'm gearing up for a 1.0 release, so things are still changing in git.
     
  25. HunterZ

    HunterZ Network Guru Member

    Trying now on apt-get dist-upgrade'd xubuntu 14.04.1 LTS x64. Probably won't have the result for another day.

    Ironically, it's probably a lot faster to build in the Debian VirtualBox VM I've been using specifically to build Tomatoware on my Windows box, as it's much more powerful than my old Xubuntu box.
     
  26. HunterZ

    HunterZ Network Guru Member

    Update:

    I get an error building asterisk. Log attached.
     

    Attached Files:

  27. HunterZ

    HunterZ Network Guru Member

    @lancethepants: It looks like you pushed the 1.0 release, but I didn't hear back on the asterisk-in-Ubuntu thing.
     
  28. leandroong

    leandroong LI Guru Member

    Thanks again.
     
  29. leandroong

    leandroong LI Guru Member

    Curious, when compiling static binary, like lighttpd, I wonder why "/opt/lib" is created in the installer when it is not needed?
     
  30. lancethepants

    lancethepants Network Guru Member

    @HunterZ I'll take a look at it sometime.

    @leandroong The lighttpd authors have stated they won't support static compilation. People have figured how to make it work, but lighttpd installation only works as if dynamic modules are being made.
     
  31. lancethepants

    lancethepants Network Guru Member

    Today I've released version 1.0.1 of Tomatoware. This is an arm only release. This release brings improved and fully functioning arm support.

    The following are the differences in the arm build.
    - uclibc 0.9.33.2 (current latest).
    - gcc 4.9.2 (current latest)
    - the toolchain uses the same kernel as the tomato-arm builds. (mipsel also uses a matching kernel to the firmware)

    Other than the previously mentioned differences, and the fact that they are architecturally different, arm and mipsel releases are identical.

    And since I haven't mention release v1.0 (for mipsel), it has also been released.
     
    AndreDVJ likes this.
  32. lancethepants

    lancethepants Network Guru Member

    Compiling on ARM routers is significantly faster than compiling on mips routers. Also, most of the new ARM routers have the added benefit of dual-core cpus. When compiling most sources, we can take advantage of both cpus and instruct it to compile things in parallel. The following results were achieved on a R7000 router.

    openvpn
    1 cpu 2m 58.297s
    2 cpu 1m 49.966s

    nginx
    1 cpu 4m 37.952s
    2 cpu 2m 43.989s

    busybox
    1 cpu 7m 0.110s
    2 cpu 4m 19.535s

    You can't quite expect things to be twice as fast, as you can see in the numbers, but you can still expect a significant increase in compilation speed.
    Compilation time on the previously mentioned applications is very reasonable. Compiling on mips at times can require the patience of a saint, particularly on large sources. +1 Arm

    When running 'make', we can just invoke the -j option to enable parallel building.

    'make -j2' will instruct the compilation process to use both cpus.
     
    johaug likes this.
  33. dskete

    dskete Serious Server Member

    Hi Lance, can you help me with the compilation ? I cloned the latest version of the tomatoware git, trying to compile it on Ubuntu 14.04.1_x64. I installed all the packages including zlib1g-dev. I am getting this error compiling Python

    File "/home/tomatoware/src/setuptools/setuptools/setuptools/command/bdist_egg.py", line 472, in make_zipfile
    z = zipfile.ZipFile(zip_filename, mode, compression=compression)
    File "/home/tomatoware/src/python/Python-2.7.3/Lib/zipfile.py", line 681, in __init__
    "Compression requires the (missing) zlib module"
    RuntimeError: Compression requires the (missing) zlib module
    make: *** [tomatoware] Error 1

    --
    Thanks.
     
  34. lancethepants

    lancethepants Network Guru Member

    @dskete See if you have lib32z1 installed.
     
  35. dskete

    dskete Serious Server Member

    @lancethepants, that package is installed and I have the latest version. Thanks.
     
  36. lancethepants

    lancethepants Network Guru Member

    Can you tell me which line on base.sh is the last command before it fails?
     
  37. dskete

    dskete Serious Server Member

    @lancethepants, here is most of the build logs I could save. Thanks.
     

    Attached Files:

  38. lancethepants

    lancethepants Network Guru Member

    Hmm, it looks like the host python was built without ssl support? Can you check that libssl-dev is installed? If that's the case, install it, delete the '.extracted' file from the src/python folder and run make again.

    I know I've successfully built tomatoware on Ubuntu 14.04 in the past, but it's been a while. If that doesn't fix the issue, I'll try compiling tomatoware on Ubuntu 14.04 again and see if I come upon the issue.

    Unless you specifically wanted the python modules, they're not essential for the tomatoware development environment. You could comment them out to at least get past them. Not sure if package.sh may stop when it's packaging looking for them though.
     
  39. dskete

    dskete Serious Server Member

    Thanks Lance. Looks like libssl-dev is already installed. I will try your suggestions to disable python compilation. I will also try the compilation on Debian 7 (Ubuntu is not a requirement :)). Thanks for all your support.
     
  40. dskete

    dskete Serious Server Member

  41. lancethepants

    lancethepants Network Guru Member

    @dskete Yes. (Just got today's latest kernel update).
     
  42. dskete

    dskete Serious Server Member

    Thanks Lance. I will setup Debian on Virtualbox and try it out.
     
  43. dskete

    dskete Serious Server Member

    compiled perfectly on Debian 7.8. Thanks.
     
  44. lancethepants

    lancethepants Network Guru Member

  45. lancethepants

    lancethepants Network Guru Member

  46. lancethepants

    lancethepants Network Guru Member

    @dskete
    Don't know if you still have your ubuntu setup around. If you do, could you try installing dpkg-dev, which as I understand is not installed by defualt in Ubuntu, but apparently is in Debian. At that point, from tomatoware run
    Code:
    rm src/python/.extracted
    
    and then running 'make'. Removing the .extracted file tells tomatoware to compile python from scratch again.
     
  47. dskete

    dskete Serious Server Member

    Thanks Lance. I will try it out.

    Sent from my A0001 using Tapatalk
     
  48. dskete

    dskete Serious Server Member

    Lance, dpkg-dev was already installed on Ubuntu; I got past the python issue by copying libz.so to /usr/lib and /lib.

    However, sometimes /usr/bin/perl gets replaced by the ARM version, I solved that by blowing up the VM and recreating it. I didn't save the build logs for the failure case, that's okay though. Thanks for all your help.

    Sent from my A0001 using Tapatalk
     
    lancethepants likes this.
  49. david3

    david3 LI Guru Member

  50. lancethepants

    lancethepants Network Guru Member

    @david3
    No, it's hard-coded for that particular prefix. Most tomato firmware's should have /mmc. I could compile and upload an /opt version if you like though.
     
  51. ceckin

    ceckin LI Guru Member

    Could you do the same for JFFS?

    Thanks in advance.
     
  52. david3

    david3 LI Guru Member

    If it's not too much trouble, that would be great.
     
  53. lancethepants

    lancethepants Network Guru Member

    @david3 @ceckin I'll get some images up today. Will take a few hours to compile.
     
  54. steom

    steom Connected Client Member

    @lancethepants
    can you take a look per plex media server?
    https://plex.tv/downloads
    there is two armv7 package .spk and .rbw (tar archives)
    can you make it run on our entware/optware?
    the armv7 has audio transconding capability.
     
  55. lancethepants

    lancethepants Network Guru Member

    @david3 @ceckin
    I've uploaded the various versions.
    Is there a particular reason you need something other than the /mmc version? Is /mmc unavailable in the particular firmware you're using?
    I was hoping to sort of make /mmc the quasi-official prefix for tomatoware.
    I really don't mind compiling the various versions, but was just curious about it.
     
  56. jerrm

    jerrm Network Guru Member

    I know it's convenient off the root, but you might consider something like /mnt/tw as the tomatoware root. Universally available then.
     
  57. lancethepants

    lancethepants Network Guru Member

  58. leandroong

    leandroong LI Guru Member

    Padavan fw doesnt have /mmc or /jffs, better make /opt/tomatoware as official in the near future.
    note2: /mnt/tomatoware would be nice as official also
     
  59. lancethepants

    lancethepants Network Guru Member

    I'll just have to deal with multiple compiled prefixes, because /opt is useful for entware
     
  60. leandroong

    leandroong LI Guru Member

  61. steom

    steom Connected Client Member

  62. AndreDVJ

    AndreDVJ LI Guru Member

    I believe a separate prefix (/mmc) is a cleaner approach and easier to maintain, and to be honest I liked the move. I know it's just a matter of downloading the sources and compile it, but I'd like to see things kept simple.
     
  63. lancethepants

    lancethepants Network Guru Member

    @steom Alright, I've worked up some of my magic and got it working. I'll post a howto in it's own thread later today probably.
     
  64. david3

    david3 LI Guru Member

    Thanks for the mipsel opt version. I do have an /mmc directory, but it's /opt that I have mounted to my flash drive right now.

    I could try to use the /mmc version, but I've always used /opt, so I'm not sure if it would break anything. I used to use optware, but when I installed tomatoware I didn't need it anymore. So I'm just using tomatoware without optware or entware.
     
  65. steom

    steom Connected Client Member

    I had no doubts about it
    thanks a lot
     
  66. leandroong

    leandroong LI Guru Member

    During my backup syncing of this /opt (tomatoware), I notice the ff files that keeps on changing, so exclude them from updating my backup copy. May I know what are these?
    \include\linux\netfilter\xt_connmark.h
    \include\linux\netfilter\xt_dscp.h
    \include\linux\netfilter\xt_mark.h
    \include\linux\netfilter\xt_tcpmss.h
    \include\linux\netfilter_ipv4\ipt_connmark.h
    \include\linux\netfilter_ipv4\ipt_DSCP.h
    \include\linux\netfilter_ipv4\ipt_ecn.h
    \include\linux\netfilter_ipv4\ipt_MARK.h
    \include\linux\netfilter_ipv4\ipt_tcpmss.h
    \include\linux\netfilter_ipv4\ipt_TTL.h
    \include\linux\netfilter_ipv6\ip6t_mark.h
    \share\gtk-doc\html\gio\gsettings.html
    \share\man\man8\PAM.8
    \include\linux\netfilter_ipv6\ip6t_hl.h
    \ssl\man\man3\BN_print.3
     
  67. lancethepants

    lancethepants Network Guru Member

    What do you mean they keep changing? Are you rsyncing your backup?, or do you mean between versions of tomatoware? They look mostly to be linux header files.
     
  68. leandroong

    leandroong LI Guru Member

    It means that file sizes keeps on changing.
    This is what i use for "Synchronize Files and Folders", http://www.freefilesync.org/

    note: I perform backup whenver I update compiled binary like transmission, aria2, lighttpd, etc.

    note2: You can easily duplicate issue by performing repeated backup for 2x.

    note3: I think there is bug in "FILESYNC" software, case sensitive problems. Example, it treats \include\linux\netfilter\xt_connmark.h and \include\linux\netfilter\xt_CONMARK.h to be the same. Perhaps, you can avoid having same name ?

    note4: http://sourceforge.net/p/freefilesync/discussion/open-discussion/thread/cf572de0/#d43e
     
    Last edited: Sep 9, 2015
  69. jerrm

    jerrm Network Guru Member

  70. HunterZ

    HunterZ Network Guru Member

    @jerrm: While waiting for an answer, you may try running UPX on the ARM binary from a Linux PC. It works on MIPS binaries that way.
     
  71. jerrm

    jerrm Network Guru Member

    I know. Curious if it was intentional or oversight. The pre-built ARM executable from the upx site doesn't run for me on the router.
     
  72. HunterZ

    HunterZ Network Guru Member

    Is it possible to build a working ARM version from source using Tomatoware?
     
  73. jerrm

    jerrm Network Guru Member

    Probably. More trouble than it was worth for my needs. I cheated and built a static version of the debian package in my debian chroot environment and copied it out to the tomatoware bin folder.

    It gets the job done and allows your unaltered Makefile to complete.
     
  74. lancethepants

    lancethepants Network Guru Member

    @jerrm @HunterZ

    In the mipsel version I've added their official binary. They don't have a pre-built version made for armel though. The version they distribute uses the nrv library which is not open source. nrv is supposed to have better compression. I have compiled an arm version using tomatoware that at the time used the latest svn commits. It instead uses the ucl library which is open source, and has pretty well comparable compression. Last I remember, the difference was pretty well indistinguishable.

    http://files.lancethepants.com/Binaries/upx/

    I've had mixed results with upx when I was using a musl based toolchain for arm. Sometimes the binaries wouldn't work after compression. I haven't tested so much with tomatoware arm yet though. Let me know if you encounter any issues.

    If it seems to work alright for you guys, I could probably compile it in tomatoware. I'd be tempted to just compile the mipsel version as well. There's a certain satisfaction knowing that tomatoware would be 100% open source (afaik).
     
  75. steom

    steom Connected Client Member

    Last edited: Nov 7, 2015
  76. lancethepants

    lancethepants Network Guru Member

    @steom
    lol, it was only earlier this year and tomatoware was using the latest everything (arm anyway).

    A few weeks ago I built a version of tomatoware which uses uclibc-ng and gcc 5.2. Just did a bunch of hacking to get it to work, so nothing committed to git. I'll probably having something in a few weeks. Upgrading the toolchain is a bigger pain than other things, so not sure it will happen as often as everything else. I don't really have much interest with mipsel any more, so I'll just continue using the entware toolchain with those releases.
     
  77. steom

    steom Connected Client Member

    sorry I neglected to say only for armv7
     
  78. lancethepants

    lancethepants Network Guru Member

    I've uploaded pre-release 1.1-RC1 with some updates. Both mipsel and ARM and have been upgraded to use uclibc-ng 1.0.9, and gcc-5.2.

    I use buildroot for the toolchain to compile Tomatoware, so I'm waiting for their final release. I also have a few other things I'd like to get done before an official release. This pre-release however should be fully functioning, and I've been using it myself. I also thought I'd let it be tested a bit since I might some big changes.

    I've currently uploaded an arm and mipsel build for the /mmc prefix. I would like to see /mmc become the home for Tomatoware. It takes a lot of time to build 3-4 versions for each architecture. Shibby so graciously added /mmc to his builds for this purpose, and it was trivial to do.
    https://bitbucket.org/pl_shibby/tomato-arm/pull-requests/7/add-mmc-directory/diff

    If you're using a firmware that doesn't include /mmc, would you please ask the maintainer to add an /mmc directory? Possibly you could also do a pull request. Does this seem reasonable to you guys?

    For now I can compile different prefixes upon request.
     
  79. leandroong

    leandroong LI Guru Member

    Can you possible provide us with tomatoware clean-up script installed on /opt?
    or Do you think better to just remove all /opt and re-install entware and optware installed ?
     
  80. lancethepants

    lancethepants Network Guru Member

    This line mentions all the directories included with tomatoware.
    https://github.com/lancethepants/tomatoware/blob/master/scripts/package.sh#L134

    You can just delete those directories to remove the tomatoware installation. Were you using /opt/tomatoware before? In that case you could just delete the /opt/tomatoware directory.

    If you have both tomatoware and entware installed in /opt (not /opt/tomatoware), sharing the /opt directory, that would not be a recommended way to have both because they could potentially clash. If that were the case, I newly install both in separate directories.
     
  81. leandroong

    leandroong LI Guru Member

    Thanks. Awaiting for Padavan FW to add FS "/mmc" before doing any tomatoware installation.
     
  82. leandroong

    leandroong LI Guru Member

    Padavan FW has FS "/mnt" which is not being use ATM, nice if I can put it there also as alternative
     
  83. lancethepants

    lancethepants Network Guru Member

    Hopefully they will be willing. I will make /mnt version while you wait for an answer from padavan. Do you need mipsel or arm? I think others have been using the tomatoware /opt version co-existing with entware. While it seems to work, it is not recommended, and may become more problematic since I'm now using newer versions of uclibc-ng.
     
  84. leandroong

    leandroong LI Guru Member

    mipsel tomatoware. Thanks again
     
  85. lancethepants

    lancethepants Network Guru Member

  86. leandroong

    leandroong LI Guru Member

    This is what I did:
    1.) due to limited space on my router "/mnt", I did the ff:
    on "Run After Firewall RulesRestarted", I add "mount -o bind /media/myssd/mnt /mnt"
    2.) reboot router, download mipsel-sfot-mnt.tgz and extract
    3.) copy and modified /mnt/etc/profile, content as follows

    Note: making this official is nice also :}
     
    Last edited: Dec 4, 2015
  87. leandroong

    leandroong LI Guru Member

    @lacetheparts, could you provide correct combine profile?

    note2: here is my working combine profile for /mnt:
    Code:
    #!/bin/sh
    
    # Please note it's not a system-wide settings, it's only for a current
    # terminal session. Point your f\w (if necessery) to execute /mnt/etc/profile
    # at console logon.
    
    export PATH='/mnt/sbin:/mnt/bin:/opt/usr/sbin:/opt/sbin:/opt/bin:/usr/local/sbin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/usr/sbin:/opt/sbin:/opt/bin'
    export TERM=xterm
    export TERMINFO=/mnt/share/terminfo
    export TMP=/mnt/tmp
    export TEMP=/mnt/tmp
    export PKG_CONFIG_LIBDIR=/mnt/lib/pkgconfig
    export M4=/mnt/bin/m4
    
    # You may define localization
    #export LANG='ru_RU.UTF-8'
    #export LC_ALL='ru_RU.UTF-8'
    
    alias ls='ls --color'
    
    # This is for interactive sessions only
    if [ "$PS1" ] ; then
      export TERM=xterm
      [ -d /mnt/share/terminfo ] && TERMINFO='/mnt/share/terminfo'
      export LANG='en_US.UTF-8'
      export LC_ALL='en_US.UTF-8'
    fi
    
     
    Last edited: Dec 4, 2015
  88. koitsu

    koitsu Network Guru Member

    This should be:
    Code:
    if [ -n "$PS1" ]; then
    -n is used to test a variable or string/value for non-zero-length. For zero-length, use -z. These test operators should work in Busybox sh (ash). The quotes around the variable are needed for situations where PS1 might not even be defined/assigned (in concept this is similar to how you test for defined variables on Solaris sh, via if [ x$foo != x ]; then
     
  89. szpunk

    szpunk LI Guru Member

    @lancethepants , Does any chance you will add CMake and Go support into Tomatoware (at least ARM version)? I compile these two on tomatoware for now, but i dont want to do this again and again when tomatoware update~

    The new router processor like 4708/4709 capacity has been able to compile and run a similar GO such program. I want take these routers as RPi with few I/O~

    BTW: For Go, we need set two special environment variable:

    Code:
    export TMPDIR=/mmc/tmp
    export GOPATH=/mmc/go
    
    Go cant identification variables TMP and TEMP, so If you don't set TMPDIR, all temp file which on compile process will be put into /tmp, and fill it up quickly.
     
  90. lancethepants

    lancethepants Network Guru Member

    @szpunk
    cmake is already included in tomatoware. I'm not too familiar with go, but I could look into maybe including it. How do you do a native compile on the router?

    Lately I've been playing around with distcc. Using this I've been able to offload compiling from the router to a x86_64 pc using the tomatoware cross-compiling toolchain. It greatly helps speed up compiling large programs while still having the "native" compiling experience and also avoiding the nightmare that is cross-compiling. This method would also make compiling large things like go much more bearable. I'm able to compile aria2 this way in ~5 minutes, while pure native compilation takes much longer.
     
  91. szpunk

    szpunk LI Guru Member

  92. lancethepants

    lancethepants Network Guru Member

    @szpunk
    So it doesn't look like you can cross-compile a go development environment for a different target architecture. go1.5 says it makes cross-compiling applications themselves for other architectures easier, but I think it gets more complicated if you need cgo. (I'm not fully educated on go, but I've learned quite a bit in the process of this).

    GCC does have gccgo, but I don't see the option in the buildroot toolchain I use. It would be using a dated version of go anyway, and people tend to want the latest and greatest.

    So what I've done is, I will bundle a script that will automate building go for you. I put the base of go in /mmc/bin/go. The location I don't think matters so much, so long as you have your environment variables and PATHs setup. Go (and docker) decided to be funny, and require themselves as a perquisite to build themselves. So it builds go1.4.3 to bootstrap building go1.5+.

    I also agree with having GOPATH=/mmc/go, so that's where all the development and compiled stuff will go. The profile file will now come with /mmc/bin/go/bin and /mmc/go/bin in its path automatically for future version. It also add the TMPDIR and GOPATH environment variables.

    Here's the script I've made which takes ~24 minutes on my R7000. It will be in /mmc/scripts/build_go.sh
    Hopefully after running that everything else should be setup and just work. I patched go so that it knows where to look for a certificate authority. I ran into a problem when testing compiling some applications that they couldn't download prerequisites because it couldn't trust them without the certificate.
    https://github.com/lancethepants/tomatoware/blob/master/src/golang/build_go.sh

    Unfortunately mipsel will not be included as it isn't a supported architecture at this time, but I've hardly looked back once I made the move to arm anyway.
     
    szpunk likes this.
  93. lancethepants

    lancethepants Network Guru Member

    AndreDVJ and szpunk like this.
  94. leandroong

    leandroong LI Guru Member

    @lancethepants, I keep getting distcc exit error code 110. I wonder if this is how to start it:
    1.) open terminal
    2.) PATH=/opt/tomatoware/mipsel-soft-mnt/usr/bin/:$PATH
    3.) distccd --allow 10.0.1.0/24
    note: my lan subnet 10.0.1.0, router ip: 10.0.1.1

    note2: Looks correct, running well, thanks again.

    note3: easiest approach:
    1.) open terminal
    2.) edit ~/.bashrc (sudo) and enter path there, save 1 step
     
    Last edited: Jan 31, 2016
  95. leandroong

    leandroong LI Guru Member

    With tomatoware toolchain installed, I just realized that I can compile binary package from virtualbox. This is becoming a good replacement for entware, I think.
     
  96. leandroong

    leandroong LI Guru Member

    Comparison of cross compilation of latest tomatoware toolchain and my old entware toolchain :
    1. Entware
    - transmission - ok
    - aria2 - ok
    -unbound - ok (segmentation error when running)

    2. tomatoware
    - transmission, failed on "gettext - 0.19.7"
    - aria2, failed on "sqlite latest - 310*"
    - unbound - ok (segementarion error when running)

    It seems that unbound source code has an issue. But for gettext and sqlite, not sure if I need additional repositories installed, I remember installing "git, libtool and autoconf" only in tomatoware environment.

    note: I was able to compile aria2 by downloading sqlite version to 3090200. Transmission still failed even if I lower gettext to 0.19.5
     
    Last edited: Feb 1, 2016
  97. lancethepants

    lancethepants Network Guru Member

    @leandroong
    https://github.com/lancethepants/unbound-mipsel-linux/issues/1
    Please don't fragment the cross-compile discussion and post in multiple places.
    This is for native-compilation & using distcc. The cross-compilation github scripts is where you can discuss cross-compilation.

    github:
    That looks like it is starting correctly. That is also what I see when I start distcc on my PC.
     
  98. leandroong

    leandroong LI Guru Member

    Sorry about double posting. Distccd is working well, only mistake i made i used wrong prefix, after correcting it works immediately. I can see it reported on config.log

    I created another virtualbox for distccd installation and in the process I learned that I can also perform cross-compilation. That gives me an idea of comparing entware toolchain and tomatoware toolchain. Binary generated by tomatoware is 100% compatible to my router.
     
    lancethepants likes this.
  99. fonos

    fonos Serious Server Member

    @lancethepants I've dabbled with the distcc option so as to reduce 1-hour+ OpenSSL compile times on my RT-N16 and while it's sort of working, I'm not sure I'm getting full performance from it. Here are some representative times for the make stage
    Code:
    Without distcc
    3444.58user 126.80system 59:41.27elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
    0inputs+0outputs (26major+4663660minor)pagefaults 0swaps
    
    With distcc
    521.22user 107.17system 12:45.40elapsed 82%CPU (0avgtext+0avgdata 0maxresident)k
    0inputs+0outputs (21major+1984331minor)pagefaults 0swaps
    So a pretty impressive 80%-ish saving in elapsed time. The strange thing is that the server, a modest i3-based Ubuntu 14.04 box, seems to be doing very little - so much so that when running top I very rarely see signs of any compilation occurring.

    1. If I run distcc -j it correctly returns a value of 4 (2 cores x 2 hyperthreading). If I run make -j4 CC="distcc mipsel-linux-gcc" I get a message
    Code:
    make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
    The message only disappears when I specify -j without a number or omit it altogether, but I suspect that it's still running a single job. I know I'm not the only person to have encountered this message, eg https://lists.samba.org/archive/distcc/2004q1/002160.html but the solution is so far elusive. Since I'm not the author of the OpenSSL Makefile, any solution will probably need to involve some kind of command line parameter, rather than hacking their code.

    2. The OpenSSL compilation process now requires a make depend after the configure and before the make. This step simply fails to work with distcc. The Makefile defines a separate compiler variable for the make depend step: MAKEDEPPROG. Using this in the command line gives us:
    Code:
    make depend -j4 MAKEDEPPROG="distcc mipsel-linux-gcc"
    making depend in crypto...
    make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
    make[1]: Entering directory '/tmp/mnt/sda1/ossl/test/crypto'
    cc: error: mipsel-linux-gcc: No such file or directory
    cc: error: unrecognized command line option '--'
    cc: error: unrecognized command line option '--'
    distcc[3110] ERROR: compile cryptlib.c on localhost failed
    mv: can't rename 'Makefile.new': No such file or directory
    Makefile:136: recipe for target 'local_depend' failed
    make[1]: *** [local_depend] Error 1
    make[1]: Leaving directory '/tmp/mnt/sda1/ossl/test/crypto'
    Makefile:471: recipe for target 'depend' failed
    make: *** [depend] Error 1
    Moving the depend to the end produces the same result. Fortunately, the make depend step only takes 15 minutes without distcc but it would be nice to offload the processing if possible.

    If you could provide some pointers, I'd be most grateful.

    Edit: With -j instead of -j4 I see 4 parallel jobs running on the router, so it's not restricted to a single job.
     
    Last edited: Mar 4, 2016
  100. lancethepants

    lancethepants Network Guru Member

    @fonos
    I actually tried doing this last week. I'm not sure what the make depend does, but it sounds like you have to run it locally. I had also done this, but on my ARM router it's a lot faster.

    To build openssl in parallel while using distcc I use this patch.
    https://raw.githubusercontent.com/droboports/git/master/src/openssl-1.0.2f-parallel-build.patch

    OpenSSL Makefiles aren't designed to be make things in parallel, and you will get errors. The devs aren't planning on this feature either because it would be specific to gnu make and not compatible with other makes like the bsds. You can however use that patch and should build successfully in parallel.

    Even when I compile things using arm and I look at the PC doing the compilation, you don't see it pop up much in htop. I sometimes will run "ps -A | grep arm" and will see some instances there. It happens so fast I don't think it has time to register, but it's obviously working as you've seen. Compile times are a fraction of what they otherwise would be.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice