Adding software packages to tomato.

Discussion in 'Tomato Firmware' started by onehomelist, Apr 5, 2010.

  1. onehomelist

    onehomelist Addicted to LI Member

    Are there any guides or wikis to help those who want to compile tomato or for those who want to add some software to it.
  2. Toastman

    Toastman Super Moderator Staff Member Member

    Compiling Tomato

    There's a lot of posts which will help, and a few articles on the web, though those tend to be very confusing. But mostly, you have to use the advanced search options and learn from posts before you get stuck in.

    This thread contains a lot of useful hints, please read through to the end.

    But before you begin to compile anything from git or other places, I would first download and make sure you can compile Victek's RAF firmware sources. This is the easiest path to follow especially if you are a beginner at this, because the source is complete and you don't need to mess with it. In other words - it SHOULD compile, and if it doesn't the fault is with your compiling environment. Most intros make you compile the whole thing from linksys code up, and to be honest, this is not so simple to do, and is very confusing at first. I think it is far, far better to begin with something that immediately proves that you have a correct build environment.

    Here's what I would do:

    Get yourself a Linux distro and install it. I would recommend Linux Mint 8 (Helena), this is a very easy one to install and use, and avoid Ubuntu which is not so straightforward. I've just run through a great many distros before dumping Ubuntu and using this one. Tip - if you can, do this on a separate hard disk, and unplug all others while you do a Linux install, this will avoid causing boot sector problems with your other disks if, like me, you are a beginner with Linux.

    Now, ignore all commonly offered advice and make sure you can log in as root, it will save your sanity. In Mint you do this by just entering a password for the root user in the configuration.

    Add the required extra packages from the online depositories. You need:


    Download source code and copy to /root directory.

    Open a command window and change directory to /root, if not there already - all work is now done from the command window. Now unzip source code, which will create a /root/tomato directory tree. [You can also use the GUI to unzip but it's much slower and sometimes I have found it to result in an incorrect tree that does not compile.]

    tar -xvf Tomato_RAF_1.25.8515.2_Source_Code.tar.bz2

    Copy /root/tomato/tools/brcm directory to /opt.

    Create simlinks:

    ln -s /opt/brcm /root/tomato/tools/brcm;ln -s hndtools-mipsel-uclibc-0.9.19 hndtools-mipsel-uclibc;ln -s hndtools-mipsel-linux-3.2.3 hndtools-mipsel-linux

    Add directories to your path:

    export PATH=$PATH:/opt/brcm/hndtools-mipsel-uclibc/bin;export PATH=$PATH:/opt/brcm/hndtools-mipsel-linux/bin

    Change to /root/tomato/src directory to compile tomato, which will appear in src/image directory when finished.

    make clean


    And hopefully, it should run through to the end, although TBH, there always seems to be something one has overlooked... so I hope I didn't forget anything! The usual sources of problems are wrong paths and lack of simlinks, so check these first.

    Check my post against others to see what different people did.
    kille72 likes this.
  3. onehomelist

    onehomelist Addicted to LI Member

    Thanks Toastman. What you wrote was more than I expected, thanks. I will give a try at compiling from victek's sources. I already have a Linux Mint 8 installed box (32bit). I have built custom Linux by following LFS guide, so I'm pretty confident.
  4. Toastman

    Toastman Super Moderator Staff Member Member

    Good luck!

    I think it would be appropriate here to add my thanks to Victek, Fedor, mstombs, SGTPepperKSU and others, for encouraging and helping me when I was stuck - which was (and still is) a daily occurrence.

    Thanks Guys!
  5. mstombs

    mstombs Network Guru Member

    You have me confused with your symlinks Toastman!

    You should either copy brcm... to /opt or set symlinks. Setting symlinks is useful to keep the toolchain with the code you are compiling, but not all firmware distros work (I've had trouble with depth of folders created sometime). Linksys, dd-wrt, and all flavours of tomato use /opt/brcm.

    I prefer to create an empty folder in /opt/brcm and do it all via symlinks - because /opt/brcm can and does get auto upgraded/ trashed by toolchain building tools etc. But I am not best guide because I often get it wrong... (ln syntax always seem wrong way round...)
  6. Toastman

    Toastman Super Moderator Staff Member Member

    Oh, feel free to point out any mistakes. The simlinks I posted were with me since I began about 18 months ago, from the Linksys code right thru Victek's code. In fact, I just reread Victek's instructions from a year ago, and he also does the same thing exactly as I described above. Without them things don't work for me. When compiling from git, and with USBmod code, I don't need to do anything though.

    TBH when I used Ubuntu, I had to continually jump through hoops to get something working, only to find it didn't work with the next guy's code. BTW - I have always had problems when using any other directory than /root. I have no idea why - I'm very much the beginner! Something to do with stripping? (Don't get excited).
  7. mstombs

    mstombs Network Guru Member

    Well my /opt is a mess, currently contains

    $ ls /opt -laF
    total 52
    drwxr-xr-x 13 root    root    4096 2010-03-30 23:14 ./
    drwxr-xr-x 23 root    root    4096 2010-04-05 12:02 ../
    drwxr-xr-x 12 user    user   4096 2006-03-10 14:33 3.4.6-uclibc-0.9.28/
    drwxr-xr-x 13 user    user   4096 2006-03-10 17:20 4.1.0-uclibc-0.9.28/
    lrwxrwxrwx  1 root    root      31 2010-03-30 23:14 brcm -> /home/user/tomato/tools/brcm/
    drwxr-xr-x  6 root    root    4096 2010-02-01 20:28 brcm.old/
    drwxrwxrwx  3 root    root    4096 2010-03-25 21:10 brcmsav/
    drwxr-xr-x  4 root    root    4096 2009-09-07 21:52 gcc/
    drwxr-xr-x  3 root    root    4096 2010-03-06 12:01 google/
    drwxr-xr-x  5 root    root    4096 2009-09-07 21:49 hardhat/
    drwxr-xr-x  3 root    root    4096 2009-09-02 21:32 Samsung/
    drwxr-xr-x 12 root    root    4096 2009-01-27 23:40 toolchain-mipsel_gcc4.1.2/
    drwxr-xr-x  3 root    root    4096 2009-11-03 23:58 usr/
    but I'm pretty sure that the toolchain in use is that linked by the virtual folder brcm, created using command

    sudo ln -s ~/tomato/tools/brcm /opt/brcm
    but to be 100% sure I need to remove the other versions that are left hanging around!

    where my tomato git sourcecode is is in /home/user/tomato, and I'm still on Ubuntu - latest 64 bit beta
  8. Toastman

    Toastman Super Moderator Staff Member Member

    Haha, finally I took the exorcist's advice ... Unless Ubuntu stop breaking things with every new issue, I will not use it again.
  9. onehomelist

    onehomelist Addicted to LI Member

    ldd.c:96:2: warning: #warning "You really should add a MATCH_MACHINE() macro for your architecture"
    ldd.c: In function 'find_dependancies':
    ldd.c:528: error: 'ELFCLASSM' undeclared (first use in this function)
    ldd.c:528: error: (Each undeclared identifier is reported only once
    ldd.c:528: error: for each function it appears in.)
    ldd.c:529: warning: implicit declaration of function 'MATCH_MACHINE'
    make[4]: *** [ldd] Error 1
    make[4]: Leaving directory `/home/bill/tomato/tools-src/uClibc/ldso/util'
    make[3]: *** [utils] Error 2
    make[3]: Leaving directory `/home/bill/tomato/tools-src/uClibc/ldso'
    make[2]: *** [shared] Error 2
    make[2]: Leaving directory `/home/bill/tomato/tools-src/uClibc'
    make[1]: *** [libc] Error 2
    make[1]: Leaving directory `/home/bill/tomato/release/src'
    make: *** [all] Error 2
    I get the above error after I give the make command. I am using victek source, buildig it on Linux Mint Helena 8 installed on a amd64 machine.
  10. mstombs

    mstombs Network Guru Member

    64 bit is always more awkward, because you need the 32 bit libs and force any host tools compile to 32 bit, or patch the code to properly support 64 bit. That particular issue is patched in the Git repository

    You need to add

     /* X86_64  */
     #if defined(__x86_64__)
     #define MATCH_MACHINE(x) (x == EM_X86_64)
    to /home/bill/tomato/tools-src/uClibc/ldso/util/ldd.c
  11. Toastman

    Toastman Super Moderator Staff Member Member

    I've seen that message before, but don't remember what fixed it. It went away when I used the setup I described. I assume you ran "make clean" already?

    The directory path is what often screwed things up here in the past, which is why I mentioned it. Try it in /root/tomato.

    EDIT crossed with mstombs. Sorry, didn't dawn on me it was 64 bit linux ... thought you meant just the machine.
  12. onehomelist

    onehomelist Addicted to LI Member

    I tried compiling on a 32 bit pc, so I was able to solve the above proble. Now the following is the new error that I get while compiling on 32 bit pc

    You will probably need to make your busybox binary
    setuid root to ensure all configured applets will
    work properly.
    make[2]: Leaving directory `/home/bcalab/tomato/release/src/router/busybox'
    make[2]: Entering directory `/home/bcalab/tomato/release/src/router/httpd'
     [httpd] Installing to /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/httpd
    make[2]: Leaving directory `/home/bcalab/tomato/release/src/router/httpd'
    make[2]: Entering directory `/home/bcalab/tomato/release/src/router/www'
    rm -rf /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/www/www
    mkdir -p /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/www/www
    cp *.gif *.png *.ico robots.txt /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/www/www
    for F in bwm-common.js bwm-hist.js md5.js protocols.js tomato.js status-data.jsx; do \
    		sed '/^\/\*\s*$/,/\*\//! { s/^\s\+//; s/\s\+$//; /^\/\/ --\+\s*/d; /^$/d }' < $F > /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/www/www/$F; \
    for F in absolon.css asus.css black.css blue.css bluedream.css bluegreen2.css bluegreen.css brown.css buffalo.css cyan.css enlightened.css linksys.css olive.css pumpkin.css purple.css red.css tijuana.css tomato.css usbblue.css; do \
    		sed '/^\/\*\s*$/,/\*\//! { s/\s\+/ /g; s/^\s\+//; s/\s\+$//; /^$/d }' < $F > /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/www/www/$F; \
    for F in about.asp admin-access.asp admin-buttons.asp admin-bwm.asp admin-cifs.asp admin-config.asp admin-debug.asp admin-jffs2.asp admin-log.asp admin-sched.asp admin-scripts.asp admin-upgrade.asp advanced-ctnf.asp advanced-dhcpdns.asp advanced-firewall.asp advanced-mac.asp advanced-misc.asp advanced-routing.asp advanced-wireless.asp basic-ddns.asp basic-ident.asp basic-network.asp basic-static.asp basic-time.asp basic-wfilter.asp bwm-24.asp bwm-daily.asp bwm-monthly.asp bwm-realtime.asp bwm-weekly.asp clearcookies.asp error.asp forward-basic.asp forward-dmz.asp forward-triggered.asp forward-upnp.asp logout.asp mnoise.asp new-arpbind.asp new-qoslimit.asp qos-classify.asp qos-detailed.asp qos-graphs.asp qos-settings.asp reboot.asp reboot-default.asp restrict.asp restrict-edit.asp saved.asp saved-moved.asp shutdown.asp status-devices.asp status-log.asp status-overview.asp tools-ping.asp tools-shell.asp tools-survey.asp tools-trace.asp tools-wol.asp bwm-graph.svg qos-graph.svg; do \
    		sed	-e "/REMOVE-BEGIN/,/REMOVE-END/d" \
    			-e "s,<script[^>]\+debug\.js[^>]\+></script>,," \
    			-e "s,color\.css,<% nv('web_css'); %>\.css," \
    			-e "s,<!-- / / / -->,," \
    			-e "s,\x0d,," \
    			-e "s,^\s\+,," \
    			-e "/^$/d" \
    			-e "/^<!--$/,/^-->$/! { s,^\s\+, , }" $F > /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/www/www/$F; \
    chmod 0644 /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/www/www/*
    for F in *.{js,jsx}; do \
    		/home/bcalab/tomato/release/src/router/www/ /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/www/www/$F c; \
    Warning: unknown mime-type for " /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/www/www/*.{js,jsx} does not exist" -- using "application/*"
    Error: no such file " /home/bcalab/tomato/release/src/router/mipsel-uclibc/install/www/www/*.{js,jsx} does not exist"
    make[2]: *** [install] Error 2
    make[2]: Leaving directory `/home/bcalab/tomato/release/src/router/www'
    make[1]: *** [www-install] Error 2
    make[1]: Leaving directory `/home/bcalab/tomato/release/src/router'
    make: *** [all] Error 2
  13. mstombs

    mstombs Network Guru Member

    That one's a dash/bash problem, can be fixed by editing the www/Makefile last few lines to read

    # remove C-style comments from java files. All "control" comments have been processed by now.
        for F in $(wildcard *.js *jsx); do \
            $(TOP)/www/ $(INSTALLDIR)/www/$$F c; \
  14. Toastman

    Toastman Super Moderator Staff Member Member

    Ok this one is easy.

    The last bit

    "for F in *.{js,jsx}; do " is the problem. If you look upwards a little - copy the earlier line instead

    for F in $(wildcard *.js *.jsx); do \

    I think it's cracked now :grin:
  15. Toastman

    Toastman Super Moderator Staff Member Member

    haha we did it again ...
  16. mstombs

    mstombs Network Guru Member

    HeHe, you had 45 mins to get there first This must also be fixed in git, but clearly not an issue on Original PC Jon uses for the old 'original tomato'
  17. Toastman

    Toastman Super Moderator Staff Member Member

    Michael, I remember something about certain files not being "stripped" (??) in earlier code if not compiled in /root. It made a difference to the ultimate build size if I remember correctly. I think it was you that raised this last year.

    It may be useful for onehomelist as he's not compiling in root.
  18. Toastman

    Toastman Super Moderator Staff Member Member

    Okay, if that worked, and you want to use the repository, the next step is to install git - can do from Helena. Follow instructions from SGTPepper to use it initially. Install also gitk and gitgui which will make your life a lot easier. Also useful is smartgit - you'll have to download this and start with a script. This one does things the others don't.

    This is assuming you would like to use the GUI rather than stick to command line. The GUI's make some things much faster and keep you better informed as to what is going on.

    I'll be following in your footsteps shortly, after screwing up my hard disk by hitting the wrong place in a partitioner, I have to re-install.
  19. onehomelist

    onehomelist Addicted to LI Member

    I solved the x86_64 issue. I work at a place where I am given freedom to use the excellent computer laboratory. Because of time constraints [lab timings] I couldn't try out your solution to my second error. I will do it soon.I never dreamt that I would get solutions to compiling errors so quickly from you. Thanks guys. I will get back to you pretty soon.
  20. mstombs

    mstombs Network Guru Member

    There was an issue with router executables not being fully stripped if the build not run as user root. This was in a specific teddy_bear git release - I assume it was fixed? A solution was to strip the files before installing into the temporary filesystem with new owner root. It is not good practise to build firmwares logged in as root, get a symlink wrong and you can find the build script overwrites your PC binaries/libraries with PC-incompatible mipsel versions which makes it hard to boot your PC (BTDTGTTS).

    Because the compressed kernel plus filesystem contain compressed time/date stamps (and sometimes references to your PC username and filesystem) your self compiled firmware is never binary identical to the published ones even if you have made no changes, but the size should be very close!
  21. a9988cd

    a9988cd LI Guru Member

    please try change this.

    unlink /bin/sh
    ln -sf /bin/bash /bin/sh
  22. Toastman

    Toastman Super Moderator Staff Member Member

    BTW, this isn't Ubuntu, it's Linux Mint 8, and the shell is bash. And when I installed Ubuntu 9.10, the shell was also bash by default. Made it confusing actually, because we were used to thinking Ubuntu used dash by default.
  23. onehomelist

    onehomelist Addicted to LI Member

    I used this solution, and it worked. I compiled it successfully.

    for F in $(wildcard *.js *.jsx); do \ 
    Now I am working on using git repository.

    I want to ask a few questions, just incase anyone knows.

    Which makefiles should be modified while adding a software package?

    Can anyone give some tips for adding or modifying tomato WEB GUI.

    If there are any wikis or guides for modifying the source, even if they are about other distributions like DDWRT or openwrt, please post the links.
  24. mstombs

    mstombs Network Guru Member

    A great tutorial on writing programs for the same hardware platform, but not tomato, is here

    For the rest you are on your own - but I recommend you set up a directory in tomato/release/src/router, have a look how miniupnpd is incorporated for example - it is a relatively late addition.
  25. onehomelist

    onehomelist Addicted to LI Member

    I used this command for compiling
     make V1=9050 V2=MIPSR2-beta20 r2m
    I get the following error while compiling beta 20 k26

    /tomato/release/src/router/busybox/scripts/ line 11: /opt/brcm/hn                                                                              dtools-mipsel-uclibc/bin/mipsel-uclibc-gcc: No such file or directory
    make[5]: Leaving directory `/tomato/release/src/router/busybox'
    make[4]: Leaving directory `/tomato/release/src/router'
    make[3]: Leaving directory `/tomato/release/src-rt'
    make[3]: Entering directory `/tomato/release/src-rt'
    Version: 1.28.9050 MIPSR2MIPSR2-beta20 K26 USB Ext (Mon, 06 Sep 2010 11:23:35 +0                                                                              530)
    Building Tomato K26 USB 1.28.9050MIPSR2MIPSR2-beta20 Ext M with N Profile
    mkdir: cannot create directory `image': File exists
    make[3]: [all] Error 1 (ignored)
    make[4]: Entering directory `/tomato/release/src/router'
    rm -rf /tomato/release/src-rt/router/mipsel-uclibc/target
    rm -rf /tomato/release/src-rt/router/mipsel-uclibc/install
    rm -f /tomato/release/src-rt/router/mipsel-uclibc/linux.trx /tomato/release/src-                                                                              rt/router/mipsel-uclibc/vmlinuz /tomato/release/src-rt/router/mipsel-uclibc/targ                                                                              et.image
    rm -rf /tomato/release/src-rt/router/mipsel-uclibc/extras
    make -C /tomato/release/src-rt/lzma-loader CROSS_COMPILE=mipsel-uclibc- LD=mipse                                                                              l-uclibc-ld
    make[5]: Entering directory `/tomato/release/src/lzma-loader'
    mipsel-uclibc-gcc -D__KERNEL__ -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fno                                                                              -strict-aliasing -fno-common -fomit-frame-pointer -G 0 -mno-abicalls -fno-pic -f                                                                              function-sections -pipe -Wa,--trap -mabi=32 -march=mips32 -Wa,-32 -Wa,-march=mip                                                                              s32 -Wa,-mips32 -DLOADADDR=0x80001000 -D_LZMA_IN_CB -I/tomato/release/src-rt/inc                                                                              lude -I/tomato/release/src-rt/lzma/C/Compress/Lzma -D__ASSEMBLY__ -DBZ_TEXT_STAR                                                                              T=0x80500000   -c -o head.o head.S
    make[5]: mipsel-uclibc-gcc: Command not found
    make[5]: *** [head.o] Error 127
    make[5]: Leaving directory `/tomato/release/src/lzma-loader'
    make[4]: *** [lzma-loader] Error 2
    make[4]: Leaving directory `/tomato/release/src/router'
    make[3]: *** [all] Error 2
    make[3]: Leaving directory `/tomato/release/src-rt'
    make[2]: *** [bin] Error 2
    make[2]: Leaving directory `/tomato/release/src-rt'
    make[1]: *** [m] Error 2
    make[1]: Leaving directory `/tomato/release/src-rt'
    make: *** [r2m] Error 2
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice