1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Mesh VPN Support (tinc)

Discussion in 'Tomato Firmware' started by rafwes, Dec 10, 2012.

  1. rafwes

    rafwes Serious Server Member

    I've been using tinc in the last months to connect roadrunners and branch offices together and it is just great. I would love to see this supported in Tomato. How do you feel about having tinc integrated into Tomato? Anybody up to the task?
     
  2. lancethepants

    lancethepants Network Guru Member

    That would be a cool feature to have in the gui. Hopefully nvram space wouldn't be too much of an issue, since you'd have to store a public key for each 'ConnectTo' statement.

    Just FYI, and perhaps you already know, you can run tinc now with tomato firmware. I've compiled a static binary available at http://lancethepants.com/files. I've placed it in /jffs along with my config and keys, beacuse I don't want to be dependant on an external device like a USB drive.

    Entware also more recently added the tinc package, so you can grab it from there too.
    So no gui, but you can run it, and I have been, and perhaps you already do too.
     
  3. rafwes

    rafwes Serious Server Member

    I am in fact using your binary to run tinc on some E2500 with Tomato. Like you, I have it placed on /jffs, since there is no other option due to the lack of USB ports on this device. The only problem is the size, making it hard to fit into routers with smaller flash size. Thank you for providing them. Given the fact that OpenVPN uses lots of NVRAM space by itself, I think typical tinc setups with some ConnectTo statements with standard key sizes would be no problem at all.

    Since I have little experience with Entware/Optware, can you tell me what is the differences with these binaries that make them smaller and if I could use them without having packaging systems. I for instance use no compression to free resources on the router and therefore do not need any of these libraries. Could you provide a stripped down binary for putting into my old WRT54G? Would it work?
     
  4. lancethepants

    lancethepants Network Guru Member

    With my static binary, it has openssl, lzo and zlib all packaged into one. The Entware and Optware binaries are smaller because those libraries are included separately, so you really have to add up all the libraries and the binary to see how much space they would take, maybe less. Entware binaries may also use a better compression. He actually has referred me to it, but I haven't yet played around with it. It may not be possible to fit into /jffs on routers with less than 8MB. What's your target size, and I can see if it's possible. Try to find the smallest firmware possible for the device that will allow the most jffs space.

    I have started to attempt at creating a firmware with tinc, but have it a wall right now. I also to get it to fit into a WRT54GL, but just ended up buying routers with more flash.
     
  5. rafwes

    rafwes Serious Server Member

    Can't tell you exactly right now, since I have no free WRT54Gs to test, they are being used as VLAN switches throughout the network. Soon I'll be replacing them against new gigabit switches, so I thought about using them as tinc gateways for remote locations.

    My goal is to fit tinc into WRT54Gs running 2.4 kernel and if needed with stripped out compression. Shibby will only give about 600k jffs, that would be my preferred choice, but your binary won't fit here. Smallest DD-WRT with jffs and openvpn (for modprobe tun) has 3.11 MB, that could do the trick. I may also try openwrt, both with your binary and their tinc package. Let's see.

    What were your problems when trying to create a firmware?
     
  6. lancethepants

    lancethepants Network Guru Member

    Ok, I figure out how to compress the binary even further. It was actually much easier than I expected. UPX 3.08 has a bug with mipsel binaries, which threw my off for a while. I had to use 3.07 to get it working.

    I've uploaded the file to the site. It reduced the binary from 864K to 392K, so less than half the size (.45 ratio).

    I was attempting to compile the binary, and merely place it in the firmware image. I've done this with other binaries, but this one wouldn't go, something about compression no less :). I got into other projects and haven't looked back into it yet.
     
  7. rafwes

    rafwes Serious Server Member

    Nice! I'll try it as soon as I can and give you a feedback. Thanks.
     
  8. rafwes

    rafwes Serious Server Member

    Hi Lance!

    Any chance you could provide a binary compiled with the new OpenSSL (Assembler Optimizations). Shibby said he would take a look at tinc but so far nothing happened.
     
  9. lancethepants

    lancethepants Network Guru Member

    I've uploaded a new version of tinc to my site (The new 1.0.20). I few noteworthy tidbits about this build.

    Generally I compile my binaries using the entware toolchain. For simplicity and quickness this time, I just compiled Shibby's latest firmware with the new OpenSSL assembly optimizations. This also means I had to compile the whole binary with the tomato toolchain, which should be fine.

    I also compiled it using the 'O3' compiler option instead of 'Os'. In another thread, the entware repo maintainer said 'O3' gives better performance.

    I've also compressed the binary using the latest version of UPX, so it's nice and small.

    If you don't mind, could you benchmark this binary to the one you're currently using? I'd like to see what kind of results it gives.
    thanks.
     
  10. rafwes

    rafwes Serious Server Member

    Sorry for the late response, had to wait until my vacation to play with this. I was expecting to see more improvement on the non-compressed stream, but lzo upstream speeds came to me as a surprise. Here are the results of some preliminary tests (more reliable ones to come):
    Code:
    OpenWRT Backfire on WRT54GL
    ===== 1.0.19 w/o optimizations =======
     
    H.264 Video:      LZO          no-LZO
    Downstream      4.07 Mbit/s    4.25 Mbit/s
    Upstream        3.45 Mbit/s    4.75 Mbit/s
     
    Iperf:
    Downstream      7.00 Mbit/s    4.25 Mbit/s
    Upstream        6.05 Mbit/s    4.75 Mbit/s
     
    ===== 1.0.20 with optimizations ======
     
    H.264 Video:      LZO          no-LZO       
    Downstream      4.17 Mbit/s    4.37 Mbit/s
    Upstream        4.43 Mbit/s    4.87 Mbit/s
     
    Iperf:
    Downstream      7.16 Mbit/s    4.37 Mbit/s
    Upstream        6.15 Mbit/s    4.87 Mbit/s
     
  11. lancethepants

    lancethepants Network Guru Member

    Interesting results. So is the "1.0.20 with optimizations" my binary running on jffs on a WRT54GL?

    It appears that trying to compress an already compressed stream (H.264) has drawbacks and actually shrinks the throughput. Compression helps the Iperf numbers quite a bit though. Have you benched against zlib compression at all? Very nice numbers, can't wait to see more of your results.
     
  12. rafwes

    rafwes Serious Server Member

    Both are your binaries. It looks like the new compiler impacts lzo performance way more than the openssl optimizations. I was quite surprised by it. Since the drawback seems to be small, I am even considering turning it on.

    On both runs I was using iperf, the only difference was to use a video file as source instead of iperf's standard generated data. If I find the time I'll retest them locally, since the results above are from a EC2 instance and a WRT54GL. I do not plan to test zlib, because it requires more computing power than lzo, but if you are interested please tell me which level would you like to see tested.
     
  13. rafwes

    rafwes Serious Server Member

    Here is the new test batch. This time the connection was local, tinc run over just 1 hop after masquerading. Since the clients were reporting different speeds, I only took the speed reported by the server (receiving end). I repeated each test 3-5 times and took a eyesight mean value. Greatest improvement is still on LZO upstream.

    Code:
    ===== 1.0.19 w/o optimizations =======
     
    H.264 Video:        LZO         no-LZO
    Downstream      4.14 Mbit/s    4.39 Mbit/s
    Upstream        3.15 Mbit/s    4.34 Mbit/s
     
    Iperf:
    Downstream      6.05 Mbit/s    4.39 Mbit/s
    Upstream        4.97 Mbit/s    4.34 Mbit/s
     
    ===== 1.0.20 with optimizations ======
     
    H.264 Video:        LZO         no-LZO           
    Downstream    4.26 Mbit/s    4.53 Mbit/s
    Upstream      3.97 Mbit/s    4.41 Mbit/s
     
    Iperf:
    Downstream    6.20 Mbit/s    4.53 Mbit/s
    Upstream      5.42 Mbit/s    4.41 Mbit/s
    
     
  14. lancethepants

    lancethepants Network Guru Member

    Thanks for the results. Just curious, are you using compression level 10 or 11? 1.0.20 appears to beat out 1.0.19 in every case, nice to know that it really the fastest binary currently. Perhaps on the next release of tinc I'll have to create two binaries with both entware and tomatousb toolchains for a better apples to apples comparison, just in case that skews the numbers at all (I wouldn't suspect much though I think).
     
  15. rafwes

    rafwes Serious Server Member

    Because of its computational complexity for encoding, I tried to asymmetrically use level 11 on the EC2 and roadrunners leaving the routers with 10 or no compression at all, but the results weren't good, so I decided just to use 10. Thanks again for providing the binaries!
     
  16. winter2

    winter2 Reformed Router Member

    Hi, lancethepants,

    I'd been using you binary 1.0.19 & 1.0.20 on dd-wrt and it's been great! But I encountered a problem when I try to upgrade it to you 1.0.21 binary, it doesn't start on dd-wrt sp1 14929 with ASUS wl-520 (with error msg "Illeagal Instruction"), while works fine with dd-wrt v24 sp2 17598 on another device.​
    version 1.0.19 & 1.0.20 are all fine, do you have any clue on why that happened?​

    I'd say your work is great and ease my life! Thanks for it!

    Jeremy
     
  17. lancethepants

    lancethepants Network Guru Member

    Check out the new binary uploaded, and let me know if it works.
     
  18. Waester

    Waester Reformed Router Member

    The new binary works and great job with the size. You almost halved it :D
     
  19. Waester

    Waester Reformed Router Member

    I noticed that I too get illegal instructions depending on how I upload the file(s), using FileZilla it happens more often.

    This happens both to tinc and dnscrypt

    EDIT: Oh double post, sorry, missed that I posted earlier
     
  20. lancethepants

    lancethepants Network Guru Member


    Awesome! This time I kept track of how I compiled it for future compilations, so hopefully we don't have any more incompatibilities. You know where to find me if there were to be though :)
     
  21. winter2

    winter2 Reformed Router Member


    Great job!
    This new build works perfectly on my dd-wrt 14929 & 17598, and dramatically small. Thank you very much for the work!
     

Share This Page