NGinx (as a proxy) for accessing the webgui (for added functionalities)

Discussion in 'Tomato Firmware' started by MatteoV, Jan 12, 2014.

  1. MatteoV

    MatteoV Networkin' Nut Member

    Hi all.
    I wanted my Tomato's webgui to answer with a reliable certificate made for it by my CA, together with all its certificate Chain. This is impossible with the classic httpd.
    So, I thought about using nginx to do just the proxy to the internal https version of the webgui.

    Here it is how to do it, for those interested:
    Administration => Admin Access => Local Access = HTTPS; HTTPS PORT = 400. Save.
    Web Server => tick Keep Config Files. Save.
    USB Support => Run after mounting => Write:
    #Executable scripts
    chmod +x /tmp/mnt/sda1/scripts/*
    /tmp/mnt/sda1/scripts/ &
    Connect with WinSCP and add script :
    #Script settings
    #Delete old certs
    rm -r "$RESIDENTNGINX"
    killall nginx
    nginx &
    In your flash drive, generate the paths written, in particular /tmp/etc/nginx/nginx.conf with this content:
    # NGinX generated config file
    user                                root;
    worker_processes                    1;
    worker_cpu_affinity                    0101;
    master_process                        off;
    worker_priority                        10;
    error_log                            /tmp/var/log/nginx/error.log;
    pid                                    /tmp/var/run/;
    worker_rlimit_nofile                8192;
    events {
        worker_connections                512;
    http {
        include                            /tmp/etc/nginx/mime.types;
        #include                        /tmp/etc/nginx/fastcgi.conf;
        default_type                    application/octet-stream;
        log_format                        main '$remote_addr - $remote_user [$time_local]  $status '
        '"$request" $body_bytes_sent "$http_referer" '
        '"$http_user_agent" "$http_x_forwarded_for"';
        sendfile                        on;
        server_tokens                    off;
        server {
            add_header                    Strict-Transport-Security "max-age=31536000; includeSubDomains";
            listen                        router-internal-ip:443 ssl;
            server_name                    ROUTER
            keepalive_timeout            70;
            ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
            ssl_ciphers AES128-SHA:AES256-SHA;
            ssl_trusted_certificate        /tmp/etc/cert.pem;
            ssl_certificate                /tmp/etc/cert.pem;
            ssl_certificate_key            /tmp/etc/key.pem;
            ssl_session_cache            shared:SSL:10m;
            ssl_session_timeout            10m;
            access_log                    /tmp/var/log/nginx/access.log    main;
            location                /    {
                proxy_pass               https://router-internal-ip:400;
                proxy_set_header        Host $host;
                proxy_set_header        X-Real-IP $remote_addr;
                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                client_max_body_size    10m;
                client_body_buffer_size    128k;
                proxy_connect_timeout    90;
                proxy_send_timeout        90;
                proxy_read_timeout        90;
                proxy_buffers            32 4k;
                 location ~* \.(sql|htaccess|htpasswd|tpl|html5|xhtml) {
                    deny all;
    Notice the key.pem and cert.pem. The first is the private key, the latter the certificate chain you generated: server certificate, intermediates, root certificate in this order. This way if you trusted just your own root CA the webgui https://router greens the browser's bar.
    Take note of the fact every path seems internal but has been symlinked to the external storage mount script.

    But, this is the point: I wrote this thread because I'd like some security advices. I feel like this beast could dangerously listen everywhere and/or have security implications I can't see. I get proxying could potentially kill some protection. Ok, it listens on the internal network, but is this enough?


    Added sslstrip protection.
    Last edited: Jan 13, 2014
  2. koitsu

    koitsu Network Guru Member

    Couple things:

    1. Don't use killall please. Please use kill `pidof nginx` or kill `cat /var/run/` (if that's where nginx stores its PID file); the latter method is best. killall is a bad habit to get into. I've covered this in some past posts.

    2. You shouldn't start nginx like nginx & if you can avoid this. This is an extremely common mistake people do when they think they can just "background a process" on a non-controlling terminal. & does not do the same thing as some other programs (like daemon on FreeBSD, and nohup most everywhere else, although on Busybox you'd need to use nohup {command} 1>/dev/null to keep the nohup.out file from being created). Doesn't nginx fork itself and run in the background anyway? If so, remove the &.

    For example, for many years FreeBSD's mysqld port would launch mysqld using mysqld_safe (which is normal), except the rc script author thought using & would be sufficient -- wrong. The problem is that stdin/stdout/stderr being kept open caused a pty/tty to be kept open/wasted. I reported this problem some time ago, and it did get addressed:

    The solution was to use daemon, which properly detaches everything in advance before fork/exec'ing something:

    Moral of the story is: using & is not the same thing as a real/true daemon.

    Footnote: in some situations/on some OSes, you can do the following to close all the fds in advance (and hope it doesn't keep a pty/tty attached):

    nohup {command} 1>/dev/null 2>/dev/null < /dev/null

    Which essentially redirects stdout and stderr to /dev/null, and also (hopefully) feeds stdin from /dev/null. This is why commands to look at fds/etc. on a *IX system (the common one for that on Linux is lsof) is useful.
    Last edited: Jan 13, 2014
    MatteoV likes this.
  3. MatteoV

    MatteoV Networkin' Nut Member

    First of all, let me thank you @koitsu, as usual your posts are very valuable.
    You are right. I did it that way in all the other works. The fact is this script runs when the usb drive is mounted (every data is there). I expect to have no nginx running at that time because I disabled the "start on boot" option of the integrated system. I did this because otherwise a nginx will start not knowing my own configuration (old nginx.conf file in the rom) and that's plain wrong. In short the killall should have no effect at all. I will make it kill "$(pidof nginx)" anyway, so it's ok, just in case something weird happens.

    Yes! nginx just starts in the background by itself! It was just a fail-safe "&", thinking exactly what you describe as an error, i.e. that means simply "go run in the background". It was my fault so many times, so. There's always much to learn from you, that's great!
    p.s. I tried starting it as a service with the service command too, but that does not work. Is it the same as daemon you are describing? If yes, how should one "teach" a new service to it?

    p.p.s. for those following my idea/instructions. After reading this post from @Mangix I changed the config file adding:
        server {
            add_header                    Strict-Transport-Security "max-age=31536000; includeSubDomains";
    to avoid sslstrip attacks.
    Also, I should investigate ciphers a bit more and be back.

    Thanks everyone!
  4. koitsu

    koitsu Network Guru Member

    service is just a "wrapper" script on most OSes that lets you control startup/shutdown/whatever else via a common syntax/means. Usually you have to drop scripts and/or symlinks into the appropriate init structure (ex. /etc/init.d or /etc/rc.d or /etc/rcX.d), and it varies per OS (and on Linux also per distribution).

    Tomato doesn't have anything like that. So the best you can do in this case is to just run nginx and cross your fingers. The user will need to know to go look at the nginx error log (or whatever the equivalent is; I'm an Apache person, not nginx) manually to determine source of problems and so on.

    There's a lot of startup framework that doesn't exist on most residential router distros given space concerns and so on. The one exception is OpenWRT, where theirs is quite unique/different from everyone else, but it works. I always tell people: if you want a router that you can treat more like an actual Linux system, use OpenWRT.
    MatteoV likes this.
  5. MatteoV

    MatteoV Networkin' Nut Member

    Ok, that clears everything up. I thought about the service wrapper because it seems there are actually some services pre-set in Tomato, for example service httpd restart is a command I use in another script and it works for sure (for changing certificates of the httpd daemon).
    Anyway, nginx just works ok, plus a cronjob checking for it and eventually restarting it ;)

    Thanks for your help as usual.
  6. koitsu

    koitsu Network Guru Member

    Here's the source to /sbin/services (which is a symlink to /sbin/rc) if you care to dig through it. It's 76KBytes. Description is below, in a tiny font to keep my post shorter (screen real estate-wise)

    service_main() is what gets called when using the service command. Once you start looking at do_service() you'll start getting confused -- in other words, "uh, what? How does this actually restart/do anything?"

    The way it works is an introduction to embedded devices and how a lot of the time crap is just thrown together in a jumbled mess: the NVRAM variable called action_service is assigned a value of something like httpd-restart-c (in the case you're calling service httpd restart) -- the -c is not a typo (this is later called a "modifier" when using exec_service() (keep reading)), and indicates that the action was issued through the service command itself). It then proceeds to send a SIGUSR1 to init (PID 1) along with some cosmetic crap (printing asterisks, dots, etc.); and all this assumes you do not have NVRAM variable debug_rc_svc set to 1.

    The code to init has a SIGUSR1 signal handler -- look around line 1678. It calls exec_service() then breaks out of the signal handler switch(), followed by calling the code from line 1760 onward.

    exec_service() is in services.c around line 2314. This is the code which reads the NVRAM variable action_service and proceeds to parse it, restarting daemons and doing other whatnots depending on what service name you gave it. For example, for httpd see line 2454.

    What's not immediately clear to the reader is how restart works. Look around line 2316 for some hints. The action variable is simply a bit field; bit 0 defines whether or not you want to start a daemon, bit 1 defines if you want to stop a daemon, and bits 0 + 1 combined (when both set, ex. 1|2) get handled in the order of stop, followed by start. Thus the if() A_STOP check and A_START conditions would both prove true for httpd, hence a restart.

    Daemon restarting/etc. is handled like this as well. A lot of the time in Tomato, when you click "Save", this type of thing goes on under the hood -- many, many daemons are fully stopped and restarted. This is all by design.

    You'll see all of this code, clunky and horrible, and ask yourself "why was it done this way? My god, a true init + rc.d or init.d would make all of this a lot saner!" Yes it would -- except again, this is an embedded environment, and shell scripts as flat files on a filesystem take up precious room.

    Did it have to be designed this way? Absolutely not. CyberTAN, the Taiwanese company who Linksys used to (and possibly still does?) contract out their work to, has historically written absolutely awful code. This framework, rather than a real init and related rc or init scripts, is living proof.

    This is one of the many areas where OpenWRT is a blessing: all of this nonsense has been scrapped and put into something sane. It's why, when some developer eventually gets the desire and has the time, Tomato's UI being "ported" or "migrated" over to OpenWRT would revolutionise the consumer router industry: you'd have a great UI with a clean back-end (heh that sounds naughty) and an excellent base infrastructure for doing what you need. No more of CyberTAN's code. I've been hating on CyberTAN for this garbage since I first saw it back when the the WRT54GL was released. It shocked me then and it still shocks me now. I'm certain OpenWRT was created some time ago solely because folks came across code like this and said "are you kidding me?!?"
    MatteoV likes this.
  7. MatteoV

    MatteoV Networkin' Nut Member

    Thank you! I will investigate it a bit more ASAP. I'm out of home and having really bad connections, remote control are really bad too.
    Ps just discovered nginx needs another folder to work, which is not created by Tomato if you disable autostart as I suggested. The folder is /tmp/var/lib/nginx. Just add it to the initial script , otherwise the server will not start at all.

    Inviato dal mio Nexus 4 utilizzando Tapatalk
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice