1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Tomato: automatic router synchronization using remote script execution

Discussion in 'Tomato Firmware' started by skyanvi1, Mar 1, 2012.

  1. skyanvi1

    skyanvi1 Addicted to LI Member

    Tomato: automatic router synchronization using remote script/remote command execution

    This is my contribution, I hope this helps someone out there attempting to automate remote execution of scripts between linux systems (in this case both systems are routers, however the scripts should translate to other linux systems.).

    I am running a dual wan setup with two RT-N16’s running `Tomato v1.28.7495 MIPSR2-Toastman-RT K26 USB VPN` (Thank you, Toastman). This is for both failover and performance ( I have found that automated failover has lead to more problems than it ever solved and is now a moot point since I can vpn in using my linux phone to either of the two routers to set things right). Of course the standard disclaimer applies… you can brick it… so I’ve heard… however I have 8 of these linux routers and the only one I have bricked so far is from plugging a 12v adaptor into a 5v router (…what can I say it was early, no coffee, the RT-N16 and WL-500g look similar… I digress.)
    This setup uses password-less ssh between a master and slave router. The scripts are stored on the jffs mount on the Master router. Synchronized settings are chosen in the script using REGEX (run away while you still can…). The primary script dynamically generates a secondary script, to be executed on the slave router, containing all the settings (a bunch of escaped nvram set commands). It is saved in the temporary file system of the Master router. The md5sum of the dynamic script is compared with the previously generated dynamic script md5sum, if they differ the settings are synchronized otherwise the script is finished. This prevents hammering the slave with data and unnecessary nvram commits. If the md5sum is different the dynamic script is transferred by password-less scp to the Slave router. The script then uses ssh to execute the synchronization script on the remote slave router.
    Synchronization is scheduled using cron (cru) from the scripts section of the administration web interface. Instructions on how to setup ssh/scp password-less login are in the comments of the routerSync.sh script.
    There are two scripts one to generate the dynamic synchronization script and another to execute commands and transfer files remotely (the remote execute script can be used standalone). I place the following at the top of my scripts to facilitate pasting directly into putty:
    Code:
    scrName=/jffs/scr/scriptName.sh
    echo "" > $scrName
    vi $scrName
    :set noautoindent
    i#!/bin/sh
    #<script>
    # press Esc and type `:wq` and press Enter.
    # chmod 755 scriptName.sh to allow execution
    

    remotexe.sh
    Code:
    scrName=/jffs/scr/remotexe.sh
    echo "" > $scrName
    vi $scrName
    :set noautoindent
    i#!/bin/sh
    # remotexe.sh
    # Remote execution of scripts and commands with arguments using scp and ssh
    # skyanvi1
    # ver: 0.11
    # usage: remotexe.sh -debug -u <username> -h <hostAddress> -i <privateKeyPath -c <command> <command arg1> <command arg2> <command arg3...>
    # usage: remotexe.sh -debug -verbose -tolog -u [remoteuser] -h [remotehost] -i [privatekeypath] [ -c [remoteCmd arg1 arg2 ...] OR [ -s [srcPathToScript] -d [destPathToScript arg1 arg2 ...] ] ]
    # ex: /jffs/remotexe.sh -debug -u root -h 192.168.1.2 -i /jffs/keys/privateKey.txt -s /tmp/bscr/sync.sh -d /tmp/bscr/sync.sh arg1 arg2
    # ex: /jffs/remotexe.sh -debug -u root -h 192.168.1.2 -i /jffs/keys/privateKey.txt -c mkdir /tmp/anewfldr
     
    # Assumes password-less login is setup.
    # Router KeyGen Command: 
    # ex: dropbearkey -t rsa -f privateKey.txt -s 2048 > publicKey.txt
     
    # exit script on use of uninitialised variable
    set -u
    # exit script on any statement return a non-true return value:
    #set -o errexit
     
    BNAME=`basename $0`
    RFLDR=`dirname $0`
    SCRID=$$.$BNAME
     
    # verbose output to the console
    VRBOSE=1
     
    # log messages
    LOGMSGS=1
     
    usage()
    {
        echo "USAGE: $0 -debug -verbose -tolog -u [remoteuser] -h [remotehost] -i [privatekeypath] [ -c [remoteCmd arg1 arg2 ...] OR [ -s [srcPathToScript] -d [destPathToScript arg1 arg2 ...] ] ]"
        echo "use full paths for everything if using cron."
        exit
    }
     
    # controls where messages are output silent, log or console
    echoGate()
    {
        if [ $VRBOSE -eq "0" ]; then
            if [ $LOGMSGS -eq "0" ]; then
                logger $SCRID ": $*"
            else
                echo "$*"
            fi
        fi
    }
    # get the directory from a file path string
    dirname()
    {
        _dirname "$1" && printf "%s\n" "$_DIRNAME"
    }
     
    # get the directory from a file path string
    _dirname()
    {
        _DIRNAME=$1;
        strip_trailing_slashes;
        case $_DIRNAME in
            "")
                _DIRNAME='/'
            return
            ;;
            */*)
                _DIRNAME="${_DIRNAME%/*}"
            ;;
            *)
                _DIRNAME='.'
            ;;
        esac;
        strip_trailing_slashes;
    }
     
    strip_trailing_slashes ()
    {
        while [ "${_DIRNAME%/}" != "$_DIRNAME" ]; do
            _DIRNAME=${_DIRNAME%/};
        done
    }
     
     
    # these variables are used to track
    # the position of the parameters so
    #they can be escaped and replicated
    #on the remote system
     
    cmdNdx=-1
    scrNdx=-1
    dstNdx=-1
     
    cmdCt=-1
     
     
    p2=$0
    # iterate through the arguments setting all the flags
    for p in $*;
    do   
        #echo $p2
        case x$p2 in
        x-verbose)
            VRBOSE=0       
        ;;
        x-u)       
            user=$p
        ;;
        x-h)       
            host=$p
        ;;
        x-i)       
            priKey=$p
        ;;
        x-c)
            cmdNdx=$cmdCt
        ;;
        x-s)
            scrNdx=$cmdCt
            scrPath=$p
        ;;   
        x-d)
            dstNdx=$cmdCt
            dstPath=$p
        ;;   
        x-tolog)
            LOGMSGS=0
        ;;
        x--h)
            usage
        ;;
        x-debug)
            VRBOSE=1
            echoGate debug mode
            set -x
        ;;
        esac
       
        # increment the index
        cmdCt=`expr $cmdCt + 1`
       
        # Assign the next parameter
        p2=$p
    done;
     
     
    echoGate $SCRID starting ...
     
    # BEGIN SCRIPT
     
    CMD=""
     
    # are we sending a source script to be executed remotely?
    if [ $scrNdx -gt -1 ]; then
        if [ $dstNdx -gt -1 ]; then
                   
            # set execute permissions before sending file     
            chmod 755 $scrPath
     
            # copy the file to the remote machine
            echoGate "copying file to remote machine..."
            scp -i $priKey $scrPath $user@$host:$dstPath
            scpExit=$?
           
            if [ $scpExit -eq 0 ];then
              echoGate "success.!"
            else
                #set -x
                echoGate "fali... attempting to create directory..."
                CMD=`dirname $scrPath`           
                CMD="$(echo "mkdir -p $CMD")"
               
                # use recursion and make the directory
                # calling this script from within iteslf to make the directory
                $0 -u $user -h $host -i $priKey -c $CMD           
                # sleep so the remote router is not getting hammered with ssh requests.
                sleep 1
               
                CMD=""
               
                echoGate "resending file..."
                scp -i $priKey $scrPath $user@$host:$dstPath
                scpExit=$?
                if [ $scpExit -eq 0 ];then
                  echoGate  "success.!"
                else
                    echoGate  "fail.!"
                    exit
                fi
            fi
     
            sleep 1
         
            # update the command index
            cmdNdx=$dstNdx
        fi   
    fi
     
    # initialize the argument counter
    iCt=0
     
    # loop through all the arguments and escape them.
    for ARG in $*;
    do   
        if [ $iCt -gt $cmdNdx ]; then     
            CMD="$CMD $(echo "$ARG" | awk '{gsub(".", "\\\\&");print}')"
        fi
        iCt=`expr $iCt + 1`
    done;
     
    echoGate "running remote command... $CMD"
     
    # ssh into the remote machine and execute the command/or script.
    ssh $user@$host -i $priKey "$CMD"
     
    echoGate "done."
    # if using vi press Esc and type `:wq` press Enter.
    # chmod 755 scriptName.sh to allow execution
    

    routerSync.sh
    Code for the dynamic generation of the synchronization script. Edit the variables in this script to change the settings to synchronize. Setup instructions are part of the comments.
    Code:
    scrName=/jffs/scr/routerSync.sh
    echo "" > $scrName
    vi $scrName
    :set noautoindent
    i#!/bin/sh
    #set -x
    # routerSync.sh
    # Tomato router synchronization script
    # ver. 0.11
    # skyanvi1
    # usage: routerSync.sh -autosync
    # chmod 755 routerSync.sh to allow execution
     
    # Assumes password-less ssh is setup between the master and slave router
    # requires the remote run script: remotexe.sh
    # use full paths.
    # Instructions:
    # 1. Setup router ssh services
    # enable SSH Daemon on MASTER and SLAVE router (remote access: false, forwarding: false, save.)
    # 2. setup MASTER router filesystem
    # enable the jffs partition on MASTER router:
    # Administration : JFFS, Enable, Save, Format/Erase
     
    #  ssh to MASTER router ( putty ) and create directories:
    # root@tomato:/jffs/# mkdir -p /jffs/keys
    # root@tomato:/jffs/# mkdir -p /jffs/scr
    # root@tomato:/jffs/# cd /jffs/keys
    # 3. configure password-less login:
    # Generate ssk keys with Router KeyGen Command: 
    # root@tomato:/jffs/keys# dropbearkey -t rsa -f privateKey.txt -s 2048 > publicKey.txt
    # place the private key on the MASTER router-> (ssh) /jffs/keys/privateKey.txt
    # place the public key on the SLAVE Router-> (web interface) Administration:Admin Access:Authorized Keys
    # root@tomato:/jffs/# cat /jffs/keys/publicKey.txt
    # copy and paste public key
     
    # login to the slave router with the following command:
    # root@tomato:/jffs/# ssh root@Address -i /jffs/keys/privateKey.txt
    # Answer yes to saving the key in the known_hosts file.
    # exit ssh ( type exit to return to the MASTER router)
    # copy the MASTER known_hosts file to the MASTER /jffs/keys folder (since ~/.ssh/authorized_keys is deleted on reboot)
    # root@tomato:/jffs/# cp ~/.ssh/authorized_keys /jffs/keys/authorized_keys
     
    # 4. configure settings to be synchronized
    # change the KEYTRIG REGEX variable in this script to include all the nvram values to be replicated on the other router.
     
    # 5. Setup the synchronization schedule
    # place the following commands into the webinterface Administration:Scripts:Init
    # <code>
    #
    #        #copy the known hosts file into the root directory.
    #        cp /jffs/keys/known_hosts /tmp/home/root/.ssh/known_hosts
    #
    #        # cron job to check for changes and sync the slave router every hour.
    #        cru a SlaveRtrSync "0 */1 * * * /jffs/scr/routerSync.sh -autosync"
    #
    #        # delete the md5sum file to trigger a slave sync... jus in case.
    #        cru a ForceResync "0 */24 * * * rm /tmp/bscr/rtrSync.sh_md5"
     
    # <\code>
     
    # 6. done.
     
    # the log message cannot have any spaces.
    LogrMsg="production"
    logger "$0("$LogrMsg"): Starting... "
     
    # exit script on use of uninitialised variable
    set -u
     
    # exit script on any statement return a non-true return value:
    set -o errexit
     
    # this script is run on the MASTER Router
     
    # SLAVE ROUTER Remote Address  (THIS WILL OVERWRITE SETTINGS ON THE SLAVE ROUTER!)
    SLAVEADD="192.168.1.2"
     
    # Local Private Key Path : path to the matching private key of public key on remote machine. (on jffs mount)
    LPKP="/jffs/keys/privateKey4.txt"
     
    # Local Remote Run Script Path (on jffs mount)
    LRRSP="/jffs/scr/remotexe.sh"
     
    # Remote Dynamic Script Path
    # (THIS FILE WILL BE DELETED!)
    RDSP="/tmp/bscr/rtrSync.sh"
     
    # Local Dynamic Script Path
    # (THIS FILE WILL BE DELETED!)
    LDSP="/tmp/bscr/rtrSync.sh"
     
    # the following determines what is to be syncd:
    # REGEX to select (MATCH=COPY) the NVRAM settings ro copy to the SLAVE ROUTER.
    KEYTRIG="\'dummyload|^dhcpd_static=.*|^dnsmasq_custom=.*|^vpn_server.*\'"
     
    #TODO make KEYTRIG a global variable or argument to be executed using button scripts.
     
    #KEYTRIG="\'dummyload|^dhcpd_static=.*|^dnsmasq_custom=.*|^ntp.*|^portforward.*|^vpn_server.*|^dhcpd_.*\'"
     
    # REGEX to EXCLUDE (MATCH=DO NOT COPY) the NVRAM settings ro copy to the SLAVE ROUTER.
    NOTRIG=''
     
    # code start:
     
    # make the temport directory (if it doesn't already exist)
    mkdir -p `dirname $RDSP`
     
    # helper variables
    VBUF=""
    VIP=""
     
    dirname()
    {
    # the dirname code was copied from: http://www.linuxquestions.org/questions/programming-9/shell-scripting-getting-just-directory-string-from-file-path-string-762339/
    # thank you.
        _dirname "$1" && printf "%s\n" "$_DIRNAME"
    }
     
    _dirname()
    {
        _DIRNAME=$1;
        strip_trailing_slashes;
        case $_DIRNAME in
            "")
                _DIRNAME='/'
            return
            ;;
            */*)
                _DIRNAME="${_DIRNAME%/*}"
            ;;
            *)
                _DIRNAME='.'
            ;;
        esac;
        strip_trailing_slashes;
    }
     
    strip_trailing_slashes ()
    {
        while [ "${_DIRNAME%/}" != "$_DIRNAME" ]; do
            _DIRNAME=${_DIRNAME%/};
        done
    }
     
     
     
    # initialize the dynamic script file
    echo "#!/bin/sh
    # AutoGenerated Sync script.
    # Created: " date > $LDSP
     
    echo "
    logger \"\$0(\$1): remote sync script starting.\"
     
    " >> $LDSP
     
    if [[ -n "$NOTRIG" ]];then
        NOTRIG='|egrep -v "$NOTRIG"'
    fi
     
    logger "$0("$LogrMsg"):building dynamic script settings."
     
    nvram show | egrep $KEYTRIG | sed "s,\(^[^/]*\)=\(.*\),\1," \
      | while read NVRAMKEY; do 
     
        VBUF=$(nvram get $NVRAMKEY | sed -e s_\'_\'\""\0"\"\'_g)
       
        # do not want to replicate the Ip addresses associated with the VPN servers or clients.
        if [[ `echo $NVRAMKEY | grep -Eo '^vpn_server1_.*|^vpn_server2_.*|^vpn_client1_.*|^vpn_client2_.*'` ]];    then
          VIP=`echo $VBUF | grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | awk -F . '$1 != 127 && $1 !=256 && $4 < 256{print $1 "." $2 "." $3 "." $4}'`
        fi
     
        if ! [[ -n "$VIP" ]]; then   
            echo "nvram set $NVRAMKEY='$VBUF'" >> $LDSP
        fi   
    done
     
    echo "
     
    logger \"\$0(\$1): remote sync complete.\"
    " >> $LDSP
     
    lastMD5sumPath=$LDSP'_md5'
    lastMD5sum=''
     
     
    if [ -f $lastMD5sumPath ]
    then
        echo "$lastMD5sum file exists"
        lastMD5sum=`cat $lastMD5sumPath`
    else
        echo "$lastMD5sum file does not exist"
    fi
     
    curMD5sum=$(md5sum $LDSP)
     
    #record the md5 for future ref.
    echo "$curMD5sum" > $lastMD5sumPath
     
    echo "
    #commit the settings.
    nvram commit
    # Created: " date >> $LDSP
    #  set permissions
    chmod 775 $LDSP
     
    set -x
     
    if [[ "$1" = "-autosync" ]]; then
        echo "checking md5s..."
        if ! [[ "$lastMD5sum" = "$curMD5sum" ]]; then
            logger "$0("$LogrMsg"):Dynamic script built. Sending to remote machine."
     
            # send the script to the remote machine and execute
            logger "executing: $LRRSP -debug -u root -h $SLAVEADD -i $LPKP -s $LDSP -d $RDSP $LogrMsg"
            logger `eval $LRRSP -u root -h $SLAVEADD -i $LPKP -s $LDSP -d $RDSP $LogrMsg`
           
            logger "$0("$LogrMsg"):Sync complete."
            sleep 2
            # clean up the local temporary file
            rm $LDSP
        else
            logger "$0("$LogrMsg"):Send abort settings have not changed since last sync push."
        fi
       
    else
        # this will force a new auto push if there are issues.
        rm $lastMD5sumPath
        echo "DEBUG>>> nothing sent run with -autosync to automatically syncronize:"
        echo "or run the following command to manually syncronize:"
        echo "$LRRSP -debug -u root -h $SLAVEADD -i $LPKP -s $LDSP -d $RDSP $LogrMsg"
    fi
    
    -skyanvi1
     
  2. skyanvi1

    skyanvi1 Addicted to LI Member

    reserved if I need more space.
     
  3. tismon

    tismon Serious Server Member

    Doesn't it stink when you put in so much work and no one comments?

    That is honestly really interesting, and oddly relevant. I've been trying to find a good way to have fail-over WANs (just simply switch when down and check on schedule to switch back) and have run into too many threads with partial info or shooting for full load-balancing that I don't need. Since we're also looking at getting a house that might need the coverage of more than one router, this may be the answer when the time comes.

    I won't be able to run through this for a while, but it looks good from a glance.
    Thanks for your hard work.
     
  4. gfunkdave

    gfunkdave LI Guru Member

    Looks really cool. I just have no need of it. :)

    But thanks for posting!
     
  5. skyanvi1

    skyanvi1 Addicted to LI Member

    "Doesn't it stink when you put in so much work and no one comments?"

    Na, the setup has been working great for the past year (maybe the lack of comments = lack of encountered issues...?). The effort pay off for me is all the time I only had to change settings on one router and having a few select settings propagate to the other router (i.e. firewall, vpn, static dhcp table). If it proves useful for someone else, that is just a bonus. thanks for commenting.
     

Share This Page