1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

NVRAM Export can not handle wrap text?

Discussion in 'Tomato Firmware' started by szpunk, Apr 30, 2013.

  1. szpunk

    szpunk Networkin' Nut Member

    I try to backup my configuration with NVRAM Export, but i found some strange things.

    My Dnsmasq Custom configuration field look like this:

    [​IMG]

    When i use 'nvram export --set nvram | grep dnsmasq_ >> backupfile', i got this:

    [​IMG]

    A double quotation marks and the second line content was gone!

    I can confirm this bug happen in all those TEXTAREA place like USB--Run after mounting, Administration--Script´╝îNVRAM Export just drop the last double quotation marks as well as the contents of it after all.

    My router is ASUS RT-N16, and i use shibby's Tomato Firmware 1.28.0000 MIPSR2-108 K26 USB VPN now.

    Any suggest?
     
  2. ryzhov_al

    ryzhov_al Networkin' Nut Member

    It's not a bug, it's a grep:))

    You cut out second string by grep. I think it's must be something like that:
    Code:
    #!/bin/sh
     
    [ -e backupfile ] && rm backupfile
    for item in $(nvram show | grep ^dnsmasq | awk -F= "{print \$1}")
    do
        echo "${item}=`nvram get ${item}`" >> backupfile
    done
    
     
    Monk E. Boy likes this.
  3. jerrm

    jerrm Network Guru Member

    Alternatively use nvram export --quote to will keep each entry on a single grep friendly line (with new lines escaped).
     
  4. szpunk

    szpunk Networkin' Nut Member

    Thanks for your reply.I use your script, it's output like this:

    Code:
    dnsmasq_custom=conf-dir=/mnt/optware/dnsmasq/custom/
    addn-hosts=/mnt/optware/dnsmasq/hosts
    
    2 line;

    But, if i try nvram show | grep dnsmasq_ , it's output like that:

    Code:
    dnsmasq_custom=conf-dir=/mnt/optware/dnsmasq/custom/ addn-hosts=/mnt/optware/dnsmasq/hosts
    
    1 line.

    I want export these settings and import later after all, but both format can't import succeed, both:


    Code:
    nvram set dnsmasq_custom="conf-dir=/mnt/optware/dnsmasq/custom/ addn-hosts=/mnt/optware/dnsmasq/hosts"
     
    or
     
    Your format have 2 line, I cant use nvram set to set this.
    Maybe i should ask: How to export settings like these in 1 line and import these settings later ?

    I need a script, when i export, it can convert "line break" to "blank space", and when i import, it can convert "blank space" to "line break".
     
  5. HunterZ

    HunterZ LI Guru Member

    I'm also looking for a solution to this. Toastman pointed me here, as I was cluttering up his QoS thread (since he discusses the selective CLI-based nvram backup/restore method there).
     
    crashnburn likes this.
  6. jerrm

    jerrm Network Guru Member

    nvram export --quote is grep friendly.
     
  7. HunterZ

    HunterZ LI Guru Member

    OP suggests in the post above mine that the output with "--quote" is not re-importable via "nvram set".
     
  8. jerrm

    jerrm Network Guru Member

    There is nothing in that post referencing --quote at all. None of his examples are in --quote format.

    But no, "set" will not work, use nvram import.
    Code:
    nvram export --quote | grep dnsmasq_ >> /tmp/filename
    nvram import /tmp/filename
    Or use something like rhyzov_al's script modified to create properly escaped "nvram set" entries.
     
    crashnburn and Monk E. Boy like this.
  9. HunterZ

    HunterZ LI Guru Member

    Thanks, I'll play around when I get a chance.
     
  10. jerrm

    jerrm Network Guru Member

    OK, I had been thinking about doing this for a while anyway, here is a backup script to pull out settings specified in nvram export --set format. This is a first draft, test and treat as beta or alpha code.

    EDIT: This version of the script is obsolete. Please see post #21 for updated version.
     
    Last edited: Jun 29, 2016
    srouquette, Monk E. Boy and szpunk like this.
  11. szpunk

    szpunk Networkin' Nut Member

    Working perfect here, thanks!!!
     
  12. HunterZ

    HunterZ LI Guru Member

    Interesting. What is the magic that makes it support multi-line nvram settings values without --quote? (assuming that it does support those)
     
  13. jerrm

    jerrm Network Guru Member

    "nvram export --set" has always supported multi-line settings, the problem when using grep to parse the "--set " output is that only the first line matches. The script works around it by issuing an "nvram get" for each match and writing out the full variable in the same format "--set" uses.
     
  14. szpunk

    szpunk Networkin' Nut Member

    jerrm's script export multi-line nvram settings values like this:

    [​IMG]

    and in Tomato's GUI can accept this!
     
  15. bagu

    bagu Network Guru Member

    I use tomatousb with vlan support.

    I have done a diff between the default nvram and the nvram after configuration.
    Here are tha nvram values witch have change : http://pastebin.com/hWzyMRVW
     
  16. Monk E. Boy

    Monk E. Boy Network Guru Member

    What values change in NVRAM are highly dependent on what options you change in the GUI.
     
  17. HunterZ

    HunterZ LI Guru Member

    Edit: Obsolete
     
  18. FlashSWT

    FlashSWT LI Guru Member

    I'm gonna re-mention this issue that got buried in the QOS thread with no responses:

     
  19. srouquette

    srouquette Network Guru Member

    you may want to add ^portforward to your script.
    it could be a good idea to create a new thread for this script, it's quite useful.
     
  20. HunterZ

    HunterZ LI Guru Member

    Is it possible that the NVRAM data is cached into RAM at startup, such that changes may not be re-read into the live environment until you reboot?

    Edit: A quick google search supports this theory: http://www.dd-wrt.com/phpBB2/viewtopic.php?p=692095
     
  21. jerrm

    jerrm Network Guru Member

    This version of the script is probably my final. It brings the original in line with most of our standards - may or may not make sense for you.

    Functional changes:
    • Output defaults to stdout unless a filename is passed on the command line.
    • Optionally reads a config file for strings to match instead of having to edit script.
    • Now assumes all entries should match at beginning of line - no need to enter the caret ^ in match strings.
    Code:
    #!/bin/sh
    
    #USE AT YOUR OWN RISK.
    #THIS SCRIPT DOES NOT COME WITH ANY WARRANTY WHATSOEVER.
    #
    #Backs up selected nvram variables in "nvram export --set" format.
    #
    #Correctly handles multi-line entries.
    #
    #Thanks to ryzhov_al for basic approach.
    #
    #Should work equally well with both MIPS and ARM builds.
    #
    #Looks for a list of items to export in $etc/scriptname.ini
    #OR enter items to grep for below.
    #
    #The items list is a list of regular expressions to match against the
    #nvram variable names.
    #
    #Script assumes all entries are at beginning of line(prefixed with ^).
    #
    #Leave items list blank to backup up all of nvram.  Resulting in essentially
    #the same output as MIPS "nvram export --set"
    #
    #The items list below is only intended as example and is not complete or
    #comprehensive. Customize for your own use.
    #
    
    #Edit list below if not using .ini file, it is ignored if .ini file is found
    items='
      dhcpd_
      dns_
      wl[0-9]_security_mode
      wl[0-9]_ssid
      wl[0-9]_wpa_psk
    '
    
    etc=/opt/etc
    base=${0##*/}; base=${base%.*}
    config=$etc/$base.ini
    
    #file to output - default to stdout
    if [ "$1" != "" ] ; then
      backupfile="$1"
    else
      backupfile=/proc/$$/fd/1
    fi
    
    grepstr=$( { [ -r $config ] && cat $config || echo "$items" ; } | sed -e 's/[\t ]//g;/^$/d' | sed ':a;N;$!ba;s/\n/\\\|\^/g')
    
    {
    echo "#Exporting $grepstr"
    for item in $(nvram show 2>/dev/null | grep "^[^[:space:]]*=" | grep "$grepstr"  | awk -F= "{print \$1}" | sort -u)
    do
      item_value="$(nvram get $item | sed 's!\([\$\"\`]\)!\\\1!g'; echo nvgetwasnull)"
      case $item_value in
      nvgetwasnull) ;;
      *) echo "nvram set ${item}=\"${item_value%
    nvgetwasnull}\"" ;;
      esac
    done
    }>"$backupfile"
    

    EDIT:
    Above Script Updated 2015-07-02 to better deal with ARM builds

    ARM's nvram utility is limited compared to MIPS. The original script was functional under ARM builds, but due to differences in ARM output could theoretically create extraneous nvram variables if a multi-line entry contains something looking like a variable assignment. Other than consuming a little nvram this should have been harmless.

    The updated script resolves these issues and should function equally well under both ARM and MIPS builds. Additionally it provides nvram export functionality otherwise missing from ARM builds.
     
    Last edited: Aug 17, 2016
    crashnburn, Xruptor, AndreDVJ and 6 others like this.
  22. HunterZ

    HunterZ LI Guru Member

    After actually testing, here's the list of stuff that is important to me:
    Code:
        cifs1
        cstats_
        ddnsx[0_]
        dhcp_
        dhcpd_
        dns_
        http_lanport
        http_passwd
        https_
        lan_hostname
        lan_ipaddr
        lan_proto
        log_
        macnames
        ntp_
        router_name
        rstats_
        sch_
        script_
        sshd_
        tm_
        upnp_
        vpn_server1_
        vpn_server_eas
        wan_dns
        wan_hostname
        wan_proto
        web_css
        web_mx
        wl0_akm
        wl0_channel
        wl0_country
        wl0_crypto
        wl0_frameburst
        wl0_key
        wl0_mac
        wl0_mode
        wl0_nbw
        wl0_nctrlsb
        wl0_security_mode
        wl0_ssid
        wl0_txpwr
        wl0_wpa_psk
        wl_maclist
        wl_macmode
    
     
  23. bagu

    bagu Network Guru Member

    And vlan ;)
     
  24. HunterZ

    HunterZ LI Guru Member

    I don't use a VLAN firmware, and some of my choices are very specific to my own needs. I've learned through this exercise that there is definitely no "one-size-fits-all" list.
     
  25. gfunkdave

    gfunkdave LI Guru Member

    Thanks for pointing me to this in Toastman's thread, jerrm.

    Anyone know why my script complains that /bin/sh isn't found? It executes the script fine anyway. I'm running
    1.28.0503 MIPSR2Toastman-RT-N K26 USB VPN on an RT-N66U.


    Code:
    root@router:/tmp/mnt/STICK# run-backup.sh
    run-backup.sh: line 1: #!/bin/sh: not found
    root@router:/tmp/mnt/STICK#
    
    Thanks!
     
    crashnburn likes this.
  26. koitsu

    koitsu Network Guru Member

    The problem there isn't that /bin/sh isn't found, it's that either the encoding in the file is UTF-8 or UTF-16 with BOM (byte-order marker), or it's that the script is DOS or Mac OS 9 file format (contains CR+LF or just CR, respectively, when UNIX uses just LF).

    If you have vim installed (from Entware), there is a utility that comes with vim called xxd which will allow you to hex dump a file. xxd run-backup.sh | head -3 would be enough to know. A properly formatted script would start with these bytes and nothing more:

    Code:
    0000000: 2321 2f62 696e 2f73 680a
    
    Which maps to #!/bin/sh<LF>
     
    ruggerof and gfunkdave like this.
  27. gfunkdave

    gfunkdave LI Guru Member

    Koitsu, you never cease to amaze me. :)

    I had already switched to Unix EOL characters, which had fixed some of the issues. I used Notepad++ to convert ot UTF-8 without BOM, and now the script runs fine.

    Thanks!
     
    crashnburn likes this.
  28. crashnburn

    crashnburn LI Guru Member

    I am considering using the knowledge in this thread for export and import, I am wondering if this would still work/ its valid? (PS: I understand settings/ fields would change for firmware)
     
  29. jerrm

    jerrm Network Guru Member

    For MIPS nothing has changed.

    There are some differences in the nvram utility under ARM. I think the script should be OK for ARM, but haven't tested in an ARM build.
     
  30. sputnik

    sputnik Networkin' Nut Member

    I just upgraded from 1.28.0505 MIPSR2Toastman-RT-N K26 USB VLAN-VPN to 1.28.0507.2 MIPSR2Toastman-RT-N K26 USB VLAN-VPN with this script and it was a dream :). I did have one issue, my qos_orules is longer than accepted on command line, so I had to paste it in manually with breaks (\). Not worth whining about (or fixing), it's just awesome.
    Edit: Don't do what I did. Been awhile, forgot, you cannot use line breaks, they get accepted into the nvram variable and I don't think tomato knows how to deal with them. Just bite the bullet and do a little manual maintenance ;)
     
    Last edited: Jul 9, 2015
  31. jerrm

    jerrm Network Guru Member

    Last edited: Aug 24, 2015
    crashnburn likes this.
  32. ilium007

    ilium007 Reformed Router Member

    Thanks for this. Been struggling with the multiline output from nvram show on ARM for a few weeks now.
     
  33. Sortec

    Sortec Network Newbie Member

    This is awesome! Is there a way to re-import the data after an upgrade?


    edit: also, I ran the script but noticed that my port forwarding table is not included.
    Does anyone know the variable names for the port forwarding table so that I can also export those as well?


    edit again: Found it
    nvram set portforward=

    Thanx!
     
    Last edited: Mar 30, 2016
    crashnburn likes this.
  34. mcbsys

    mcbsys Networkin' Nut Member

    This script was extremely helpful in copying a configuration from one router to another (in my case two E2000s). I wrote up a basic How-To in case it helps anyone else:

    How to Use the MultiLine Script to Copy a Tomato Configuration
    May 17, 2016

    These instructions are about how to copy a Tomato configuration to another router of the same model using the same firmware version. With care, some of it should work when copying configurations to a different firmware version. You can use the instructions in step 6 to compare the settings available in each router. I'm working from Windows machines, hence the references to WinSCP and PuTTY.

    Note that when copying a router configuration, the new router will have the same IP range as the old one, so be sure work on a separate, isolated computer when connecting to the new router. (I set my laptop to Airplane Mode to disconnect it from the source router's Wi-Fi.)

    1. The multilineexport.sh script is from http://www.linksysinfo.org/index.ph...t-can-not-handle-wrap-text.68478/#post-228672 above. The default script expects to be working in the /opt/etc directory, but my Shibby build routers do not have that directory and I could not create it. So I changed this line in the script

    etc=/opt/etc​

    to

    etc=/tmp/etc​

    2. Create and edit a multilineexport.ini file for whatever values you want to export. My list is a compilation of entries in this post:


    as well as of entries in an old "back up settings" script and some trial and error.

    3. Connect to the "source" Tomato router (whose config you want to copy) using WinSCP (must enable SSH first). Use WinSCP to upload multilineexport.sh and multilineexport.ini to the router's /tmp/etc directory. Make the script executable (permissions 0744). Note: files in /tmp are apparently deleted when the router reboots, so save them locally on your computer.

    4. From WinSCP, choose Commands > Open in PuTTY. Use PuTTY to run the script:

    cd /tmp/etc
    ./multilineexport.sh > multilineexport.txt​

    5. Use WinSCP to open multilinexport.txt and paste its contents into the new router to import (System > Tools).

    5a. Delete the first line (long comment line about included items).
    5b. Add "nvram commit" as the last line!
    5c. Reboot the router. It should come up under the source router's IP address.​

    6. Check your work. Run the following command in PuTTY on both routers, get the output files onto the same computer, and compare in Notepad++ (Plugins > Compare):

    nvram export --set --quote | sort > config_old.txt​

    Some differences are expected, e.g. hardware addresses, WAN not connected, etc., but for the most part, the files should match.
     
    Last edited: May 18, 2016
    Monk E. Boy likes this.
  35. lolento

    lolento Reformed Router Member

    hi guys,

    can someone explain to me how do I get a comprehensive list of NVRAM variables to add? I like to backup my static IPs, port forwards and QoS settings...
     
  36. mcbsys

    mcbsys Networkin' Nut Member

    lolento, to just list everything, break down step 6 of the post just before yours:

    nvram export --set --quote

    or to sort it and write it to a text file:

    nvram export --set --quote | sort > config_old.txt
     
  37. lolento

    lolento Reformed Router Member

    Hi,

    I tried both lines on Putty:

    nvram export --set --quote

    nvram export --set --quote | sort > config_old.txt

    Nothing happens. I get an empty config_old.txt.
     
  38. mcbsys

    mcbsys Networkin' Nut Member

    What model router do you have? From skimming this thread, there are differences if you have an ARM processor. What happens if you try

    nvram export --set
     
  39. lolento

    lolento Reformed Router Member

    i have the netgear r8000 running shibby v137 fw.
     
  40. mcbsys

    mcbsys Networkin' Nut Member

    The R-7000 is an ARM router so I assume the R8000 is as well: http://tomato.groov.pl/?page_id=69.

    See post #21 above: "ARM's nvram utility is limited compared to MIPS." So maybe it doesn't allow the --quote parameter.

    You may need to play around with nvram to learn its capabilities on your router, e.g. "nvram --help". Does "nvram export --set" work?
     
  41. jerrm

    jerrm Network Guru Member

    ARM has no "export" options at all, only "nvram show."
     
  42. lolento

    lolento Reformed Router Member

    Awesome, yes.

    "nvram show" works.

    EDIT: But I'm running into issues actually running the script. Putty keep saying the .sh file is not found. The script has the permission stated above and encoding is just UTF-8
     
    Last edited: Jun 29, 2016
  43. crashnburn

    crashnburn LI Guru Member

    I'd like to do this but I am not sure about 2 things:

    1. I believe this has to be done on the Shell prompt after logging in via SSH - Typical SSH port? Login/ Pwd? Using Bitvise or other SSH client?

    2. How & WHERE do I copy the script files ON to the Router after I download the scripts from here/ forum etc? Are we to use SFTP or something? Where to keep the scripts for execution? What privileges and how to execute?
     

Share This Page