NVRAM Export can not handle wrap text?

Discussion in 'Tomato Firmware' started by szpunk, Apr 30, 2013.

  1. szpunk

    szpunk LI Guru Member

    I try to backup my configuration with NVRAM Export, but i found some strange things.

    My Dnsmasq Custom configuration field look like this:


    When i use 'nvram export --set nvram | grep dnsmasq_ >> backupfile', i got this:


    A double quotation marks and the second line content was gone!

    I can confirm this bug happen in all those TEXTAREA place like USB--Run after mounting, Administration--Script´╝îNVRAM Export just drop the last double quotation marks as well as the contents of it after all.

    My router is ASUS RT-N16, and i use shibby's Tomato Firmware 1.28.0000 MIPSR2-108 K26 USB VPN now.

    Any suggest?
  2. ryzhov_al

    ryzhov_al LI Guru Member

    It's not a bug, it's a grep:))

    You cut out second string by grep. I think it's must be something like that:
    [ -e backupfile ] && rm backupfile
    for item in $(nvram show | grep ^dnsmasq | awk -F= "{print \$1}")
        echo "${item}=`nvram get ${item}`" >> backupfile
    Monk E. Boy likes this.
  3. jerrm

    jerrm Network Guru Member

    Alternatively use nvram export --quote to will keep each entry on a single grep friendly line (with new lines escaped).
  4. szpunk

    szpunk LI Guru Member

    Thanks for your reply.I use your script, it's output like this:

    2 line;

    But, if i try nvram show | grep dnsmasq_ , it's output like that:

    dnsmasq_custom=conf-dir=/mnt/optware/dnsmasq/custom/ addn-hosts=/mnt/optware/dnsmasq/hosts
    1 line.

    I want export these settings and import later after all, but both format can't import succeed, both:

    nvram set dnsmasq_custom="conf-dir=/mnt/optware/dnsmasq/custom/ addn-hosts=/mnt/optware/dnsmasq/hosts"
    Your format have 2 line, I cant use nvram set to set this.
    Maybe i should ask: How to export settings like these in 1 line and import these settings later ?

    I need a script, when i export, it can convert "line break" to "blank space", and when i import, it can convert "blank space" to "line break".
  5. HunterZ

    HunterZ Network Guru Member

    I'm also looking for a solution to this. Toastman pointed me here, as I was cluttering up his QoS thread (since he discusses the selective CLI-based nvram backup/restore method there).
    crashnburn likes this.
  6. jerrm

    jerrm Network Guru Member

    nvram export --quote is grep friendly.
  7. HunterZ

    HunterZ Network Guru Member

    OP suggests in the post above mine that the output with "--quote" is not re-importable via "nvram set".
  8. jerrm

    jerrm Network Guru Member

    There is nothing in that post referencing --quote at all. None of his examples are in --quote format.

    But no, "set" will not work, use nvram import.
    nvram export --quote | grep dnsmasq_ >> /tmp/filename
    nvram import /tmp/filename
    Or use something like rhyzov_al's script modified to create properly escaped "nvram set" entries.
    crashnburn and Monk E. Boy like this.
  9. HunterZ

    HunterZ Network Guru Member

    Thanks, I'll play around when I get a chance.
  10. jerrm

    jerrm Network Guru Member

    OK, I had been thinking about doing this for a while anyway, here is a backup script to pull out settings specified in nvram export --set format. This is a first draft, test and treat as beta or alpha code.

    EDIT: This version of the script is obsolete. Please see post #21 for updated version.
    Last edited: Jun 29, 2016
    srouquette, Monk E. Boy and szpunk like this.
  11. szpunk

    szpunk LI Guru Member

    Working perfect here, thanks!!!
  12. HunterZ

    HunterZ Network Guru Member

    Interesting. What is the magic that makes it support multi-line nvram settings values without --quote? (assuming that it does support those)
  13. jerrm

    jerrm Network Guru Member

    "nvram export --set" has always supported multi-line settings, the problem when using grep to parse the "--set " output is that only the first line matches. The script works around it by issuing an "nvram get" for each match and writing out the full variable in the same format "--set" uses.
  14. szpunk

    szpunk LI Guru Member

    jerrm's script export multi-line nvram settings values like this:


    and in Tomato's GUI can accept this!
  15. bagu

    bagu Network Guru Member

    I use tomatousb with vlan support.

    I have done a diff between the default nvram and the nvram after configuration.
    Here are tha nvram values witch have change : http://pastebin.com/hWzyMRVW
  16. Monk E. Boy

    Monk E. Boy Network Guru Member

    What values change in NVRAM are highly dependent on what options you change in the GUI.
  17. HunterZ

    HunterZ Network Guru Member

    Edit: Obsolete
  18. FlashSWT

    FlashSWT Network Guru Member

    I'm gonna re-mention this issue that got buried in the QOS thread with no responses:

  19. srouquette

    srouquette Network Guru Member

    you may want to add ^portforward to your script.
    it could be a good idea to create a new thread for this script, it's quite useful.
  20. HunterZ

    HunterZ Network Guru Member

    Is it possible that the NVRAM data is cached into RAM at startup, such that changes may not be re-read into the live environment until you reboot?

    Edit: A quick google search supports this theory: http://www.dd-wrt.com/phpBB2/viewtopic.php?p=692095
  21. jerrm

    jerrm Network Guru Member

    This version of the script is probably my final. It brings the original in line with most of our standards - may or may not make sense for you.

    Functional changes:
    • Output defaults to stdout unless a filename is passed on the command line.
    • Optionally reads a config file for strings to match instead of having to edit script.
    • Now assumes all entries should match at beginning of line - no need to enter the caret ^ in match strings.
    #Backs up selected nvram variables in "nvram export --set" format.
    #Correctly handles multi-line entries.
    #Thanks to ryzhov_al for basic approach.
    #Should work equally well with both MIPS and ARM builds.
    #Looks for a list of items to export in $etc/scriptname.ini
    #OR enter items to grep for below.
    #The items list is a list of regular expressions to match against the
    #nvram variable names.
    #Script assumes all entries are at beginning of line(prefixed with ^).
    #Leave items list blank to backup up all of nvram.  Resulting in essentially
    #the same output as MIPS "nvram export --set"
    #The items list below is only intended as example and is not complete or
    #comprehensive. Customize for your own use.
    #Edit list below if not using .ini file, it is ignored if .ini file is found
    base=${0##*/}; base=${base%.*}
    #file to output - default to stdout
    if [ "$1" != "" ] ; then
    grepstr=$( { [ -r $config ] && cat $config || echo "$items" ; } | sed -e 's/[\t ]//g;/^$/d' | sed ':a;N;$!ba;s/\n/\\\|\^/g')
    echo "#Exporting $grepstr"
    for item in $(nvram show 2>/dev/null | grep "^[^[:space:]]*=" | grep "$grepstr"  | awk -F= "{print \$1}" | sort -u)
      item_value="$(nvram get $item | sed 's!\([\$\"\`]\)!\\\1!g'; echo nvgetwasnull)"
      case $item_value in
      nvgetwasnull) ;;
      *) echo "nvram set ${item}=\"${item_value%
    nvgetwasnull}\"" ;;

    Above Script Updated 2015-07-02 to better deal with ARM builds

    ARM's nvram utility is limited compared to MIPS. The original script was functional under ARM builds, but due to differences in ARM output could theoretically create extraneous nvram variables if a multi-line entry contains something looking like a variable assignment. Other than consuming a little nvram this should have been harmless.

    The updated script resolves these issues and should function equally well under both ARM and MIPS builds. Additionally it provides nvram export functionality otherwise missing from ARM builds.
    Last edited: Aug 17, 2016
    crashnburn, Xruptor, AndreDVJ and 6 others like this.
  22. HunterZ

    HunterZ Network Guru Member

    After actually testing, here's the list of stuff that is important to me:
  23. bagu

    bagu Network Guru Member

    And vlan ;)
  24. HunterZ

    HunterZ Network Guru Member

    I don't use a VLAN firmware, and some of my choices are very specific to my own needs. I've learned through this exercise that there is definitely no "one-size-fits-all" list.
  25. gfunkdave

    gfunkdave LI Guru Member

    Thanks for pointing me to this in Toastman's thread, jerrm.

    Anyone know why my script complains that /bin/sh isn't found? It executes the script fine anyway. I'm running
    1.28.0503 MIPSR2Toastman-RT-N K26 USB VPN on an RT-N66U.

    root@router:/tmp/mnt/STICK# run-backup.sh
    run-backup.sh: line 1: #!/bin/sh: not found
    crashnburn likes this.
  26. koitsu

    koitsu Network Guru Member

    The problem there isn't that /bin/sh isn't found, it's that either the encoding in the file is UTF-8 or UTF-16 with BOM (byte-order marker), or it's that the script is DOS or Mac OS 9 file format (contains CR+LF or just CR, respectively, when UNIX uses just LF).

    If you have vim installed (from Entware), there is a utility that comes with vim called xxd which will allow you to hex dump a file. xxd run-backup.sh | head -3 would be enough to know. A properly formatted script would start with these bytes and nothing more:

    0000000: 2321 2f62 696e 2f73 680a
    Which maps to #!/bin/sh<LF>
    ruggerof and gfunkdave like this.
  27. gfunkdave

    gfunkdave LI Guru Member

    Koitsu, you never cease to amaze me. :)

    I had already switched to Unix EOL characters, which had fixed some of the issues. I used Notepad++ to convert ot UTF-8 without BOM, and now the script runs fine.

    crashnburn likes this.
  28. crashnburn

    crashnburn Network Guru Member

    I am considering using the knowledge in this thread for export and import, I am wondering if this would still work/ its valid? (PS: I understand settings/ fields would change for firmware)
  29. jerrm

    jerrm Network Guru Member

    For MIPS nothing has changed.

    There are some differences in the nvram utility under ARM. I think the script should be OK for ARM, but haven't tested in an ARM build.
  30. sputnik

    sputnik Networkin' Nut Member

    I just upgraded from 1.28.0505 MIPSR2Toastman-RT-N K26 USB VLAN-VPN to 1.28.0507.2 MIPSR2Toastman-RT-N K26 USB VLAN-VPN with this script and it was a dream :). I did have one issue, my qos_orules is longer than accepted on command line, so I had to paste it in manually with breaks (\). Not worth whining about (or fixing), it's just awesome.
    Edit: Don't do what I did. Been awhile, forgot, you cannot use line breaks, they get accepted into the nvram variable and I don't think tomato knows how to deal with them. Just bite the bullet and do a little manual maintenance ;)
    Last edited: Jul 9, 2015
  31. jerrm

    jerrm Network Guru Member

    Last edited: Aug 24, 2015
    crashnburn likes this.
  32. ilium007

    ilium007 Serious Server Member

    Thanks for this. Been struggling with the multiline output from nvram show on ARM for a few weeks now.
  33. Sortec

    Sortec Reformed Router Member

    This is awesome! Is there a way to re-import the data after an upgrade?

    edit: also, I ran the script but noticed that my port forwarding table is not included.
    Does anyone know the variable names for the port forwarding table so that I can also export those as well?

    edit again: Found it
    nvram set portforward=

    Last edited: Mar 30, 2016
    crashnburn likes this.
  34. mcbsys

    mcbsys Networkin' Nut Member

    This script was extremely helpful in copying a configuration from one router to another (in my case two E2000s). I wrote up a basic How-To in case it helps anyone else:

    How to Use the MultiLine Script to Copy a Tomato Configuration
    May 17, 2016

    These instructions are about how to copy a Tomato configuration to another router of the same model using the same firmware version. With care, some of it should work when copying configurations to a different firmware version. You can use the instructions in step 6 to compare the settings available in each router. I'm working from Windows machines, hence the references to WinSCP and PuTTY.

    Note that when copying a router configuration, the new router will have the same IP range as the old one, so be sure work on a separate, isolated computer when connecting to the new router. (I set my laptop to Airplane Mode to disconnect it from the source router's Wi-Fi.)

    1. The multilineexport.sh script is from http://www.linksysinfo.org/index.ph...t-can-not-handle-wrap-text.68478/#post-228672 above. The default script expects to be working in the /opt/etc directory, but my Shibby build routers do not have that directory and I could not create it. So I changed this line in the script




    2. Create and edit a multilineexport.ini file for whatever values you want to export. My list is a compilation of entries in this post:

    as well as of entries in an old "back up settings" script and some trial and error.

    3. Connect to the "source" Tomato router (whose config you want to copy) using WinSCP (must enable SSH first). Use WinSCP to upload multilineexport.sh and multilineexport.ini to the router's /tmp/etc directory. Make the script executable (permissions 0744). Note: files in /tmp are apparently deleted when the router reboots, so save them locally on your computer.

    4. From WinSCP, choose Commands > Open in PuTTY. Use PuTTY to run the script:

    cd /tmp/etc
    ./multilineexport.sh > multilineexport.txt​

    5. Use WinSCP to open multilinexport.txt and paste its contents into the new router to import (System > Tools).

    5a. Delete the first line (long comment line about included items).
    5b. Add "nvram commit" as the last line!
    5c. Reboot the router. It should come up under the source router's IP address.​

    6. Check your work. Run the following command in PuTTY on both routers, get the output files onto the same computer, and compare in Notepad++ (Plugins > Compare):

    nvram export --set --quote | sort > config_old.txt​

    Some differences are expected, e.g. hardware addresses, WAN not connected, etc., but for the most part, the files should match.
    Last edited: May 18, 2016
    crashnburn and Monk E. Boy like this.
  35. lolento

    lolento Serious Server Member

    hi guys,

    can someone explain to me how do I get a comprehensive list of NVRAM variables to add? I like to backup my static IPs, port forwards and QoS settings...
  36. mcbsys

    mcbsys Networkin' Nut Member

    lolento, to just list everything, break down step 6 of the post just before yours:

    nvram export --set --quote

    or to sort it and write it to a text file:

    nvram export --set --quote | sort > config_old.txt
  37. lolento

    lolento Serious Server Member


    I tried both lines on Putty:

    nvram export --set --quote

    nvram export --set --quote | sort > config_old.txt

    Nothing happens. I get an empty config_old.txt.
  38. mcbsys

    mcbsys Networkin' Nut Member

    What model router do you have? From skimming this thread, there are differences if you have an ARM processor. What happens if you try

    nvram export --set
  39. lolento

    lolento Serious Server Member

    i have the netgear r8000 running shibby v137 fw.
  40. mcbsys

    mcbsys Networkin' Nut Member

    The R-7000 is an ARM router so I assume the R8000 is as well: http://tomato.groov.pl/?page_id=69.

    See post #21 above: "ARM's nvram utility is limited compared to MIPS." So maybe it doesn't allow the --quote parameter.

    You may need to play around with nvram to learn its capabilities on your router, e.g. "nvram --help". Does "nvram export --set" work?
  41. jerrm

    jerrm Network Guru Member

    ARM has no "export" options at all, only "nvram show."
  42. lolento

    lolento Serious Server Member

    Awesome, yes.

    "nvram show" works.

    EDIT: But I'm running into issues actually running the script. Putty keep saying the .sh file is not found. The script has the permission stated above and encoding is just UTF-8
    Last edited: Jun 29, 2016
    crashnburn likes this.
  43. crashnburn

    crashnburn Network Guru Member

    I'd like to do this but I am not sure about 2 things:

    1. I believe this has to be done on the Shell prompt after logging in via SSH - Typical SSH port? Login/ Pwd? Using Bitvise or other SSH client?

    2. How & WHERE do I copy the script files ON to the Router after I download the scripts from here/ forum etc? Are we to use SFTP or something? Where to keep the scripts for execution? What privileges and how to execute?
  44. crashnburn

    crashnburn Network Guru Member

    I am running as per your instructions on ASUS RT-N16 (MIPS) and I get the following:

    Tomato v1.28.9054 MIPSR2-beta K26 USB vpn3.6
    root@AS_TOM_HOST:/tmp/home/root# cd /tmp/etc
    root@AS_TOM_HOST:/tmp/etc# ./multilineexport.sh > multilineexport.txt
    root@AS_TOM_HOST:/tmp/etc# ./multilineexport.sh > multilineexport.txt
    -sh: ./multilineexport.sh: not found
    root@AS_TOM_HOST:/tmp/etc# multilineexport.sh > multilineexport.txt
    -sh: multilineexport.sh: not found
    root@AS_TOM_HOST:/tmp/etc# multilineexport.sh > multilineexport3.txt
    -sh: multilineexport.sh: not found
    Getting this not found thing

    What's going wrong?
    Permissions were given,
    Sample ini file (Code copied as Is - has 4/5 leading spaces before the text on each line) is the one from above post,
    The encoding of the .sh and .ini is ANSI (was UTF-8-BOM before causing error).
    Is that correct?

    Please advise. Thanks.

    For a sample nvramtest.sh:
    nvram export --set | grep rrule1
    nvram export --set | grep qos_
    nvram export --set | grep dhcp_
    nvram export --set | grep dhcpd_
    nvram export --set | grep ddnsx0=
    nvram export --set | grep lan_hostname=
    nvram export --set | grep lan_ipaddr=
    nvram export --set | grep wan_proto=
    nvram export --set | grep wan_hostname=
    nvram export --set | grep wan_domain
    nvram export --set | grep pppoe_username=
    nvram export --set | grep pppoe_passwd=
    nvram export --set | grep http_
    nvram export --set | grep router_name
    root@AS_TOM_HOST:/tmp/etc# nvramtest.sh >nvramtest001.txt
    I get a file output:
    nvram set router_name="AS_TOMATO"
    But running the script with .INI file or without gets an empty file with output "not found" as above.
    Last edited: Mar 30, 2017
  45. jerrm

    jerrm Network Guru Member

    The "not found" error is almost certainly because the script file has DOS/Windows line endings.
    Last edited: Mar 31, 2017
  46. crashnburn

    crashnburn Network Guru Member

    Interesting. I am using Notepad++ to change how it's saved. So, how can that be going wrong?
    Or is there a better way to do this on Windows?
    Do advise how (which tool/ app) I can resolve this issue?

    PS: But the above sample nvramtest.sh: seems to have worked without the issue.
    PPS: I also got the Hex Plugin and a Hex Editor - what should I look for and how/ what should I change stuff to?
    Last edited: Mar 31, 2017
  47. crashnburn

    crashnburn Network Guru Member

    Ok. Figured out there's 2 things that can be done (not sure if there's more) in Notepad++.
    - Change EOL to Unix (other options Windows / Mac)
    - Change encoding of file .sh and .ini
    - Should I make it ANSI or UTF-8 / UTF-8-BOM etc etc.. Which one is preferred for Tomato's variation of Hyper WRT? Please advise. Thanks.

    - Also, there are leading spaces in the .INI code sample above, are those needed/ not needed/ must have/ or are dont care/ doesnt matter as long as valid EOL is present? How should I design my .INI file?

    Could some of us please share their preferred INI files so that we can also save some of those (just to learn by example)?
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice