Does Shibby have mtd-erase?

Discussion in 'Tomato Firmware' started by Bird333, May 22, 2018.

  1. Bird333

    Bird333 Network Guru Member

    Router RT-AC68U. Does anybody know if Shibby has mtd-erase that can erase the mtd5 partition? If not, anybody know where I can download it?

    Or an alternate way to write 'FF' to this partition will be fine.
    Last edited: May 22, 2018
  2. eibgrad

    eibgrad Network Guru Member

    find / -iname mtd-erase*
    In my case, this returned ...

    Note, messing w/ the partitions is risky business. I make no claims as to the appropriateness of using these mtd-erase commands. Use at your own risk.
  3. Bird333

    Bird333 Network Guru Member

    I read somewhere that mtd-erase2 won't let you erase that partition. I really just need a way to write 'FF' over the whole thing. You got any other ideas how to do that?
  4. eibgrad

    eibgrad Network Guru Member

    dd command?
  5. koitsu

    koitsu Network Guru Member

    This is actually painfully difficult to do on UNIX, despite it seeming like all the tools you need are there and at your disposal. dd is all about copying data or writing zeros, not other values. This probably will not work on stock Tomato/stock Busybox because the tools (ex. dd) are so limited in their functionality; Entware is almost certainly going to be required, and I expect it to be so below (but will note where it's needed). UNIX is surprisingly unfriendly when it comes to things like this, which is why people make their own C-based programs to do the job.


    First thing you should try, if you have the available disk for it (ex. a USB flash drive or something), is to generate a file that contains the same number of bytes as what your mtd5 partition contains. They need to be identical. If they aren't, all bets are off.

    /proc/mtd will disclose the size of each partition. For sake of making things difficult for ourselves, let's pretend the output you get looks like this (I've edited these values for sake of example):

    root@gw:/tmp/home/root# cat /proc/mtd
    dev:    size   erasesize  name
    mtd0: 00080000 00020000 "boot"
    mtd1: 00180000 00020000 "nvram"
    mtd2: 01e00000 00020000 "linux"
    mtd3: 01c5dc14 00020000 "rootfs"
    mtd4: 06000000 00020000 "brcmnand"
    mtd5: 02c00000 00020000 "jffs2"
    size refers to the partition size, erasesize refers to the NAND erase block size.

    You need to convert size 2c00000 from hex into decimal, since that's what dd works with. Sadly there are no native tools on Busybox to do this, so you'll either need Entware's bc package or some other *IX machine with bc. You'll also need to turn all lowercase a-f characters in the hexadecimal string into uppercase, otherwise bc won't parse them and will treat them as zeros or no-ops (often without any error message):

    root@gw:/tmp/home/root# bc
    bc 1.06.95
    Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
    This is free software with ABSOLUTELY NO WARRANTY.
    For details type `warranty'.
    You're also going to need to convert erase size for the partition to a decimal value too:

    As stated before, if possible, the easiest thing to do is to generate a file 46137344 bytes long containing nothing but 0xFF bytes, which we'll copy to mtd5 using dd. The way get a file filled with nothing but 0xFF is through dd and tr, turning all zeros into 0xFF. The tr command converts any bytes of octal value 000 (zero) to octal value 377 (0xFF); the only raw numeric input syntax it supports is octal.

    However, tr can sometimes result in "unexpected" output due supporting locales. This can drive people mad unless they're aware of it. The LANG= LC_CTYPE=C part of the line is ensuring LANG is set to nothing, and LC_CTYPE is set to C, rather than something like a UTF-8 locale -- which would return very different values (you'd get 0xC3BF instead of 0xFFFF).

    Also note that in the tr command, those are apostrophes, not double-quotes.

    So let's try this, assuming /tmp can hold a 46MByte file. On TomatoUSB this is RAM, so you may want to pick a different destination, ex. a mounted USB flash drive. Don't pick /tmp unless it can hold the results! You've been warned!

    root@gw:/tmp/home/root# /bin/dd if=/dev/zero ibs=1 count=46137344 | LANG= LC_CTYPE=C /usr/bin/tr '\000' '\377' > /tmp/blank
    This will take a while because the input block size is 1 (i.e. read 1 byte from /dev/zero) rather than in larger chunks. The reason I did it this way is because I didn't want to explain the math involved in dealing with, say, an erase size boundary, 1024-byte boundary, etc.. Usually the partitions align to an erase size boundary, but not always (see rootfs), so I assume the worst. Otherwise this file generation would be a lot faster.

    The end result should be a /tmp/blank file that contains nothing but 0xFF characters, matching the size of your mtd5 partition:

    root@gw:/tmp/home/root# ls -l /tmp/blank
    -rw-r--r--    1 root     root      46137344 May 22 13:47 /tmp/blank
    But how can we verify the contents? You can't without Entware, because there's no hex-dump-like utility that comes with Busybox/TomatoUSB. I might be able to do some awk madness to accomplish the equivalent, but would rather not. We'll use xxd from Entware to verify. Make sure the start and end of the file contain 0xFF as expected (everything else in between would be identical):

    root@gw:/tmp/home/root# xxd /tmp/blank | head -5
    00000000: ffff ffff ffff ffff ffff ffff ffff ffff  ................
    00000010: ffff ffff ffff ffff ffff ffff ffff ffff  ................
    00000020: ffff ffff ffff ffff ffff ffff ffff ffff  ................
    00000030: ffff ffff ffff ffff ffff ffff ffff ffff  ................
    00000040: ffff ffff ffff ffff ffff ffff ffff ffff  ................
    root@gw:/tmp/home/root# xxd /tmp/blank | tail -5
    02bfffb0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
    02bfffc0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
    02bfffd0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
    02bfffe0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
    02bffff0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
    At this point, you can literally do a dd and accomplish the task. You should try to use a read/write block size that matches erase size for the partition so that you are As Nice As Possible(tm) to the underlying flash during writes:

    root@gw:/tmp/home/root# dd if=/tmp/blank of=/dev/mtd5 bs=41472
    And hopefully that's it.

    If your mtd5 partition is a size that isn't evenly divisible by erasesize (ex. mtd3 / the rootfs partition above), then dd will work, but at the end emit something that looks like an error/warning that's scary. Off the top of my head I can't remember what the error looks like, but it usually results in a value of records in != records out, and that's OK! It just means that the input block size it read was N bytes, but what write() returned was less than N bytes, and thus when reaching the end of the device there was only a "partial write", which is OK.

    Is it possible to do this without generating a temporary file (ex. /tmp/blank), using entirely pipes? Yes, it should be possible, but I chose not to do it out of safety.

    Good luck.
  6. Bird333

    Bird333 Network Guru Member

    I was hoping you would chime in. :) I ran across this command while googling. It said it would error out when it ran out of space. Would this work?

    tr '\0' '\377' < /dev/zero | > /dev/mtd5
  7. jerrm

    jerrm Network Guru Member

    printf "%d\n" 0x2C00000
  8. koitsu

    koitsu Network Guru Member

    As I stated: I do not recommend trying to play with pipes and redirects to accomplish the goal, as we're talking about a flash partition here, where the risks seem quite high if something goes wrong.

    And no, that will not do the same thing as what dd will do. Block size matters here. Not trying to be rude or anything (honest!), but I did explain what exactly the dd command is doing and why the arguments are what they are. You will wear on your flash if you do it the way quoted above.
  9. koitsu

    koitsu Network Guru Member

    Thanks -- I always forget that some printf userland utilities can parse arguments this way. I'm always reluctant to try things on Busybox because the documentation is sometimes lacking (or lacking altogether), combined with too many years with too many UNIXes (esp. Solaris) making one do things the hard way. :)
  10. Bird333

    Bird333 Network Guru Member

    Another thought. I have backed up this partition to a .bin file. If I make a copy of this and fill it with 'FF' (it's mostly that anyway) with HxD and write this back will I accomplish the same thing? This file is 64 MB btw.
  11. koitsu

    koitsu Network Guru Member

    Yes, that should be fine too. Just make sure the file size matches that of mtd5 (it should, but verify anyway);a simple dd if=whatever of=/dev/mtd5 bs={whatever-the-erase-size-is-in-decimal} should do it.

    Edit: Not sure if you should use /dev/mtdblock5 or /dev/mtd5. Latter is a character device ("c"), former is a block device ("b"). I understand the difference between the two types, but I don't know which you'd use for this purpose. I'd suggest using whatever device string you used to back it up in the first place. :)
  12. Bird333

    Bird333 Network Guru Member

    This didn't work. It seems like it completed (512+0 in and 512+0 out) but the same data was there. Also, I got 131072 when converting 20000 to decimal not 41472.
    Last edited: Jun 2, 2018
  13. koitsu

    koitsu Network Guru Member

    I can probably explain this but not with authoritative knowledge. Here's my brain dump. I suggest reading this linearly, and not doing anything until you've read it all, as there are some findings/conclusions at the end.

    The /dev used varies -- ex. there may be a difference between /dev/mtd5 vs. /dev/mtdblock5. This has to do with how the underlying mtd(4) driver behaves. You will be surprised to see this. This is from an RT-AC56U so your situation may differe:

    root@gw:/tmp/home/root# ls -l /dev/mtd*
    crw-rw-rw-    1 root     root       90,   0 Dec 31  1969 /dev/mtd0
    crw-rw-rw-    1 root     root       90,   1 Dec 31  1969 /dev/mtd0ro
    crw-rw-rw-    1 root     root       90,   2 Dec 31  1969 /dev/mtd1
    crw-rw-rw-    1 root     root       90,   3 Dec 31  1969 /dev/mtd1ro
    crw-rw-rw-    1 root     root       90,   4 Dec 31  1969 /dev/mtd2
    crw-rw-rw-    1 root     root       90,   5 Dec 31  1969 /dev/mtd2ro
    crw-rw-rw-    1 root     root       90,   6 Dec 31  1969 /dev/mtd3
    crw-rw-rw-    1 root     root       90,   7 Dec 31  1969 /dev/mtd3ro
    crw-rw-rw-    1 root     root       90,   8 Dec 31  1969 /dev/mtd4
    crw-rw-rw-    1 root     root       90,   9 Dec 31  1969 /dev/mtd4ro
    crw-rw-rw-    1 root     root       90,  10 Dec 31  1969 /dev/mtd5
    crw-rw-rw-    1 root     root       90,  11 Dec 31  1969 /dev/mtd5ro
    brw-rw-rw-    1 root     root       31,   0 Dec 31  1969 /dev/mtdblock0
    brw-rw-rw-    1 root     root       31,   1 Dec 31  1969 /dev/mtdblock1
    brw-rw-rw-    1 root     root       31,   2 Dec 31  1969 /dev/mtdblock2
    brw-rw-rw-    1 root     root       31,   3 Dec 31  1969 /dev/mtdblock3
    brw-rw-rw-    1 root     root       31,   4 Dec 31  1969 /dev/mtdblock4
    brw-rw-rw-    1 root     root       31,   5 Dec 31  1969 /dev/mtdblock5
    This will probably come as a surprise to anyone who hasn't worked with block drivers before, ditto with no familiarity with OSes where this is super common (ex. Solaris), where you end up with several /dev entries for a single "true" device, all of which behave differently.

    I have little to no familiarity with the mtd(4) driver behaviour, so I'm grasping at straws here. That said, the above to me implies the following:

    * /dev/mtdN may in fact be the /dev with read/write capability and may in fact update/change NAND
    * /dev/mtdNro is likely the read-only version of /dev/mtdN
    * Note that both above device types are character (c), not block (b) -- this may matter depending on what userland utility is used to access them
    * /dev/mtdblockN is the block (b) device type of /dev/mtdN, but may behave differently than /dev/mtdN.

    What's overlooked a lot of the time too is that many devices get memory-mapped, i.e. I/O done to them ends up in RAM/memory space that isn't permanent. This theory is strongly supported by evidence: nvram set foo=bar will not permanently save something to NVRAM, but nvram get foo will certainly work afterward, until the router is rebooted. The way you make that change permanent is nvram commit. NVRAM is usually part of the NAND flash (it has its own partition, ex. in the above, it's /proc/mtd1).

    There may be special ioctl()s that have to be issued to "switch" the /dev/mtdXXXXXXXX device into a mode that writes to actual flash. (Edit: this turns out to be correct/true, keep reading)

    You will probably need to look deep into the Tomato source to read what utility/program is used to write a new firmware to the router, and see if that utility can be used to write to a specific partition. Details on that are below, and I'm talking about Toastman-ARM -- all of this may be different for MIPS:

    The responsible endpoint for upgrading F/W on TomatoUSB is upgrade.cgi. This is C code within Tomato's httpd webserver, and will show up in router/httpd/tomato.c's mime_handlers structure.

    upgrade.cgi has two functions used: wi_upgrade() and wo_flash(). The former is for handling input (i.e. what happens after you click Submit/Upgrade/whatever) and tends to do the heavy lifting, and the latter for handling output (i.e. what HTML/CSS/etc. the web browser gets fed afterward). Both functions are in router/httpd/upgrade.c.

    I've described this process in the past somewhere in a previous post, but I cannot find it right now.

    Digging through wi_upgrade() shows me this:

     18 #if 1
     19 #define MTD_WRITE_CMD   "mtd-write2"
     40 void wi_upgrade(char *url, int len, char *boundary)
     41 {
     42         uint8 buf[1024];
     43         const char *error = "Error reading file";
     44         int ok = 0;
     45         int n;
     46         int reset;
     48         check_id(url);
     49         reset = (strcmp(webcgi_safeget("_reset", "0"), "1") == 0);
     83         char fifo[] = "/tmp/flashXXXXXX";
     93         char *wargv[] = { MTD_WRITE_CMD, fifo, "linux", NULL };
     94         if (_eval(wargv, ">/tmp/.mtd-write", 0, &pid) != 0) {
     95                 error = "Unable to start flash program";
     96                 goto ERROR2;
     97         }
     99         if ((f = fopen(fifo, "w")) == NULL) {
    100                 error = "Unable to start pipe for mtd write";
    101                 goto ERROR2;
    102         }
    104         // !!! This will actually write the boundary. But since mtd-write
    105         // uses trx length... -- zzz
    107         while (len > 0) {
    108                  if ((n = web_read(buf, MIN(len, sizeof(buf)))) <= 0) {
    109                          goto ERROR2;
    110                  }
    111                  len -= n;
    112                  if (safe_fwrite(buf, 1, n, f) != n) {
    113                          error = "Error writing to pipe";
    114                          goto ERROR2;
    115                  }
    116         }
    118         error = NULL;
    119         ok = 1;
    121 ERROR2:
    122         rboot = 1;
    124         if (f) fclose(f);
    125         if (pid != -1) waitpid(pid, &n, 0);
    127         if (error == NULL && reset) {
    128                 set_action(ACT_IDLE);
    129                 eval("mtd-erase2", "nvram");
    130         }
    131         set_action(ACT_REBOOT);
    133         if (resmsg_fread("/tmp/.mtd-write"))
    134                 error = NULL;
    135 ERROR:
    136         if (error) resmsg_set(error);
    137         web_eat(len);
    138 }
    The command mtd-write2 /tmp/flashXXXXXX linux is issued, followed by the C code opening up /tmp/flashXXXXXX (the fifo), effectively "piping" data read from the web browser (firmware) through /tmp/flashXXXXXX into whatever mtd partition maps to the name linux. In my case, it would be mtd2, because:

    root@gw:/tmp/home/root# cat /proc/mtd
    dev:    size   erasesize  name
    mtd0: 00080000 00020000 "boot"
    mtd1: 00180000 00020000 "nvram"
    mtd2: 01e00000 00020000 "linux"
    mtd3: 01c5dc04 00020000 "rootfs"
    mtd4: 06000000 00020000 "brcmnand"
    mtd5: 04000000 00020000 "jffs2"
    We can also see that there's a command mtd-erase2 nvram used to erase NVRAM assuming that the user picked such in the GUI and there was no error during the firmware upgrade process.

    Both of these commands emit the following usage syntaxes, which say "device" but looks to me like they take a partition name, not a literal device:

    root@gw:/tmp/home/root# mtd-write2
    usage: mtd-write2 [path] [device]
    root@gw:/tmp/home/root# mtd-erase2
    usage: mtd-erase2 [device]
    So what are mtd-write2 and mtd-erase2? No surprise, they're custom Tomato code, handled entirely within Tomato rc, so source router/rc/rc.c. This also explains why on the filesystem we see:

    root@gw:/tmp/home/root# ls -l /sbin/mtd*
    lrwxrwxrwx    1 root     root             2 May 27 03:34 /sbin/mtd-erase2 -> rc
    lrwxrwxrwx    1 root     root             2 May 27 03:34 /sbin/mtd-unlock -> rc
    lrwxrwxrwx    1 root     root             2 May 27 03:34 /sbin/mtd-write2 -> rc
    Digging around in that source, we find relevant bits:

    110 #ifdef TCONFIG_BCMARM
    111         { "mtd-write",                  mtd_write_main_old              },
    112         { "mtd-erase",                  mtd_unlock_erase_main_old       },
    113         { "mtd-unlock",                 mtd_unlock_erase_main_old       },
    114 #else
    115         { "mtd-write",                  mtd_write_main                  },
    116         { "mtd-erase",                  mtd_unlock_erase_main           },
    117         { "mtd-unlock",                 mtd_unlock_erase_main           },
    118 #endif
    159 int main(int argc, char **argv)
    160 {
    234 #ifdef TCONFIG_BCMARM
    235         if (!strcmp(base, "nvram_erase")){
    236                 erase_nvram();
    237                 return 0;
    238         }
    239         /* mtd-erase2 [device] */
    240         else if (!strcmp(base, "mtd-erase2")) {
    241                 if (argv[1] && ((!strcmp(argv[1], "boot")) ||
    242                 (!strcmp(argv[1], "linux")) ||
    243                 (!strcmp(argv[1], "linux2")) ||
    244                 (!strcmp(argv[1], "rootfs")) ||
    245                 (!strcmp(argv[1], "rootfs2")) ||
    246                 (!strcmp(argv[1], "nvram")))) {
    248                 return mtd_erase(argv[1]);
    249         } else {
    250                 fprintf(stderr, "usage: mtd-erase2 [device]\n");
    251                 return EINVAL;
    252                 }
    253         }
    254         /* mtd-write2 [path] [device] */
    255         else if (!strcmp(base, "mtd-write2")) {
    256         if (argc >= 3)
    257                 return mtd_write(argv[1], argv[2]);
    258         else {
    259                 fprintf(stderr, "usage: mtd-write2 [path] [device]\n");
    260                 return EINVAL;
    261                 }
    262         }
    263 #endif
    mtd-unlock looks interesting, and also relevant to what I said earlier.

    Here we have proof that MIPS and ARM differ as well. But we can see that mtd_erase() and mtd_write() are the underlying C functions that do the heavy lifting. We can also see that for mtd_erase(), the utility only lets you erase things by name -- NOT BY ACTUAL /dev DEVICE.

    The code for mtd_erase() and mtd_write() is in router/rc/mtd.c, and has a large copyright atop of indicating it's Broadcom code. *** HERE BE DRAGONS ***

    At this stage, I would recommend skimming through that code. It's going to be hairy, I can assure you. Yes I have skimmed it -- no I'm not going to do a write-up because it would take me probably a few hours. But skimming that code does confirm that it wants partition names, not literal /dev paths. That code also has mtd API functions used, because like I said, the mtd(4) driver is responsible. Linux and BSD differ sometimes in how userland communicates with kernel, so it could be done in any number of ways, but I'm used to seeing ioctl()s issued. The code does look to use ioctls() to accomplish this (MEMUNLOCK, MEMERASE, etc.).

    Thus, the dd may have actually resulted in data being thrown away silently (because of lack of unlocking, or because the mtd(4) driver expects you to use its API, or issue certain commands before/after certain operations are done.

    In short: it doesn't look like you can just dd directly to the device. Instead, you must use Broadcom tools.

    My suggestion: use mtd-erase2 whatever-the-name-of-the-partition-is and see what happens. Broadcom's code in router/rc/mtd.c for erasing has very unique situations, including handling bad blocks and all sorts of other goodies, including some headers and god knows what else. As I said: hairy.

    It looks like this should suffice for "erasing a partition", and not require you to populate a partition filled with a raw bunch of 0xFFs. I don't know where you got that value from, but my gut feeling is that you're wanting 0xFF because that's what EPROM and EEPROMs require rather than zeros. NAND flash may be different in this regard. But regardless if it's the same or different, I think in this case it'd be easiest to just use the vendors' own erase utility/code to do the erasing for you.

    If you still want to write 0xFFs to a partition, then you'll need to do as discussed: make a file that matches the partition size in bytes, fill it with 0xFFs, then probably use mtd-write2 /path/to/file partition-name to write it. Error messages/etc. you see are coming directly from code in router/rc/mtd.c written by Broadcom, so you will need to refer to that code to figure out what may go wrong (if anything).

    Good luck and let us know how this turns out!

    Then it's a good thing you followed my instructions rather than just copy-pasted! :) *thumbs up*

    I pulled 41472 out of my butt, because I wrote my instructions using some false/fake erase sizes -- I used 0xA200 -- then later went back and rephrased/rewrote parts but did not correct my example. This is quite common with documentation I write, because I'm thorough, and people tend to ask "simple" questions that involve long answers.
  14. Sean B.

    Sean B. LI Guru Member

    Am I correcting in guessing what you're trying to do is erase the T-Mobile partition on mtd5 to prevent Asus detecting it as converted to an RT-AC68U and downgrading + locking the CFE?

    If so:

    #Write all FFs to the mdt5 (in case that didn't happen when you removed the TMO partition in the previous steps)
    admin@RT-AC68U:/tmp/home/root# ln -s /sbin/rc mtd-erase
    admin@RT-AC68U:/tmp/home/root# ./mtd-erase -d asus
    Supposedly this is how. Try at your own risk, I make no claim as to it's accuracy nor responsibility for what comes of it.
    koitsu likes this.
  15. Bird333

    Bird333 Network Guru Member

    Yes.;) At this point I was going to just install Merlin and do the required steps but I can't seem to get Merlin or OEM firmware to upload even using the Asus recovery tool.
  16. koitsu

    koitsu Network Guru Member

    I don't bother using the Asus Recovery Tool -- I've literally never had it work. I've gotten TFTP to work many times over, but it's finicky as we all know. Instead, what seems to work reliably is the built-in miniCFE web server ("CFE miniWeb Server").
  17. Sean B.

    Sean B. LI Guru Member

    So are you actually running Tomato now? Or are you still on T-Mobile firmware and unable to load 3rd party builds?
  18. Bird333

    Bird333 Network Guru Member

    Yeah, I'm on Tomato. I've started a new thread.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice