1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

GPT Cookbook?

Discussion in 'Tomato Firmware' started by asterger, Jul 29, 2012.

  1. asterger

    asterger Network Guru Member

    Anyone have a recipe for formatting a usable GPT disk for Tomato? I have used several Linux partitioning tools none of which allow Tomato to recognized my 4TB USB drive.

    Thanks on advance for any assistance.

    -- asterger

    From the log:
    Code:
    ul 28 15:58:57 foobar user.info kernel: usb 1-1: new high speed USB device using ehci_hcd and address 4
    Jul 28 15:58:57 foobar user.info kernel: usb 1-1: configuration #1 chosen from 1 choice
    Jul 28 15:58:57 foobar user.info kernel: scsi2 : SCSI emulation for USB Mass Storage devices
    Jul 28 15:58:57 foobar user.debug hotplug[15294]: Attached USB device 1-1:1.0 [INTERFACE=8/6/80 PRODUCT=152d/2509/100]
    Jul 28 15:58:57 foobar user.info kernel: Registered led device: 1-1
    Jul 28 15:58:59 foobar user.notice kernel: scsi 2:0:0:0: Direct-Access    JMicron  RAID-0          0000 PQ: 0 ANSI: 5
    Jul 28 15:58:59 foobar user.notice kernel: sd 2:0:0:0: [sda] 976757292 4096-byte hardware sectors (4000798 MB)
    Jul 28 15:58:59 foobar user.notice kernel: sd 2:0:0:0: [sda] Write Protect is off
    Jul 28 15:58:59 foobar user.debug kernel: sd 2:0:0:0: [sda] Mode Sense: 28 00 00 00
    Jul 28 15:58:59 foobar user.err kernel: sd 2:0:0:0: [sda] Assuming drive cache: write through
    Jul 28 15:58:59 foobar user.notice kernel: sd 2:0:0:0: [sda] 976757292 4096-byte hardware sectors (4000798 MB)
    Jul 28 15:58:59 foobar user.err kernel: sd 2:0:0:0: [sda] Assuming drive cache: write through
    Jul 28 15:58:59 foobar user.info kernel:  sda:<6>sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 15:58:59 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 15:58:59 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 15:58:59 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 15:59:00 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 15:59:00 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 04 00
    Jul 28 15:59:00 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 15:59:00 foobar user.err kernel: Buffer I/O error on device sda, logical block 0
    Jul 28 15:59:00 foobar user.err kernel: Buffer I/O error on device sda, logical block 1
    Jul 28 15:59:00 foobar user.err kernel: Buffer I/O error on device sda, logical block 2
    Jul 28 15:59:00 foobar user.err kernel: Buffer I/O error on device sda, logical block 3
    Jul 28 15:59:00 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 15:59:00 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 15:59:00 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 15:59:00 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 15:59:00 foobar user.err kernel: Buffer I/O error on device sda, logical block 0
    Jul 28 16:00:52 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:00:52 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:00:52 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 04 00
    Jul 28 16:00:52 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:00:52 foobar user.err kernel: Buffer I/O error on device sda, logical block 0
    Jul 28 16:00:52 foobar user.err kernel: Buffer I/O error on device sda, logical block 1
    Jul 28 16:00:52 foobar user.err kernel: Buffer I/O error on device sda, logical block 2
    Jul 28 16:00:52 foobar user.err kernel: Buffer I/O error on device sda, logical block 3
    Jul 28 16:00:52 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:00:52 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:00:52 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 16:00:52 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:00:52 foobar user.err kernel: Buffer I/O error on device sda, logical block 0
    Jul 28 16:01:01 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:01 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:01 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 04 00
    Jul 28 16:01:01 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:01 foobar user.err kernel: Buffer I/O error on device sda, logical block 0
    Jul 28 16:01:01 foobar user.err kernel: Buffer I/O error on device sda, logical block 1
    Jul 28 16:01:01 foobar user.err kernel: Buffer I/O error on device sda, logical block 2
    Jul 28 16:01:01 foobar user.err kernel: Buffer I/O error on device sda, logical block 3
    Jul 28 16:01:01 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:01 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:01 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 16:01:01 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:01 foobar user.err kernel: Buffer I/O error on device sda, logical block 0
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 04 00
    Jul 28 16:01:05 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:05 foobar user.err kernel: Buffer I/O error on device sda, logical block 0
    Jul 28 16:01:05 foobar user.err kernel: Buffer I/O error on device sda, logical block 1
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 16:01:05 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 04 00
    Jul 28 16:01:05 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:05 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 16:01:05 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:07 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:07 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:07 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 04 00
    Jul 28 16:01:07 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:07 foobar user.warn kernel: printk: 8 messages suppressed.
    Jul 28 16:01:07 foobar user.err kernel: Buffer I/O error on device sda, logical block 0
    Jul 28 16:01:07 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:07 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:07 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 16:01:07 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 04 00
    Jul 28 16:01:17 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:17 foobar user.warn kernel: printk: 4 messages suppressed.
    Jul 28 16:01:17 foobar user.err kernel: Buffer I/O error on device sda, logical block 0
    Jul 28 16:01:17 foobar user.err kernel: Buffer I/O error on device sda, logical block 1
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 16:01:17 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 04 00
    Jul 28 16:01:17 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:17 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 16:01:17 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:21 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:21 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:21 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 04 00
    Jul 28 16:01:21 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 16:01:21 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 16:01:21 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 16:01:21 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 16:01:21 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    
     
  2. koitsu

    koitsu Network Guru Member

    I don't see any indication that the problem you're seeing has anything to do with a GPT partition scheme vs. an MBR partition scheme. What I see from the kernel log looks to be a communication error between your generic/cheap "NAS" driven by a JMicron USB-to-SATA bridge, and the underlying router using Linux. Note that all the I/O errors are for accessing the first LBA (LBA 0, a.k.a. sector 0). Thus this has nothing to do with GPT vs. MBR.

    There are two possibilities as I see them:

    1. There may be needed kernel changes to introduce what are known as "quirks" for certain storage devices using USB interfaces. I can tell the generic/cheap "NAS" you have is some JMicron-based thing using RAID-0 (I hope you do backups regularly; a single I/O error or disk issue will make you lose 4TB of data). These quirks are configured on a per-device basis, meaning one may need to be written for JMicron devices matching that particular USB product/model/class code. A newer version of the Linux kernel might already provide this.

    Similar quirks may be needed to deal with 4KByte sector sizes (which your device does claim to need). I'm not talking about the sector size of each disk in your generic/cheap "NAS", I'm talking about the sector size that the underlying NAS itself advertises across USB.

    2. There may be issues with the USB-to-SCSI translation code in Linux pertaining to devices that are more than exactly 4,000,000MBytes in size. To test out this theory, make a 2TB array instead of 4TB (e.g. if you have 4 1TB disks to make a 4TB array, remove 2 disks and rebuild the array). See if that works. If it doesn't, then the problem is issue #1 described above.

    Good luck.
     
  3. asterger

    asterger Network Guru Member

    USB device is actually a Fantom Drive G-Force3 MegaDisk containing 2-2TB disks that can be configured as RAID0, RAID1, JBOD or SPAN (concatenated). It wasn't cheap IMO. Fantom Drives/Micronet uses third-party JMicron JMS539 USB-to-SATA bridge. Although I agree, the JMicron USB-to-SATA bridge is problematic to Linux, Windows Vista flawlessly detected the RAID0 (4TB) configured device and formatted as GPT disk also using JMicron drivers.

    Yes I have the device working in Tomato as RAID1, formatted as ext3, at 2TB. But would like to have the device's full 4TB capacity available. The drive will be used for short term backup retention. As I'm in test phase, there isn't much data to worry about.

    I can work with anyone who can suggest kernel quirks or I/O parameter changes that will enable GPT disk configuration.
    -- asterger

    Code:
    root@foobar:/tmp/home/root# lsusb
    Bus 001 Device 003: ID 152d:2509  <<<
    Bus 001 Device 001: ID 1d6b:0002
    
    And:

    Code:
    root@foobar:/tmp/home/root# cat /proc/bus/usb/devices
     
    T:  Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480 MxCh= 2
    B:  Alloc=  0/800 us ( 0%), #Int=  0, #Iso=  0
    D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=00 MxPS=64 #Cfgs=  1
    P:  Vendor=1d6b ProdID=0002 Rev= 2.06
    S:  Manufacturer=Linux 2.6.22.19 ehci_hcd
    S:  Product=EHCI Host Controller
    S:  SerialNumber=0000:00:04.1
    C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
    I:* If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
    E:  Ad=81(I) Atr=03(Int.) MxPS=  4 Ivl=256ms
     
    T:  Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#=  3 Spd=480 MxCh= 0
    D:  Ver= 2.10 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs=  1
    P:  Vendor=152d ProdID=2509 Rev= 1.00
    S:  Manufacturer=DMI    External HDD
    S:  Product=Usb production
    S:  SerialNumber=2011090602EA
    C:* #Ifs= 1 Cfg#= 1 Atr=c0 MxPwr=  2mA
    I:* If#= 0 Alt= 0 #EPs= 2 Cls=08(stor.) Sub=06 Prot=50 Driver=usb-storage
    E:  Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
    E:  Ad=02(O) Atr=02(Bulk) MxPS= 512 Ivl=0ms
     
  4. koitsu

    koitsu Network Guru Member

    I guess I wasn't very clear in my last message. It appears that for whatever reason the
    kernel can't read certain LBAs -- actual I/O errors are returned. This has nothing to do
    with GPT support or MBR support (yes I'm aware to get >2TB support you need to use GPT). I'm
    incredibly familiar with storage protocols (particularly ATA, and a bit of SCSI as well)
    so this stuff isn't "magic" to me, but is to most people.

    Look closely here:

    Code:
    Jul 28 15:58:59 foobar user.notice kernel: sd 2:0:0:0: [sda] 976757292 4096-byte hardware sectors (4000798 MB)
    Jul 28 15:58:59 foobar user.err kernel: sd 2:0:0:0: [sda] Assuming drive cache: write through
    Jul 28 15:58:59 foobar user.info kernel:  sda:<6>sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 15:58:59 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 15:58:59 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    Jul 28 15:58:59 foobar user.info kernel: sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 15:59:00 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 15:59:00 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 04 00
    Jul 28 15:59:00 foobar user.warn kernel: end_request: I/O error, dev sda, sector 0
    Jul 28 15:59:00 foobar user.err kernel: Buffer I/O error on device sda, logical block 0
    Jul 28 15:59:00 foobar user.err kernel: Buffer I/O error on device sda, logical block 1
    Jul 28 15:59:00 foobar user.err kernel: Buffer I/O error on device sda, logical block 2
    Jul 28 15:59:00 foobar user.err kernel: Buffer I/O error on device sda, logical block 3
    
    The first CDB submit from the Linux kernel to the controller (JMicron / Fantom thing) results in an error of some kind. I don't quite understand what "hostbyte" and "driverbyte" represent here (I would need to look at the Linux kernel source to figure this out), but the CDB can be decoded.

    Linux uses SCSI emulation for both USB-attached disks as well as natively-attached SATA disks, so there is a "translation layer" that happens within the kernel to make sure that actual/true SCSI CDBs (which ATA drives do not understand) get "converted" into their respective ATA CDBs. One of the complexities here is that the CDB messages are almost certainly the SCSI equivalent and not necessarily what the underlying device (controller on the remote end) sees. I don't know how to get Linux to output what it really intends to send to the device.

    SCSI CDBs are always in big endian.

    So let's decode the first 10-byte CDB which gets sent / receives some kind of error:

    Code:
    Jul 28 15:58:59 foobar user.info kernel:  sda:<6>sd 2:0:0:0: [sda] Unhandled error code
    Jul 28 15:58:59 foobar user.info kernel: sd 2:0:0:0: [sda] Result: hostbyte=0x07 driverbyte=0x10
    Jul 28 15:58:59 foobar user.info kernel: sd 2:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 00 00 00 00 00 00 01 00
    
    Code:
    Byte 0 = 0x28 = SCSI READ EXTENDED command, which can address up to 32-bit LBAs.
                    32-bit LBA support means an addressing maximum of 2048GB, or 2TB.
                    But that limitation doesn't apply to this request (per se) because
                    it's trying to read LBA 0.  More on that in a moment.
     
    Byte 1 = 00 =
     
      Bits 7-5 = %000 = LUN number (in this case 0)
      Bit 4    = %0  = disable page out (so DPO is enabled)
      Bit 3    = %0  = forced unit access (so FUA is disabled); FUA is
                        basically direct-disk reading, e.g. telling the drive to not
                        use its on-board cache),
      Bits 2-1 = %00  = reserved/unused
      Bit 0    = %0  = relative addressing (defines whether or not the LBA offset
                        in the next byte is relative or literal; rather not get into
                        explaining this)
     
    Bytes 2-5 = 00 00 00 00 = LBA number (32-bit).
                              Like I said, it's trying to read LBA 0 (sector 0).
     
    Byte 6 = 00    = reserved/unused
     
    Bytes 7-8 = 00 01 = transfer length (16-bit).  Value is in big endian as
                        I said above, so the number of sectors being requested
                        to be read is 1.  Sector size, as admitted earlier by
                        the kernel in its log, is 4096.
     
    Byte 9 = 00    = control byte
    
    This request, which is quite possibly the simplest LBA read possible (aside from the classic 21-bit LBA read command (SCSI READ) which is a 6-byte CDB; very very old!), fails. Why it fails is anyone's guess.

    The next CDB sent to the drive is the exact same, except it asks to read 4 sectors (LBA 0 through 3), and that also fails with the same result.

    If you use RAID-1, thus limiting the array to 2TB in size, can you provide dmesg output from when you do that? I wonder if the underlying JMicron controller uses 512-byte sectors for 2TB volumes or less, and uses 4096-byte sectors for >2TB, and thus the bug would be with the Linux kernel possibly not doing the right thing with 4096-byte sectors.

    Remember: the Linux kernel used in Tomato is extremely old and is not even remotely the latest 2.6.xx kernel available. So there may be a bug with 4096-byte sector support in the version of the kernel used. If that turns out to be the case, you're absolutely SOL.

    The important thing to take away from what I've written here is that this has nothing to do with GPT vs. MBR or any other partition scheme method. The problem is that Linux, when talking to the JMicron controller on you SATA/USB bridge device, cannot read any sectors from the underlying volume/array. Take a look at the GPT documentation on Wikipedia, specifically the diagram on the right, and you'll see which LBAs are used for what. Then ask yourself: if the kernel can't read any sectors/LBAs from the underlying device, then how/why does partition schemes even matter?

    Finally I want to point out something: earlier in the kernel log (before the LBA reading occurs), SCSI MODE SENSE is sent to the device. That works correctly (and is how the kernel figures out that the underlying device uses 4096-byte sectors :) ). Meaning: the Linux kernel and the JMicron device can indeed talk successfully; it's just LBA/sector reading that fails. This could be, as I said, a Linux kernel bug (could be with the USB/SATA translation layer in the Linux kernel, could be with USB quirks needing to make certain models of hardware devices compatible with the kernel (this is very common)), or it could be some very wonky/screwed up idiocy on the JMicron device. Just because a consumer-grade device supports multitudes of RAID models does not mean it's a good product, no matter how much it costs. I can point you to all sorts of retarded bugs on even more expensive devices like the Drobo and ReadyNAS.

    Now you understand why I recommended using RAID-1 or simply JBOD (two 2TB disks) to see if this works around the problem. Knowing where the problem is is difficult to truly determine, but my above analysis indicates the kernel and the JMicron device can talk, just that sector/LBA reads do not work, at least when using a 4TB volume. I wouldn't be surprised if this was a kernel bug, nor would I be surprised if it was a firmware bug on the JMicron device.

    Just remember: vendors of these products more often than not do not test their product on any OS other than Windows. Even if they say "works under Linux" it probably means one idiotic guy at the company plugged it in to some random Linux machine he had and "wow the kernel shows it appearing" and thus "it works with Linux". Welcome to how crappy QC/QA is these days.
     
  5. asterger

    asterger Network Guru Member

    Koitsu,

    Thank you for the informative reply. I truly appreciate the completeness. The working 2TB RAID-1 configuration dmesg below. While Fantom factory formats as NTFS, they also promote reformatting to HFS for MacOSX.

    Code:
    =~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2012.07.30 20:40:52 =~=~=~=~=~=~=~=~=~=~=~=
    dmesg
    Linux version 2.6.22.19 (root@tomato) (gcc version 4.2.4) #11 Sat Mar 31 00:14:56 ICT 2012
    CPU revision is: 00019740
    Determined physical RAM map:
    memory: 04000000 @ 00000000 (usable)
    Entering add_active_range(0, 0, 16384) 0 entries of 256 used
    Zone PFN ranges:
      Normal          0 ->    16384
      HighMem    16384 ->    16384
    early_node_map[1] active PFN ranges
        0:        0 ->    16384
    On node 0 totalpages: 16384
      Normal zone: 0 pages used for memmap
      Normal zone: 0 pages reserved
      Normal zone: 16384 pages, LIFO batch:3
      HighMem zone: 0 pages used for memmap
    Built 1 zonelists.  Total pages: 16384
    Kernel command line: root=/dev/mtdblock2 noinitrd console=ttyS0,115200
    Primary instruction cache 32kB, physically tagged, 4-way, linesize 32 bytes.
    Primary data cache 32kB, 4-way, linesize 32 bytes.
    Synthesized TLB refill handler (20 instructions).
    Synthesized TLB load handler fastpath (32 instructions).
    Synthesized TLB store handler fastpath (32 instructions).
    Synthesized TLB modify handler fastpath (31 instructions).
    PID hash table entries: 512 (order: 9, 2048 bytes)
    CPU: BCM4716 rev 1 pkg 10 at 480 MHz
    Using 240.000 MHz high precision timer.
    console [ttyS0] enabled
    Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
    Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
    Memory: 61304k/65536k available (33k kernel code, 4172k reserved, 2701k data, 124k init, 0k highmem)
    Calibrating delay loop... 239.20 BogoMIPS (lpj=1196032)
    Mount-cache hash table entries: 512
    NET: Registered protocol family 16
    PCI: Using membase 8000000
    PCI: Initializing host
    PCI: Reset RC
    PCI: no core
    PCI: Fixing up bus 0
    PCI/PCIe coreunit 0 is set to bus 1.
    PCI: Fixing up bridge
    PCI: Setting latency timer of device 0000:01:00.0 to 64
    PCI: Fixing up bridge
    PCI: Setting latency timer of device 0000:01:00.1 to 64
    PCI: Enabling device 0000:01:00.1 (0004 -> 0006)
    PCI: Fixing up bus 1
    NET: Registered protocol family 2
    Time: MIPS clocksource has been installed.
    IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
    TCP established hash table entries: 2048 (order: 2, 16384 bytes)
    TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
    TCP: Hash tables configured (established 2048 bind 2048)
    TCP reno registered
    squashfs: version 3.0 (2006/03/15) Phillip Lougher
    io scheduler noop registered (default)
    HDLC line discipline: version $Revision: 4.8 $, maxframe=4096
    N_HDLC line discipline registered.
    Serial: 8250/16550 driver $Revision: 1.90 $ 2 ports, IRQ sharing disabled
    serial8250: ttyS0 at MMIO 0xb8000300 (irq = 8) is a 16550A
    PPP generic driver version 2.4.2
    MPPE/MPPC encryption/compression module registered
    NET: Registered protocol family 24
    PPPoL2TP kernel driver, V0.17
    PPTP driver version 0.8.5
    Physically mapped flash: Found 1 x16 devices at 0x0 in 8-bit bank
    Physically mapped flash: Found an alias at 0x800000 for the chip at 0x0
    Physically mapped flash: Found an alias at 0x1000000 for the chip at 0x0
    Physically mapped flash: Found an alias at 0x1800000 for the chip at 0x0
    Amd/Fujitsu Extended Query Table at 0x0040
    number of CFI chips: 1
    cfi_cmdset_0002: Disabling erase-suspend-program due to code brokenness.
    Flash device: 0x800000 at 0x1c000000
    Creating 5 MTD partitions on "Physically mapped flash":
    0x00000000-0x00040000 : "pmon"
    0x00040000-0x007f0000 : "linux"
    0x00127000-0x00740000 : "rootfs"
    0x00740000-0x007f0000 : "jffs2"
    0x007f0000-0x00800000 : "nvram"
    Found an  serial flash with 0 0KB blocks; total size 0MB
    sflash: found no supported devices
    u32 classifier
        OLD policer on
    Netfilter messages via NETLINK v0.30.
    nf_conntrack version 0.5.0 (512 buckets, 4096 max)
    ip_tables: (C) 2000-2006 Netfilter Core Team
    ipt_account 0.1.21 : Piotr Gasidlo <quaker@barbara.eu.org>, http://www.barbara.eu.org/~quaker/ipt_account/
    net/ipv4/netfilter/tomato_ct.c [Mar 31 2012 00:11:10]
    NET: Registered protocol family 1
    NET: Registered protocol family 10
    ip6_tables: (C) 2000-2006 Netfilter Core Team
    NET: Registered protocol family 17
    802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
    All bugs added by David S. Miller <davem@redhat.com>
    VFS: Mounted root (squashfs filesystem) readonly.
    Freeing unused kernel memory: 124k freed
    Warning: unable to open an initial console.
    emf: module license 'Proprietary' taints kernel.
    PCI: Setting latency timer of device 0000:00:02.0 to 64
    eth0: Broadcom BCM47XX 10/100/1000 Mbps Ethernet Controller 5.100.138.9
    wl_module_init: passivemode set to 0x0
    PCI: Setting latency timer of device 0000:00:01.0 to 64
    eth1: Broadcom BCM4329 802.11 Wireless Controller 5.100.138.9
    PCI: Enabling device 0000:01:01.0 (0000 -> 0002)
    PCI: Setting latency timer of device 0000:01:01.0 to 64
    eth2: Broadcom BCM4322 802.11 Wireless Controller 5.100.138.9
    Algorithmics/MIPS FPU Emulator v1.5
    usbcore: registered new interface driver usbfs
    usbcore: registered new interface driver hub
    usbcore: registered new device driver usb
    SCSI subsystem initialized
    Initializing USB Mass Storage driver...
    usbcore: registered new interface driver usb-storage
    USB Mass Storage support registered.
    ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
    PCI: Setting latency timer of device 0000:00:04.1 to 64
    ehci_hcd 0000:00:04.1: EHCI Host Controller
    ehci_hcd 0000:00:04.1: new USB bus registered, assigned bus number 1
    ehci_hcd 0000:00:04.1: irq 5, io mem 0x18004000
    ehci_hcd 0000:00:04.1: USB 0.0 started, EHCI 1.00
    usb usb1: configuration #1 chosen from 1 choice
    hub 1-0:1.0: USB hub found
    hub 1-0:1.0: 2 ports detected
    usbcore: registered new interface driver usblp
    usblp: USB Printer Device Class driver
    usb 1-1: new high speed USB device using ehci_hcd and address 2
    vlan1: add 33:33:00:00:00:01 mcast address to master interface
    vlan1: add 01:00:5e:00:00:01 mcast address to master interface
    vlan1: dev_set_allmulti(master, 1)
    vlan1: dev_set_promiscuity(master, 1)
    device eth0 entered promiscuous mode
    device vlan1 entered promiscuous mode
    device eth1 entered promiscuous mode
    device eth2 entered promiscuous mode
    br0: port 3(eth2) entering forwarding state
    br0: port 2(eth1) entering forwarding state
    br0: port 1(vlan1) entering forwarding state
    vlan2: Setting MAC address to  98 fc 11 8e e0 55.
    vlan2: add 33:33:00:00:00:01 mcast address to master interface
    vlan2: add 01:00:5e:00:00:01 mcast address to master interface
    IMQ starting with 3 devices...
    IMQ driver loaded successfully.
            Hooking IMQ after NAT on PREROUTING.
            Hooking IMQ before NAT on POSTROUTING.
    usb 1-1: device descriptor read/8, error -71
    usb 1-1: device descriptor read/8, error -71
    br0: port 2(eth1) entering disabled state
    br0: port 2(eth1) entering forwarding state
    usb 1-1: new high speed USB device using ehci_hcd and address 3
    br0: port 3(eth2) entering disabled state
    br0: port 3(eth2) entering forwarding state
    usb 1-1: configuration #1 chosen from 1 choice
    scsi0 : SCSI emulation for USB Mass Storage devices
    Registered led device: 1-1
    scsi 0:0:0:0: Direct-Access    JMicron  RAID-1          0000 PQ: 0 ANSI: 2 CCS
    sd 0:0:0:0: [sda] 3907029152 512-byte hardware sectors (2000399 MB)
    sd 0:0:0:0: [sda] Write Protect is off
    sd 0:0:0:0: [sda] Mode Sense: 28 00 00 00
    sd 0:0:0:0: [sda] Assuming drive cache: write through
    sd 0:0:0:0: [sda] Assuming drive cache: write through
    sda: sda1
    sd 0:0:0:0: [sda] Attached SCSI disk
    kjournald starting.  Commit interval 5 seconds
    EXT3 FS on sda1, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    
     
  6. koitsu

    koitsu Network Guru Member

    Yeah, notice the difference:

    Code:
    scsi 0:0:0:0: Direct-Access    JMicron  RAID-1          0000 PQ: 0 ANSI: 2 CCS
    sd 0:0:0:0: [sda] 3907029152 512-byte hardware sectors (2000399 MB)
    sd 0:0:0:0: [sda] Write Protect is off
    sd 0:0:0:0: [sda] Mode Sense: 28 00 00 00
    sd 0:0:0:0: [sda] Assuming drive cache: write through
    sd 0:0:0:0: [sda] Assuming drive cache: write through
    sda: sda1
    sd 0:0:0:0: [sda] Attached SCSI disk
    
    Code:
    Jul 28 15:58:59 foobar user.notice kernel: scsi 2:0:0:0: Direct-Access    JMicron  RAID-0          0000 PQ: 0 ANSI: 5
    Jul 28 15:58:59 foobar user.notice kernel: sd 2:0:0:0: [sda] 976757292 4096-byte hardware sectors (4000798 MB)
    Jul 28 15:58:59 foobar user.notice kernel: sd 2:0:0:0: [sda] Write Protect is off
    Jul 28 15:58:59 foobar user.debug kernel: sd 2:0:0:0: [sda] Mode Sense: 28 00 00 00
    Jul 28 15:58:59 foobar user.err kernel: sd 2:0:0:0: [sda] Assuming drive cache: write through
    
    With a 2TB volume, the sector size advertised by the JMicron USB/SATA bridge is 512-bytes.

    With a 4TB volume, the sector size advertised by the JMicron USB/SATA bridge is 4096-bytes.

    Furthermore, in the first example (512-byte sectors), the drive reports no I/O errors thus all LBAs are readable/accessible. This is why the kernel says "sda: sda1" -- it's able to read the partition table since reading LBAs works. With 4K sectors, this doesn't happen and I/O errors are returned.

    I'm not too surprised they use 4K sectors for a 4TB volume, but I would say that it's fairly likely the bug lies there. So again, this is either a Linux kernel bug (again the kernel is very, VERY old -- a few years before 4K-sector drives even existed) with 4K-sector USB-attached drives, or the JMicron controller claims 4K sectors but actually wants 512 (this would be a firmware bug in the USB/SATA bridge). I'm inclined to think the bug in is in the kernel.

    The only recommendation I can give is to try something like OpenWRT which has a newer Linux kernel and see if it behaves the same way -- and that's assuming OpenWRT works on your model of router. I have an Asus RT-N16 and there are beta builds which do work with it, but the GUI features are pretty shitty; that is to say, lots and lots of bugs/wonky behaviour. CLI is much different (much better than Tomato IMO).
     
  7. asterger

    asterger Network Guru Member

    Koitsu,

    Yes noticed the sector size difference. Assumed the kernel would adjust sector size automatically based on the device's advertised size. Aren't all "modern" GPT disks at 4096-byte sectors?

    Since the kernel is not adjusting, what about manual tuning /proc/bus/usb/devices underlying parameters?

    Don't mind testing OpenWRT, however, I have lots of history with Tomato. I'd rather cobble a solution together on a platform I'm familiar with, than to start over.

    Thanks for your understanding.
     
  8. koitsu

    koitsu Network Guru Member

    There is no such thing as a "GPT disk". GPT is a partitioning scheme/model, just like MBR. A disk is a disk; you can use whatever partitioning scheme (GPT or MBR) you want. The problem is that MBR was engineered with a 2TB limitation per partition; GPT solves this by permitting a much larger partition length. It has nothing to do with the disk or even the device that the disk attaches to. Again, refer to the document I pointed you to at Wikipedia to understand what GPT is. Please make sure you understand the terminology correctly (I say this politely); misuse of terms can/will result in massive confusion.

    If you mean to say "large disks" -- no, not all disks today use 4096-byte sectors. Some do, some don't. I have some 2TB disks that use 512-byte sectors and were manufactured a couple years ago. Tons and tons and tons of software (and kernels, including that of Solaris, FreeBSD, and many others) had to be fixed when 4096-byte sector disks were introduced into the market. Almost all software on the planet was written with the assumption of 512-byte sectors. I cannot stress this fact enough. I've been using FreeBSD since the later 2.2.x series (that would be roughly 1997) and I've watched kernel commits come through over the past few years to fix 4096-byte sector support. In fact, there were even fixes/commits for this sort of thing just a few months ago (in FreeBSD 9.x) -- so do not think that just because something has existed for a few years that "everything else will just magically work", because that just isn't the case. I'm using FreeBSD as an example because I follow it/use it, while I do not follow Linux as much. But the same problem/issue/concept I described applies to all OSes.

    The problem here is that you aren't really using ""a disk"" per se. You're using a USB/SATA bridge device that either stripes/mirrors/concatenates/merges/whateveryouwanttocallit multiple physical disks into a volume that appears (to the underlying kernel) as a single disk. The underlying kernel has no real idea what's going on upstream; it only knows what the USB/SATA bridge chip (JMicron) tells it. I use the term "volume" to refer to either JBOD, RAID-0, RAID-1, RAID-5, or whatever else that JMicron USB/SATA bridge is configured as.

    We know with absolute certainty that 2TB volumes work; you proved that with RAID-1 (mirror) and two 2TB disks.

    The only underlying difference that I can see between 2TB and 4TB volumes is that the JMicron USB/SATA bridge advertises 4096-byte sectors for the latter, and 512-byte sectors for the former. LBA accesses fail in the latter case, while works fine with the former. That's all I can determine.

    /proc/bus/usb/devices has nothing in it that will fix this problem (that I know of). The problem is probably within either the USB-to-SCSI translation layer within the kernel (it may be that the kernel says "yeah, MODE SENSE resulted in the device saying it uses 4096-byte sectors", but that the underlying USB-to-SCSI translation layer code in the kernel still internally uses/assumes 512 bytes), or purely with the SCSI-to-ATA translation layer (same issue may be happening). I am completely, 100%, unable to step you through how to determine exactly where the issue lies. Embedded routers do not have decent debugging tools/environments; they just don't. We're talking about kernel-level debugging here, not userland daemons/services/programs.

    The reason I recommended trying OpenWRT is because the kernel OpenWRT uses is newer. If your 4TB volume works on OpenWRT, then that is pure 100% confirmation that the bug is in the Linux kernel used on Tomato/TomatoUSB. Again: the Linux kernel used on Tomato/TomatoUSB is extremely old, and was written/maintained before 4096-byte sector disks (or devices) were available. So the likelihood of this being a kernel bug is high. There is no workaround other than to fix the kernel, and the likelihood of that on TomatoUSB is very, very low given the number of custom patches that have to be applied to it. I'm not saying "run OpenWRT", I'm saying "try OpenWRT and see if the problem exists there too. If it doesn't, then we know it's a kernel bug."
     

Share This Page