New "Rebuild Assist" feature in Serial ATA Revision 3.2

[Cris]

I was doing some research and noticed (probably old news for some) there is a new "Rebuild Assist" feature in "Serial ATA Revision 3.2" which is intended to improve RAID rebuilds when a drive fails, but could also be beneficial for us in data recovery.

Although the "Serial ATA Revision 3.2" document is not published freely by Serial ATA International Organization, there is some interesting information around the place:

http://www.seagate.com/files/staticfile ... covery.pdf

"...Upon this command, the drive will eliminate any unnecessary background activity, determine if it contains any unusable heads, write-protect the media, and enter a special mode that includes limiting error recovery to the free retries. This mode will stay active until the drive is power-cycled..."

"...As a contributing member to leading industry standards bodies, Seagate has submitted an open-standards proposal of the RAID Rebuild™ functionality under the name Rebuild Assist to the T103 (SAS Proposal: 11-298) and SATA-IO4 (SATA Proposal: SATA31_TPR_D144) committees for inclusion in their published standards specification..."

http://www.hgst.com/tech/techlib.nsf/te ... Rev1.0.pdf

"...The Rebuild Assist mode provides a method for a host controlling the rebuild process to determine that logical sectors on the failed device are unreadable without having to read every LBA to determine the unreadable logical sectors (i.e., the read command is terminated with an error and the failed LBA is reported in the sense data)..."

I wonder if we will start to see hardware imagers supporting this feature.

[maximus]

I wonder if we will start to see hardware imagers supporting this feature.

It has been 4 years since this post. So do any of the hardware images implement this yet? Might be on my bucket list now

[Spildit]

I wonder if we will start to see hardware imagers supporting this feature.

It has been 4 years since this post. So do any of the hardware images implement this yet? Might be on my bucket list now

Not that i know of ...

Also this feature might not be present in te majority of the drives yet .... So it would only be of use in modern drives but yes, it looks like a good feature to have and if you can disable one head (or more) on the drive untill a power cicle and clone/image the rest of the LBAs belonging to the heads that are still working this might be a very cool thing to implement in cloning / imaging software (and hardware).

One would send the commands to the drive to check for heads and prevent the drive from reading from faling heads. Only then one would clone/image.

This assuming of course that the LBA translation is preserved and that the drive will respond with some error to all LBAs that translate to the broken head without skipping those LBAs or assigning them to another physical space (like when one depopulate heads for drive refurb)...

[maximus]

It seems that there are a few HGST models that support this new feature. And as @fzabkar has pointed out to me, the new HGST Helium drives support this feature. I could not find any direct evidence that Seagate has actually produced any drives that support this feature, even though they appear to have been involved with the idea early on.

I have a HGST drive that should support this feature on its way to me to test with, so that I can try to implement this into hddsuperclone.

[Spildit]

Great ! Let us know if you do make any progress with this ! I will test as well as soon as i get one drive that does support that feature.

[fzabkar]

I could not find any direct evidence that Seagate has actually produced any drives that support this feature ...

Rebuild Assist appears to be a feature of some Seagate SAS drives. Seagate muddies the waters by calling it "RAID Rebuild", in which case it is also a feature of Exos X12 SATA models.

https://www.seagate.com/files/www-content/datasheets/pdfs/exos-x-12-DS1946-2-1712US-en_US.pdf

[maximus]

I could not find any direct evidence that Seagate has actually produced any drives that support this feature ...

Rebuild Assist appears to be a feature of some Seagate SAS drives. Seagate muddies the waters by calling it "RAID Rebuild", in which case it is also a feature of Exos X12 SATA models.

https://www.seagate.com/files/www-content/datasheets/pdfs/exos-x-12-DS1946-2-1712US-en_US.pdf

Go figure. And if you look at the bottom of the document, you can find where Seagate is calling the "RAID Rebuild" feature theirs...

Seagate RAID Rebuild and Seagate Secure are either trademarks or registered trademarks of Seagate Technology LLC or one of its affiliated companies in the United States and/or other countries.

[maximus]

The drive arrived, and according to the identify device results it does support the rebuild assist feature. The DOM is August 2015, so HGST has been making drives that support this feature for at least 3 years now. Now I just have to find the time to test, and this will take some work so I don't expect to have results right away.

The drive is HGST Ultrastar 7K6000 HUS726040ALE614 4TB SATA. The SMART is pretty clean, only 11 power on hours. It was sold as "New other, under 100 power on hours". Just curious, how easy and common is it for someone to clear the SMART on these and sell as newer than they really are? I am thinking of making it a backup drive when I am done playing with it.

[fzabkar]

The drive is HGST Ultrastar 7K6000 HUS726040ALE614 4TB SATA. The SMART is pretty clean, only 11 power on hours. It was sold as "New other, under 100 power on hours". Just curious, how easy and common is it for someone to clear the SMART on these and sell as newer than they really are?

AIUI, these models are not yet supported by any data recovery tool. Therefore I doubt that many people would know how to access the SA on these models.

One possible scenario is that your drive was purchased as a RAID spare.

[Spildit]

The drive is HGST Ultrastar 7K6000 HUS726040ALE614 4TB SATA. The SMART is pretty clean, only 11 power on hours. It was sold as "New other, under 100 power on hours". Just curious, how easy and common is it for someone to clear the SMART on these and sell as newer than they really are?

AIUI, these models are not yet supported by any data recovery tool. Therefore I doubt that many people would know how to access the SA on these models.

One possible scenario is that your drive was purchased as a RAID spare.

Well ... Thos newer HGST drives still support the old IBM command to access the "SUPER SMART" so most likely the clear SMART command will work as well ... Its just a question of testing that out. Issue the super smart on and the clear S.M.A.R.T. check if it does work ... Most likely it will work even on the latest HGST He filled drives ...

[Spildit]

Also can you please DUMP the ROM content of your flash chip ? I might figure out the Super On VSC from the ROM dump.

[maximus]

Well ... Thos newer HGST drives still support the old IBM command to access the "SUPER SMART" so most likely the clear SMART command will work as well ... Its just a question of testing that out. Issue the super smart on and the clear S.M.A.R.T. check if it does work ... Most likely it will work even on the latest HGST He filled drives ...

That is interesting to know.

Also can you please DUMP the ROM content of your flash chip ? I might figure out the Super On VSC from the ROM dump.

I don't have any firmware tools or chip reader to do that. So unless there is a free firmware tool or VSC you can point me to that can dump the ROM, it won't happen.

[Spildit]

The point of dump the ROM would be to figure out the Super ON VSC. Without that forget about dumping the ROM by VSCs ... Idea would be to use external programmer, ger ROM image, figure out from ROM the Super On and later use that to access SA and ROM/NVRam ...

[Spildit]

Well ... Thos newer HGST drives still support the old IBM command to access the "SUPER SMART" so most likely the clear SMART command will work as well ... Its just a question of testing that out. Issue the super smart on and the clear S.M.A.R.T. check if it does work ... Most likely it will work even on the latest HGST He filled drives ...

That is interesting to know.

Are you willing to do some testing on your drive ?

http://www.hddoracle.com/viewtopic.php?f=118&t=2171

On the link you have the commands to enable super SMART and then clear SMART. Can you test this and check if it does work with your (modern) HGST drive ?

[maximus]

Are you willing to do some testing on your drive ?
http://www.hddoracle.com/viewtopic.php?f=118&t=2171
On the link you have the commands to enable super SMART and then clear SMART. Can you test this and check if it does work with your (modern) HGST drive ?

The results do show that some smart data is cleared. But the power on hours did not clear. Both outputs are with super smart on.

Before clearing smart:

Code:

ID#   FLAG  VALUE WORST THRESH   RAW DATA          ATTRIBUTE NAME
  1  0x000b  100   100    16   0x00000000000001   Read Error Rate
  2  0x0005  135   100    54   0x00000000000070   Throughput Performance
  3  0x0007  146   100    24   0x00000801970142   Spin-Up Time
  4  0x0012  100   100     0   0x0000000000001f   Start/Stop Count
  5  0x0033  100   100     5   0x00000000000000   Reallocated Sectors Count
  7  0x000b  100   100    67   0x00000000000000   Seek Error Rate
  8  0x0005  128   100    20   0x00000000000012   Seek Time Performance
  9  0x0012  100   100     0   0x00000000000030   Power-On Hours Count
10  0x0013  100   100    60   0x00000000000000   Spin Retry Count
12  0x0032  100   100     0   0x0000000000001d   Power Cycle Count
191  0x000a  100   100     0   0x00000000000000   G-Sense Errors
192  0x0032  100   100     0   0x00000000000020   Power-Off Retract Cycles
193  0x0012  100   100     0   0x00000000000020   Load/Unload Cycles
194  0x0002  253   130     0   0x00003000140016   Temperature
196  0x0032  100   100     0   0x00000000000000   Reallocation Events
197  0x0022  100   100     0   0x00000000000000   Current Pending Sectors
198  0x0008  100   100     0   0x00000000000000   Off-line Uncorrectable
199  0x000a  200   200     0   0x00000000000000   UDMA CRC Error Rate
211  0x0002  100   100     0   0x00007800160195   Vibration During Write
222  0x0012  100   100     0   0x00000000000029   Loaded Hours
223  0x000b  100   100    50   0x00000000000000   Load/Unload Retries
226  0x0007  187   187    40   0x00000000410002   Load-in Time
230  0x0012   94    94     0   0x00000000100001   GMR Head Amplitude

After clearing smart:

Code:

ID#   FLAG  VALUE WORST THRESH   RAW DATA          ATTRIBUTE NAME
  1  0x000b  100   100    16   0x00000000000000   Read Error Rate
  2  0x0005  100   100    54   0x00000000000000   Throughput Performance
  3  0x0007  100   100    24   0x00000000000000   Spin-Up Time
  4  0x0012  100   100     0   0x0000000000001f   Start/Stop Count
  5  0x0033  100   100     5   0x00000000000000   Reallocated Sectors Count
  7  0x000b  100   100    67   0x00000000000000   Seek Error Rate
  8  0x0005  100   100    20   0x00000000000000   Seek Time Performance
  9  0x0012  100   100     0   0x00000000000030   Power-On Hours Count
10  0x0013  100   100    60   0x00000000000000   Spin Retry Count
12  0x0032  100   100     0   0x0000000000001d   Power Cycle Count
191  0x000a  100   100     0   0x00000000000000   G-Sense Errors
192  0x0032  100   100     0   0x00000000000020   Power-Off Retract Cycles
193  0x0012  100   100     0   0x00000000000020   Load/Unload Cycles
194  0x0002  250   250     0   0x00001900190018   Temperature
196  0x0032  100   100     0   0x00000000000000   Reallocation Events
197  0x0022  100   100     0   0x00000000000000   Current Pending Sectors
198  0x0008  100   100     0   0x00000000000000   Off-line Uncorrectable
199  0x000a  200   200     0   0x00000000000000   UDMA CRC Error Rate
211  0x0002  100   100     0   0x00007800180192   Vibration During Write
222  0x0012  100   100     0   0x00000000000000   Loaded Hours
223  0x000b  100   100    50   0x00000000000000   Load/Unload Retries
226  0x0007  100   100    40   0x00000000000000   Load-in Time
230  0x0012   95    95     0   0x000000000c0004   GMR Head Amplitude


[maximus]

I would just like to add that I don't want to stray too far off topic here. I see there is a potential that maybe the drive I have could have more power on hours than indicated if someone knew the proper commands. But that is not really important for the topic, so I think I need to get back to the real testing part.

The first step was to figure out how to properly perform a READ FPDMA QUEUED command (required for the rebuild assist). That took me about a full day. Next step is to handle a read error with that command, as it reacts differently.

[Spildit]

Ok ! Thanks for letting us know that those VSCs still do work with your modern drive. Most likely Format Unit, etc will work as well.

Keep up with the good work.

It will be very cool if hddsuperclone is able to use the Rebuild Assist of modern drives !!!

With Rebuild Assist even if you don't have the capability to do head map or disable heads, etc you can simply make the drive "skip" all sectors on a damaged head by not even attempt to read those and clone / image the drive like if you were using head maps ! The drive itself will not even attempt to read if the head is gone and if hddsuperclone can correctly get the error send by the drive it can simply skip the bad sector on the bad head and move on to the next one.

Great !

And asthis is a ATA standard even on modern drives that are fully locked and that tools like PC-3000 can't work with you might be able to "clone by heads" or skip bad heads using this feature ! Very cool.

[maximus]

And asthis is a ATA standard even on modern drives that are fully locked and that tools like PC-3000 can't work with you might be able to "clone by heads" or skip bad heads using this feature ! Very cool.

I have been bouncing ideas around in my head on a few different ways to utilize this feature, likely offering options to choose from. And one of those is to image by head. And if errors are encountered, it can move on to the next head, or maybe use some sort of aggressive skipping algorithm to skip out of a potential bad spot. I think the aggressive skipping is the way to go at first, because there is always the possibility of a good head giving errors when one of the heads hits physical media damage. I have seen logs where it looks like one surface had a bad spot, and the other heads also showed signs of reading difficulty when in this area.

[Spildit]

Cool !

Sooner or later i will have to buy some modern drives that support that feature to play with as well.

This might be a very cool thing as newer drives are highly locked (firmware level) and not supported by standard firmware tools like PC-3000 and MRT ... If we can use the Rebuild Assist then we can use something to get the data out !

[maximus]

So according to the ATA standard, this feature requires using the READ FPDMA QUEUED command for the reads when using the rebuild assist. It took me awhile, but I have managed to tame that read command to be able to utilize it. So then I actually tried testing by disabling a head, and it also worked using regular READ DMA commands. Although I would like to point out that to be able to get an LBA value where the "disabled head" ends and the next good read begins does require using READ FPDMA QUEUED. So the testing was looking good...

And then I tried to disable more than one head at a time, and it won't let me! WTF!!! The standard explains it as you just can't disable all read elements, but you can disable multiple read elements. But I can only disable one at a time on this drive, that kills the idea of reading by head