Switch to full style
In-depth technology research: finding new ways to recover data, accessing firmware, writing programs, reading bits off the platter, recovering data from dust.

Forum rules

Please do not post questions about data recovery cases here (use this forum instead). This forum is for topics on finding new ways to recover data. Accessing firmware, writing programs, reading bits off the platter, recovering data from dust...
Post a reply

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 25th, 2018, 21:32

So maybe frustration is a good thing. Now I am thinking of an efficient way to map the heads while only being able to disable one at a time, assuming the return data from some commands follows the standard and gives proper results. The mad scientist in me will make this work one way or another :twisted:

And oh yeah, Merry Christmas! :D

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 25th, 2018, 21:53

It sounds like you want to build a virtual translator. Perhaps you could use the following method to determine the size of the serpentine segments, at least for testing purposes.

http://www.hddoracle.com/viewtopic.php?f=59&t=650

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 25th, 2018, 22:15

If my idea works, I should be able to use the data from using the rebuild assist to get an exact head mapping, while disabling one at a time. It would be exact based on some return data which I have not verified yet, but if that data does work, the head mapping would be exact. Still a theory at this time, but will be tested soon.

The one thing that could cause headaches is the possibility of reallocated sectors that perhaps are reallocated to a different head, and how they are presented when disabling a head. Not sure if that would ever be an issue, but always a possibility that I have to be aware of.

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 25th, 2018, 22:31

It would be exact based on some return data which I have not verified yet, but if that data does work, the head mapping would be exact. Still a theory at this time, but will be tested soon.

Just did a simple test, and the data seems valid and usable. :D
Still much more work and testing to do...

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 26th, 2018, 17:50

Ok let's think ...

Assuming a drive have 4 heads and 2 are gone .... let's say head 0 and 3 are ok.

You use Rebuild Assist commands and READ the data. When the READ is send to a LBA that belongs to a head that is damaged the drive will not attempt to read and will send an error, correct ? So you skip that sector and move on ...

Now can't you just use the READ command and the drive itself will know that head 0 and 3 are ok and will return the error on head 1 and 2 ? If so you can read in sequence and get the data out of the good heads getting error on LBAs that belong to bad heads.

G-List should have few entries. Even if you can't get the same error when reading a re-located sector even if the drive does have re-located sectors there will be just a bunch of LBAs and it will be way better than using one enite bad head. Even if some eentries on G-List do end up leading the drive to read sectors on bad head it will be just a few of them as G-List can have just a bunch of entries as the drive does have very limited number of spare sectors ...

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 26th, 2018, 18:18

AIUI, if the drive has determined that particular read elements are bad, then it will set the appropriate bits in the Rebuild Assist log (15h). One can then transparently clone the drive without having to skip bad areas. The user can additionally disable a marginally good head, ie one that the drive thinks is OK. However, the problem AIUI is that this particular Hitachi firmware deviates from the ATA standard in that it does not permit the user to disable more than one additional head at any one time.

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 26th, 2018, 18:59

Ok ...

Let's see how this ends up implemented in hddsuperclone !

:D

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 26th, 2018, 20:45

Spildit wrote:Ok let's think ...

Assuming a drive have 4 heads and 2 are gone .... let's say head 0 and 3 are ok.

You use Rebuild Assist commands and READ the data. When the READ is send to a LBA that belongs to a head that is damaged the drive will not attempt to read and will send an error, correct ? So you skip that sector and move on ...

Now can't you just use the READ command and the drive itself will know that head 0 and 3 are ok and will return the error on head 1 and 2 ? If so you can read in sequence and get the data out of the good heads getting error on LBAs that belong to bad heads.

G-List should have few entries. Even if you can't get the same error when reading a re-located sector even if the drive does have re-located sectors there will be just a bunch of LBAs and it will be way better than using one enite bad head. Even if some eentries on G-List do end up leading the drive to read sectors on bad head it will be just a few of them as G-List can have just a bunch of entries as the drive does have very limited number of spare sectors ...

Yes, that is more or less how it would work, in the easy raw form.



fzabkar wrote:AIUI, if the drive has determined that particular read elements are bad, then it will set the appropriate bits in the Rebuild Assist log (15h). One can then transparently clone the drive without having to skip bad areas. The user can additionally disable a marginally good head, ie one that the drive thinks is OK. However, the problem AIUI is that this particular Hitachi firmware deviates from the ATA standard in that it does not permit the user to disable more than one additional head at any one time.

Exactly. Another thing to realize is that if you want to disable a marginally good head yourself, the only way to know which head to disable is by trial and error. There is no direct way to know which head any bad sectors are in until you start switching off heads one at a time to see which one makes the read different.


Spildit wrote:Ok ...
Let's see how this ends up implemented in hddsuperclone !
:D

Yes, let’s see how I can manage to do this in a beneficial way! My current idea is to be able to create a head map using the available functions. Because I can only disable one head at a time, I won’t be able to fully test it in a scenario where the drive itself disables one or more heads. There is also a possible timing issue, where every time you enable the rebuild assist to switch heads, there is a delay. I have not officially timed it, but I am guessing between ¼ and ½ second. That can add up, I did the math on my first idea, and it could easily take 30-45 minutes to map the heads that way on my test drive. So now I have another idea to speed things up, but it starts to get more complicated.

Today was not very productive for me in a programming sense. So I used it as a think day, running ideas around in my head when I had the chance. I think I might have a good idea how to map the heads in a reasonable amount of time, but I won’t know until I actually get the programming in place. My ultimate goal is to be able to map the heads, and then be able to perform read operations knowing exactly which areas belong to what head, giving the program the control to handle the operation of skipping areas and head(s) as needed.

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 26th, 2018, 21:01

Sounds very cool !!!

Hopefully it will work as planned !

Keep up with the great work !

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 28th, 2018, 0:03

I would like to point out that the head mapping will only work in direct AHCI mode in hddsuperclone. I did a quick test of the READ FPDMA QUEUED command using the ata-passthrough, and it choked as I expected. Those NCQ commands are a different kind of animal at the driver level.

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 28th, 2018, 14:41

At any rate it's way better than nothing ...

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 29th, 2018, 21:13

So I have now run into another issue. My test drive indicates it has 9 heads that can be disabled (0-8). This is according to the results of reading the rebuild assist log. But if I disable head 8, the rebuild assist completely quits working until I start over. I thought maybe it could have 5 platters with 9 heads, but when I just skip trying to disable head 8, everything lines up using heads 0-7, just like it should for a 4 platter drive with 8 heads. This is getting more and more frustrating. I am thinking of putting this on hold for now. This drive is no good for testing. Maybe now I know why no other tools are implementing this feature. I can’t implement head mapping using a test drive that is non-conforming.

And just to be clear, the drive will seemingly allow me to disable the mystery head 8, just as it should according to the bitmap returned by the rebuild assist log. It won’t let me disable any head that is not in the bitmap, attempting that will return with an aborted command as it should. So it is reacting properly according to what heads are available to be disabled. It just seems to have a phantom head that messes everything up. It is possible that while this drive appears to support the rebuild assist, it may have been in a beta stage at the time of manufacture, and therefore has issues.

So I think it is mission aborted for now, and time for a few beers… maybe more than a few :)

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 29th, 2018, 21:52

And I would like to point out that the need to map heads is somewhat overrated, at least for my software. HDDSuperClone has a self learning head skipping algorithm that doesn’t need to know the head mapping. I was attempting this head mapping as it could be more exact, leading to maybe a few more bytes of data recovered a bit faster. Maybe in some extreme cases it could be worth more. But I won’t spend too much time on something that my software already can do in a way that already works. Unless this new feature can be utilized in a way that works and works well, it is not worth the effort.

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 30th, 2018, 9:38

Ok ... nice try anyway ....

Just a small request, can you PM me the ATA specific command that you did use to turn on the rebuild assist and to make the drive test itself and produce the log ?

Thanks.

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 30th, 2018, 13:20

Spildit wrote:Ok ... nice try anyway ....
Just a small request, can you PM me the ATA specific command that you did use to turn on the rebuild assist and to make the drive test itself and produce the log ?
Thanks.

No need to be secret with ATA commands. The Rebuild Assist log (log page 15h) is a standard log page that can be read and written to by the normal READ LOG EXT (2Fh) and WRITE LOG EXT (3Fh) commands. Here is the basic outline of the log:

Offset 0, Flag bits, bit 0, MANAGE REBUILD ASSIST bit (0 = off, 1 = on, write 1 to turn on, 0 to turn off)
Offset 7, PHYSICAL ELEMENT LENGTH field (p)
Offset 8..7+p, DISABLE PHYSICAL ELEMENT MASK field (bit mask for available heads, read only)
Offset 8+p..7+(2*p), DISABLED PHYSICAL ELEMENTS field (bit map for disabled head, set a bit to 1 to disable a head)

Want to know more, do a search for "ATA Command Set - 4 (ACS-4)", and you should find a public PDF draft document from t13.org. It is from October 2016, so it is a couple years old, but it is what I am currently using.

I am not abandoning this feature in hddsuperclone, I have much code written in place for the future. I just need a drive that obeys the rules. Plus I think I need to sign up to t13 to be able to get access to newer documents. When searching online for “Rebuild Assist log”, I found some interesting submissions. There is one from WD that outlines a feature to read a log that would appear to have the actual head map to LBA data… wouldn’t that be nice!

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 30th, 2018, 14:07

Looks cool ! Thanks for the info as well.

Maybe on some drive turning on Rebuild Assist might be all that i would need for example to prevent a slow issue or a pending bug if direct access to the firmware is not available and Rebuild Assist do indeed turn off internal background processes of the drive ...

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 30th, 2018, 14:25

Spildit wrote:Looks cool ! Thanks for the info as well.

Maybe on some drive turning on Rebuild Assist might be all that i would need for example to prevent a slow issue or a pending bug if direct access to the firmware is not available and Rebuild Assist do indeed turn off internal background processes of the drive ...

That is true. I will say that while the documentation states that the drive should also only perform a limited recovery attempt for a read when the rebuild assist is on, I did not see any different in time for my sudo error created by write uncorrectable. It will be up to the drive firmware to determine what level of things it does to assist the recovery, and we won't know that until we try on a real failing drive :)

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 30th, 2018, 15:07

Ok !!! It might be worth trying on newer WD drives that do have locked SA access and slow issue ....

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 30th, 2018, 15:43

@maximus, would you mind posting the contents of typical examples of log 15h?

Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2

December 30th, 2018, 16:04

fzabkar wrote:@maximus, would you mind posting the contents of typical examples of log 15h?

Here are a few log read examples. The log page is 512 bytes, but I am cutting it to the first 32 bytes, and really only the first 16 are used here.

Default after power on:
Code:
0: 00 00 00 00 00 00 00 04 ff 01 00 00 00 00 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................


Rebuild assist enabled, but no heads depopped
Code:
0: 01 00 00 00 00 00 00 04 ff 01 00 00 00 00 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................


Head 0 depopped
Code:
0: 01 00 00 00 00 00 00 04 ff 01 00 00 01 00 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................


The mysterious head 8 depopped
Code:
0: 01 00 00 00 00 00 00 04 ff 01 00 00 00 01 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
Post a reply