All times are UTC - 5 hours [ DST ]


Forum rules


Please do not post questions about data recovery cases here (use this forum instead). This forum is for topics on finding new ways to recover data. Accessing firmware, writing programs, reading bits off the platter, recovering data from dust...



Post new topic Reply to topic  [ 76 posts ]  Go to page Previous  1, 2, 3, 4  Next
Author Message
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 25th, 2018, 21:32 
Offline

Joined: January 29th, 2012, 1:43
Posts: 671
Location: United States
So maybe frustration is a good thing. Now I am thinking of an efficient way to map the heads while only being able to disable one at a time, assuming the return data from some commands follows the standard and gives proper results. The mad scientist in me will make this work one way or another :twisted:

And oh yeah, Merry Christmas! :D

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 25th, 2018, 21:53 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 11904
Location: Australia
It sounds like you want to build a virtual translator. Perhaps you could use the following method to determine the size of the serpentine segments, at least for testing purposes.

http://www.hddoracle.com/viewtopic.php?f=59&t=650

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 25th, 2018, 22:15 
Offline

Joined: January 29th, 2012, 1:43
Posts: 671
Location: United States
If my idea works, I should be able to use the data from using the rebuild assist to get an exact head mapping, while disabling one at a time. It would be exact based on some return data which I have not verified yet, but if that data does work, the head mapping would be exact. Still a theory at this time, but will be tested soon.

The one thing that could cause headaches is the possibility of reallocated sectors that perhaps are reallocated to a different head, and how they are presented when disabling a head. Not sure if that would ever be an issue, but always a possibility that I have to be aware of.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 25th, 2018, 22:31 
Offline

Joined: January 29th, 2012, 1:43
Posts: 671
Location: United States
Quote:
It would be exact based on some return data which I have not verified yet, but if that data does work, the head mapping would be exact. Still a theory at this time, but will be tested soon.

Just did a simple test, and the data seems valid and usable. :D
Still much more work and testing to do...

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 26th, 2018, 17:50 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 10828
Location: Portugal
Ok let's think ...

Assuming a drive have 4 heads and 2 are gone .... let's say head 0 and 3 are ok.

You use Rebuild Assist commands and READ the data. When the READ is send to a LBA that belongs to a head that is damaged the drive will not attempt to read and will send an error, correct ? So you skip that sector and move on ...

Now can't you just use the READ command and the drive itself will know that head 0 and 3 are ok and will return the error on head 1 and 2 ? If so you can read in sequence and get the data out of the good heads getting error on LBAs that belong to bad heads.

G-List should have few entries. Even if you can't get the same error when reading a re-located sector even if the drive does have re-located sectors there will be just a bunch of LBAs and it will be way better than using one enite bad head. Even if some eentries on G-List do end up leading the drive to read sectors on bad head it will be just a few of them as G-List can have just a bunch of entries as the drive does have very limited number of spare sectors ...

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 26th, 2018, 18:18 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 11904
Location: Australia
AIUI, if the drive has determined that particular read elements are bad, then it will set the appropriate bits in the Rebuild Assist log (15h). One can then transparently clone the drive without having to skip bad areas. The user can additionally disable a marginally good head, ie one that the drive thinks is OK. However, the problem AIUI is that this particular Hitachi firmware deviates from the ATA standard in that it does not permit the user to disable more than one additional head at any one time.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 26th, 2018, 18:59 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 10828
Location: Portugal
Ok ...

Let's see how this ends up implemented in hddsuperclone !

:D

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 26th, 2018, 20:45 
Offline

Joined: January 29th, 2012, 1:43
Posts: 671
Location: United States
Spildit wrote:
Ok let's think ...

Assuming a drive have 4 heads and 2 are gone .... let's say head 0 and 3 are ok.

You use Rebuild Assist commands and READ the data. When the READ is send to a LBA that belongs to a head that is damaged the drive will not attempt to read and will send an error, correct ? So you skip that sector and move on ...

Now can't you just use the READ command and the drive itself will know that head 0 and 3 are ok and will return the error on head 1 and 2 ? If so you can read in sequence and get the data out of the good heads getting error on LBAs that belong to bad heads.

G-List should have few entries. Even if you can't get the same error when reading a re-located sector even if the drive does have re-located sectors there will be just a bunch of LBAs and it will be way better than using one enite bad head. Even if some eentries on G-List do end up leading the drive to read sectors on bad head it will be just a few of them as G-List can have just a bunch of entries as the drive does have very limited number of spare sectors ...

Yes, that is more or less how it would work, in the easy raw form.



fzabkar wrote:
AIUI, if the drive has determined that particular read elements are bad, then it will set the appropriate bits in the Rebuild Assist log (15h). One can then transparently clone the drive without having to skip bad areas. The user can additionally disable a marginally good head, ie one that the drive thinks is OK. However, the problem AIUI is that this particular Hitachi firmware deviates from the ATA standard in that it does not permit the user to disable more than one additional head at any one time.

Exactly. Another thing to realize is that if you want to disable a marginally good head yourself, the only way to know which head to disable is by trial and error. There is no direct way to know which head any bad sectors are in until you start switching off heads one at a time to see which one makes the read different.


Spildit wrote:
Ok ...
Let's see how this ends up implemented in hddsuperclone !
:D

Yes, let’s see how I can manage to do this in a beneficial way! My current idea is to be able to create a head map using the available functions. Because I can only disable one head at a time, I won’t be able to fully test it in a scenario where the drive itself disables one or more heads. There is also a possible timing issue, where every time you enable the rebuild assist to switch heads, there is a delay. I have not officially timed it, but I am guessing between ¼ and ½ second. That can add up, I did the math on my first idea, and it could easily take 30-45 minutes to map the heads that way on my test drive. So now I have another idea to speed things up, but it starts to get more complicated.

Today was not very productive for me in a programming sense. So I used it as a think day, running ideas around in my head when I had the chance. I think I might have a good idea how to map the heads in a reasonable amount of time, but I won’t know until I actually get the programming in place. My ultimate goal is to be able to map the heads, and then be able to perform read operations knowing exactly which areas belong to what head, giving the program the control to handle the operation of skipping areas and head(s) as needed.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 26th, 2018, 21:01 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 10828
Location: Portugal
Sounds very cool !!!

Hopefully it will work as planned !

Keep up with the great work !

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 28th, 2018, 0:03 
Offline

Joined: January 29th, 2012, 1:43
Posts: 671
Location: United States
I would like to point out that the head mapping will only work in direct AHCI mode in hddsuperclone. I did a quick test of the READ FPDMA QUEUED command using the ata-passthrough, and it choked as I expected. Those NCQ commands are a different kind of animal at the driver level.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 28th, 2018, 14:41 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 10828
Location: Portugal
At any rate it's way better than nothing ...

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 29th, 2018, 21:13 
Offline

Joined: January 29th, 2012, 1:43
Posts: 671
Location: United States
So I have now run into another issue. My test drive indicates it has 9 heads that can be disabled (0-8). This is according to the results of reading the rebuild assist log. But if I disable head 8, the rebuild assist completely quits working until I start over. I thought maybe it could have 5 platters with 9 heads, but when I just skip trying to disable head 8, everything lines up using heads 0-7, just like it should for a 4 platter drive with 8 heads. This is getting more and more frustrating. I am thinking of putting this on hold for now. This drive is no good for testing. Maybe now I know why no other tools are implementing this feature. I can’t implement head mapping using a test drive that is non-conforming.

And just to be clear, the drive will seemingly allow me to disable the mystery head 8, just as it should according to the bitmap returned by the rebuild assist log. It won’t let me disable any head that is not in the bitmap, attempting that will return with an aborted command as it should. So it is reacting properly according to what heads are available to be disabled. It just seems to have a phantom head that messes everything up. It is possible that while this drive appears to support the rebuild assist, it may have been in a beta stage at the time of manufacture, and therefore has issues.

So I think it is mission aborted for now, and time for a few beers… maybe more than a few :)

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 29th, 2018, 21:52 
Offline

Joined: January 29th, 2012, 1:43
Posts: 671
Location: United States
And I would like to point out that the need to map heads is somewhat overrated, at least for my software. HDDSuperClone has a self learning head skipping algorithm that doesn’t need to know the head mapping. I was attempting this head mapping as it could be more exact, leading to maybe a few more bytes of data recovered a bit faster. Maybe in some extreme cases it could be worth more. But I won’t spend too much time on something that my software already can do in a way that already works. Unless this new feature can be utilized in a way that works and works well, it is not worth the effort.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 9:38 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 10828
Location: Portugal
Ok ... nice try anyway ....

Just a small request, can you PM me the ATA specific command that you did use to turn on the rebuild assist and to make the drive test itself and produce the log ?

Thanks.

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 13:20 
Offline

Joined: January 29th, 2012, 1:43
Posts: 671
Location: United States
Spildit wrote:
Ok ... nice try anyway ....
Just a small request, can you PM me the ATA specific command that you did use to turn on the rebuild assist and to make the drive test itself and produce the log ?
Thanks.

No need to be secret with ATA commands. The Rebuild Assist log (log page 15h) is a standard log page that can be read and written to by the normal READ LOG EXT (2Fh) and WRITE LOG EXT (3Fh) commands. Here is the basic outline of the log:

Offset 0, Flag bits, bit 0, MANAGE REBUILD ASSIST bit (0 = off, 1 = on, write 1 to turn on, 0 to turn off)
Offset 7, PHYSICAL ELEMENT LENGTH field (p)
Offset 8..7+p, DISABLE PHYSICAL ELEMENT MASK field (bit mask for available heads, read only)
Offset 8+p..7+(2*p), DISABLED PHYSICAL ELEMENTS field (bit map for disabled head, set a bit to 1 to disable a head)

Want to know more, do a search for "ATA Command Set - 4 (ACS-4)", and you should find a public PDF draft document from t13.org. It is from October 2016, so it is a couple years old, but it is what I am currently using.

I am not abandoning this feature in hddsuperclone, I have much code written in place for the future. I just need a drive that obeys the rules. Plus I think I need to sign up to t13 to be able to get access to newer documents. When searching online for “Rebuild Assist log”, I found some interesting submissions. There is one from WD that outlines a feature to read a log that would appear to have the actual head map to LBA data… wouldn’t that be nice!

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 14:07 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 10828
Location: Portugal
Looks cool ! Thanks for the info as well.

Maybe on some drive turning on Rebuild Assist might be all that i would need for example to prevent a slow issue or a pending bug if direct access to the firmware is not available and Rebuild Assist do indeed turn off internal background processes of the drive ...

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 14:25 
Offline

Joined: January 29th, 2012, 1:43
Posts: 671
Location: United States
Spildit wrote:
Looks cool ! Thanks for the info as well.

Maybe on some drive turning on Rebuild Assist might be all that i would need for example to prevent a slow issue or a pending bug if direct access to the firmware is not available and Rebuild Assist do indeed turn off internal background processes of the drive ...

That is true. I will say that while the documentation states that the drive should also only perform a limited recovery attempt for a read when the rebuild assist is on, I did not see any different in time for my sudo error created by write uncorrectable. It will be up to the drive firmware to determine what level of things it does to assist the recovery, and we won't know that until we try on a real failing drive :)

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 15:07 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 10828
Location: Portugal
Ok !!! It might be worth trying on newer WD drives that do have locked SA access and slow issue ....

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 15:43 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 11904
Location: Australia
@maximus, would you mind posting the contents of typical examples of log 15h?

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 16:04 
Offline

Joined: January 29th, 2012, 1:43
Posts: 671
Location: United States
fzabkar wrote:
@maximus, would you mind posting the contents of typical examples of log 15h?

Here are a few log read examples. The log page is 512 bytes, but I am cutting it to the first 32 bytes, and really only the first 16 are used here.

Default after power on:
Code:
0: 00 00 00 00 00 00 00 04 ff 01 00 00 00 00 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................


Rebuild assist enabled, but no heads depopped
Code:
0: 01 00 00 00 00 00 00 04 ff 01 00 00 00 00 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................


Head 0 depopped
Code:
0: 01 00 00 00 00 00 00 04 ff 01 00 00 01 00 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................


The mysterious head 8 depopped
Code:
0: 01 00 00 00 00 00 00 04 ff 01 00 00 00 01 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 76 posts ]  Go to page Previous  1, 2, 3, 4  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group