All times are UTC - 5 hours [ DST ]


Forum rules


Please do not post questions about data recovery cases here (use this forum instead). This forum is for topics on finding new ways to recover data. Accessing firmware, writing programs, reading bits off the platter, recovering data from dust...



Post new topic Reply to topic  [ 53 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 29th, 2018, 21:52 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
And I would like to point out that the need to map heads is somewhat overrated, at least for my software. HDDSuperClone has a self learning head skipping algorithm that doesn’t need to know the head mapping. I was attempting this head mapping as it could be more exact, leading to maybe a few more bytes of data recovered a bit faster. Maybe in some extreme cases it could be worth more. But I won’t spend too much time on something that my software already can do in a way that already works. Unless this new feature can be utilized in a way that works and works well, it is not worth the effort.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 13:20 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Spildit wrote:
Ok ... nice try anyway ....
Just a small request, can you PM me the ATA specific command that you did use to turn on the rebuild assist and to make the drive test itself and produce the log ?
Thanks.

No need to be secret with ATA commands. The Rebuild Assist log (log page 15h) is a standard log page that can be read and written to by the normal READ LOG EXT (2Fh) and WRITE LOG EXT (3Fh) commands. Here is the basic outline of the log:

Offset 0, Flag bits, bit 0, MANAGE REBUILD ASSIST bit (0 = off, 1 = on, write 1 to turn on, 0 to turn off)
Offset 7, PHYSICAL ELEMENT LENGTH field (p)
Offset 8..7+p, DISABLE PHYSICAL ELEMENT MASK field (bit mask for available heads, read only)
Offset 8+p..7+(2*p), DISABLED PHYSICAL ELEMENTS field (bit map for disabled head, set a bit to 1 to disable a head)

Want to know more, do a search for "ATA Command Set - 4 (ACS-4)", and you should find a public PDF draft document from t13.org. It is from October 2016, so it is a couple years old, but it is what I am currently using.

I am not abandoning this feature in hddsuperclone, I have much code written in place for the future. I just need a drive that obeys the rules. Plus I think I need to sign up to t13 to be able to get access to newer documents. When searching online for “Rebuild Assist log”, I found some interesting submissions. There is one from WD that outlines a feature to read a log that would appear to have the actual head map to LBA data… wouldn’t that be nice!

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 14:25 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Spildit wrote:
Looks cool ! Thanks for the info as well.

Maybe on some drive turning on Rebuild Assist might be all that i would need for example to prevent a slow issue or a pending bug if direct access to the firmware is not available and Rebuild Assist do indeed turn off internal background processes of the drive ...

That is true. I will say that while the documentation states that the drive should also only perform a limited recovery attempt for a read when the rebuild assist is on, I did not see any different in time for my sudo error created by write uncorrectable. It will be up to the drive firmware to determine what level of things it does to assist the recovery, and we won't know that until we try on a real failing drive :)

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 15:43 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15440
Location: Australia
@maximus, would you mind posting the contents of typical examples of log 15h?

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 16:04 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
fzabkar wrote:
@maximus, would you mind posting the contents of typical examples of log 15h?

Here are a few log read examples. The log page is 512 bytes, but I am cutting it to the first 32 bytes, and really only the first 16 are used here.

Default after power on:
Code:
0: 00 00 00 00 00 00 00 04 ff 01 00 00 00 00 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................


Rebuild assist enabled, but no heads depopped
Code:
0: 01 00 00 00 00 00 00 04 ff 01 00 00 00 00 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................


Head 0 depopped
Code:
0: 01 00 00 00 00 00 00 04 ff 01 00 00 01 00 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................


The mysterious head 8 depopped
Code:
0: 01 00 00 00 00 00 00 04 ff 01 00 00 00 01 00 00    ................
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 17:10 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15440
Location: Australia
It seems odd that HGST chose to use 4 bytes to define a bit mask for 9 (?) heads. :?

I wonder if this Rebuild Assist feature could be (or is planned to be) extended to SSDs. Perhaps each NAND IC could be a "read element".

Also, the ATA standard already has a "depopulate" command, but this involves permanently switching off a specified head and then recertifying the drive at a lower capacity. Obviously this is not the same thing as Rebuild Assist.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 17:21 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15440
Location: Australia
Have you tried switching off the "phantom" head and one of the other heads at the same time?

Perhaps HD Tune's graph might be able to show the actual number of heads.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 18:07 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
I cannot switch off the phantom head and another head. When/after the phantom head is switched off, it will no longer respond with the aborted command with any of the other heads switched off, until I start over. I think maybe starting back at sector 0 with head 0 switched off gets it working again, not totally sure. If I just work with heads 0-7, the mapping lines up as if there are only 8 heads, and I could actually map the heads like that if I wanted to. I think the 9th head is an anomaly. I don't think it is a switch or flag, I think it is just not supposed to be there, and it messes things up when it is switched. Maybe this drive does have 5 platters but only 8 active heads, and something went wrong with the rebuild assist dealing with that.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: December 30th, 2018, 18:48 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Spildit wrote:
Do you know if your test drive still attempts to re-map bad sectors or mark them as "pending" even when rebuild assist is on ? HGST drives do store pensing sectors as well at reading so if you create a bad sector and if you attempt to read it it should go to the pending re-location list and should show on S.M.A.R.T. . If you switch rebuild assist on if you read a bad sector does it go to the pending list as well ?

Tested with sudo uncorrectable sector with logging... no change with rebuild assist on, just like I said before it doesn't shorten the attempted read time either, it takes the full 2 seconds before failing. Of course this is a sudo uncorrectable sector, so it is always possible that it does not work exactly the same, depending on the function.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 13th, 2019, 19:32 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Just a quick update on this. I am now using this drive as an external backup (I bought a docking station), so I won't be as willing to do any data destructive commands. But mapping the heads is non-destructive, so I can still use it for that. And I still have intentions of working on this, just maybe not real fast, low priority kind of thing.

I did use HD Tune to verify that it does have 8 heads. I can't say for sure if physically it only has 4 platters and 8 heads. I have found the following technical documentation that indicates it should be a 5 platter 10 head drive:
https://documents.westerndigital.com/co ... m-spec.pdf

I did a little bit more testing, and figured out that after disabling the mysterious head 8, the only way to get the rebuild assist to work correctly again is after performing a soft reset. So trying to work with this phantom head really messes with it. But now that I know what it takes to bring it back to working again, I can move forward. But I do currently have a more important issue that I am working on, so this will not be a high priority. But I did want it known that I am still playing with it a bit (I can’t seem to help my curiosity).

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 13th, 2019, 20:55 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Spildit wrote:
Thanks for the update !

Maybe the simple fact that one can turn on Rebuild Assist might be of use to disable background process of defects on modern drives with slow problem like WD and Seagate ...

Sure, if they support it, and if they follow the idea of actually disabling the background processes. I don't know if WD is even supporting this feature yet on its new drives.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 19th, 2019, 19:37 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
So as I get back to playing with this, I have a question. Is is possible to safely (I do not want to kill the drive!) short some points on the pcb to effectively make a head unable to read? I don't know exactly how the preamp works, so I don't even know if this is possible. I am attaching pcb pics, two pics of each side with the camera flash in a different spot.
Attachment:
20190119_181110.jpg
20190119_181110.jpg [ 4.62 MiB | Viewed 28998 times ]
Attachment:
20190119_181145.jpg
20190119_181145.jpg [ 4.36 MiB | Viewed 28998 times ]
Attachment:
20190119_181251.jpg
20190119_181251.jpg [ 4.45 MiB | Viewed 28998 times ]
Attachment:
20190119_181303.jpg
20190119_181303.jpg [ 4.87 MiB | Viewed 28998 times ]

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 19th, 2019, 20:09 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15440
Location: Australia
The head select logic is multiplexed, so it is not possible to disable a particular head via manipulation of the PCB, AFAICS.

These preamp block diagrams should illustrate how things work:

http://www.hddoracle.com/viewtopic.php?f=59&t=229

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 19th, 2019, 21:15 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
fzabkar wrote:
The head select logic is multiplexed, so it is not possible to disable a particular head via manipulation of the PCB, AFAICS.

These preamp block diagrams should illustrate how things work:

http://www.hddoracle.com/viewtopic.php?f=59&t=229

I was afraid of that. Oh well, you can't win 'em all...

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 20th, 2019, 21:38 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
So since this drive does not like to be able to map heads the easy way, I was working on it trying to map heads the hard way (by estimating a pattern), and it even has a screwed up spot that makes that complicated. So I did a brute force head mapping (which took 40 minutes and hammered on the drive by constantly re-enabling the rebuild assist), and found a spot where the head order skips. This drive is making the task of mapping heads efficiently very difficult!

Here is the raw data showing the read order in that spot. There are spaces between the lines where it jumps.

head=00 position=11a790000 size=140000 final=11a8cffff
head=01 position=11a8d0000 size=140000 final=11aa0ffff
head=02 position=11aa10000 size=120000 final=11ab2ffff
head=03 position=11ab30000 size=130000 final=11ac5ffff
head=04 position=11ac60000 size=110000 final=11ad6ffff
head=05 position=11ad70000 size=130000 final=11ae9ffff
head=06 position=11aea0000 size=130000 final=11afcffff
head=07 position=11afd0000 size=130000 final=11b0fffff
head=00 position=11b100000 size=120000 final=11b21ffff
head=01 position=11b220000 size=130000 final=11b34ffff
head=02 position=11b350000 size=130000 final=11b47ffff
head=03 position=11b480000 size=130000 final=11b5affff

head=05 position=11b5b0000 size=130000 final=11b6dffff
head=06 position=11b6e0000 size=020000 final=11b6fffff
head=07 position=11b700000 size=120000 final=11b81ffff

head=01 position=11b820000 size=130000 final=11b94ffff
head=02 position=11b950000 size=130000 final=11ba7ffff
head=03 position=11ba80000 size=140000 final=11bbbffff

head=05 position=11bbc0000 size=130000 final=11bceffff

head=07 position=11bcf0000 size=120000 final=11be0ffff

head=01 position=11be10000 size=140000 final=11bf4ffff
head=02 position=11bf50000 size=080000 final=11bfcffff
head=03 position=11bfd0000 size=130000 final=11c0fffff

head=05 position=11c100000 size=130000 final=11c22ffff

head=07 position=11c230000 size=120000 final=11c34ffff

head=01 position=11c350000 size=130000 final=11c47ffff

head=03 position=11c480000 size=140000 final=11c5bffff

head=05 position=11c5c0000 size=130000 final=11c6effff

head=07 position=11c6f0000 size=130000 final=11c81ffff

head=01 position=11c820000 size=140000 final=11c95ffff

head=03 position=11c960000 size=080000 final=11c9dffff

head=05 position=11c9e0000 size=110000 final=11caeffff

head=07 position=11caf0000 size=120000 final=11cc0ffff
head=00 position=11cc10000 size=110000 final=11cd1ffff
head=01 position=11cd20000 size=060000 final=11cd7ffff
head=02 position=11cd80000 size=0a0000 final=11ce1ffff
head=03 position=11ce20000 size=040000 final=11ce5ffff
head=04 position=11ce60000 size=130000 final=11cf8ffff
head=05 position=11cf90000 size=0b0000 final=11d03ffff
head=06 position=11d040000 size=0b0000 final=11d0effff
head=07 position=11d0f0000 size=070000 final=11d15ffff
head=00 position=11d160000 size=130000 final=11d28ffff
head=01 position=11d290000 size=130000 final=11d3bffff
head=02 position=11d3c0000 size=120000 final=11d4dffff
head=03 position=11d4e0000 size=130000 final=11d60ffff
head=04 position=11d610000 size=130000 final=11d73ffff
head=05 position=11d740000 size=120000 final=11d85ffff
head=06 position=11d860000 size=130000 final=11d98ffff
head=07 position=11d990000 size=120000 final=11daaffff

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 20th, 2019, 22:25 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15440
Location: Australia
Quote:
head=06 position=11b6e0000 size=020000 final=11b6fffff

A size 0f 0x20000 bytes corresponds to 0x20 physical sectors. That's got to be much less than a full track. :-?

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 20th, 2019, 22:31 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
fzabkar wrote:
Quote:
head=06 position=11b6e0000 size=020000 final=11b6fffff

A size 0f 0x20000 bytes corresponds to 0x20 physical sectors. That's got to be much less than a full track. :-?

All values are LBA, not bytes. Multiply by 512 for bytes.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 21st, 2019, 18:35 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Well, technically I am creating a head map table with this drive now, although it is just prototype alpha code that isn’t actually useful yet. The brute force method took 40 minutes, which is too long as far as I am concerned. But I now have my pattern method working and able to deal with the odd spot on this drive (more or less, still have work to do to have it verify and fix any inconsistencies). And it is doing it in just over 6 minutes. It will be longer, maybe several seconds or more, when I add in the verify and fix for the non-conforming spot. I still expect under 7 minutes, and my goal was under 10.

Using the brute force method, I am disabling a different head for each read. This causes the rebuild assist to seemingly do some quick internal testing, which takes a bit of time and causes noticeable head movement. I am assuming that it is doing a quick read test of the heads, as per the suggestion in the ATA standard, although I have no way to know exactly what it is really doing. The time it takes for each disabling event is what causes it to take 40 minutes to create the head map.

But there is something else interesting going on even with the pattern method. I am disabling one head at a time, and reading a sector in that head to get the final sector value, then using the pattern to jump to the next run in that head for the next sector read. So the same head stays disabled for multiple reads, until the data from that head is complete. With this method every read causes rapid head movement. It makes the drive sound kind of like a steam locomotive at full speed… chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug-chug............

I have no idea what it is doing that causes that much head chatter. It also causes a delay, but the delay is smaller than the brute force method. The idea behind this should be that the drive would just respond with an abort without attempting to read any actual data. So the head movement and delay makes no sense…

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 21st, 2019, 19:29 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Spildit wrote:
Question would be if a damaged drive would handle that "mapping" process at all or if it wouldn't even be slower to map heads on a drive with damaged heads ....
Well ... I guess more testing would be required....
At this time I have no idea what would happen if it had damaged heads.

I would also like to point out that the rebuild assist feature on this drive is wonky. I have to use soft resets on it when it gets “confused” and does not abort head reads when the head is disabled. This is even in addition to the ghost head. That area where it skips heads definitely confuses it in brute force mode. The confusion seems to happen when disabling a different head. I would say that either the rebuild assist feature on this drive is either just buggy from the factory, or maybe this drive was refurbished which screwed it up.

And I have absolutely no idea what would happen if this drive were to disable a head on its own. My code currently does not have any sort of provision for that. Plus I don’t know how the drive would react with an actual bad head (with this drive it is a guessing game, anything could happen). I will need more R&D before being able to produce a working feature. I will probably need to test with a few different drives, which I don’t have, and don’t really have the money for.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: New "Rebuild Assist" feature in Serial ATA Revision 3.2
PostPosted: January 21st, 2019, 20:45 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Here is yet another interesting thing about this drive. During the pattern mode, as the reads approach the spot where the heads skip, the pitch of the head movement ramps up, and then ramps back down after passing it. This happens for every head, and it starts well before it gets to the spot, and ramps back down at the same pace. This drive is doing some funky stuff with the rebuild assist. At some point in time I will definitely need to get a different drive to test with.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 53 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 12 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group