All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 40 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 24th, 2019, 20:32 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Someone asked me to try to recover the data from a WD40EFRX 4TB HDD. At first the SMART status showed 43 “pending” sectors, then that number increased to 55 during the first transfer attempts, but remained stable for several hours afterwards.
I proceeded to clone the whole drive with HDDSuperClone. There was a bad area near the begining, but then it copied at a steady rate around 60MB/s. Then, about halfway through, it started to have more severe issues; at that point the “pending sector count” increased dramatically, reaching 1167. Looking at the graphic presentation of the recovery provided by HDDSCViewer, I noticed a pattern of alternating stripes, with larger stripes of easily accessible sectors (copied at a rate above 50MB/s), and smaller stripes of unreadable sectors. That kind of pattern indicates that one head is failing, right ? I stopped the copy and tried again starting from sector 7000000000, near the end (which is not practical by the way with HDDSuperClone GUI, I had to edit the log file, changing the input value in the parameters would result in an error) : it showed the same pattern, and again, even when it could transfer something, it appeared as 00s only in the preview.

Now, the owner told me that the drive contained about 1TB of data. Since I successfully cloned about 2TB, everything should be there. But :
– according to GParted, the larger partition contains 553.46GB out of 3.63TB, which could just mean that the owner overestimated the actual amount of data ;
– GParted could gather valid information from the source drive, albeit with difficulty (I could even mount it and see the contents), but it gave an error warning when analysing the corresponding partition from the recovery drive {1} ;
– I noticed that it copied only 00s quite early on, when the copy was at about 12% if I remember correctly, and there was no sign of trouble at that point ;
– when the whole drive is opened in WinHex, it appears totally empty beyond sector 918566948 / offset 470306277376 (it doesn't stop abruptly, it seems to be the end of a video file, and the last non-empty sector ends with zeroes), or sector 909137956 / offset 465478633472 relative to the start of the partition, which is indeed about 12%. So there is a ~120GB discrepancy. How can it be ? Where can it be ? Could the missing data be located at the end of the partition, with a lot of free space in between ? How could I know for sure ?

When I saw that it was copying only zeroes, I re-read HDDSuperClone's manual and tried to apply the method described there to clone only the used portion, but apparently it requires the “virtual disk” feature which is only available on the “Pro” version. Since the main partition is formatted in Ext4, I couldn't use ddru_ntfsbitmap which easily creates ddrescue mapfiles for NTFS partitions. I also tried to increase the “skip size” to a much larger value, like 10MB, but it would go back to 4096 almost immediately. So unfortunately I could only let it clone everything, and it's frustrating because I may have been able to recover much more useful sectors (provided that there is indeed more data somewhere) if that head hadn't been worn out by reading 1.5TB of 00s...
What could be done at this point to complete the clone as much as possible and as efficiently as possible ? (Short of replacing the bad head...)

I tried using ddru_findbad, but it returned nothing, apparently it can't analyse the partitions.
Code:
lubuntu@lubuntu:~$ sudo ddru_findbad /dev/sda /media/lubuntu/Q_SD/WD40EFRX_ddrescue_export.log -o /media/lubuntu/Q_SD/WD40EFRX_findbad.log
Command line input was processed succesfully
ddru_findbad 1.11 20141015
Target = /dev/sda
Logfile = /media/lubuntu/Q_SD/WD40EFRX_ddrescue_export.log
Output base name = /media/lubuntu/Q_SD/WD40EFRX_findbad.log
Sector size = 512
Loop wait time = 2
More info = false
Extra output = false
Quick = false
Quick ntfs = false
Target /dev/sda is detected to be a block device
Neither mmstat or fsstat gave results
The partition table or file system could be corrupt
Cannot determine if target is whole drive or single partition
See file /media/lubuntu/Q_SD/WD40EFRX_findbad2.log_debug.txt for more info


The main partition is formatted in Ext4; I don't know much about that file-system. Apparently the metadata structures are disseminated in “inode” files located all over the partition, instead of being stored in a single structure like the MFT in NTFS. And it turns out that neither WinHex nor R-Studio can reconstruct the complete file-tree. If I can't complete the clone, are there better tools to analyse partial Ext4 partitions ?
At worst I could resort to raw file carving, but it would be frustrating since the partition can still be mounted and the directory structure can still be parsed. Is there at least a way of copying the empty directory structure, to help the owner sort out the mountain of files with nondescript names that comes out of that recovery method ? On Windows I use Robocopy /CREATE for that purpose, so I'm looking for an equivalent command in Linux / Lubuntu.


As a side question : I noticed that the “power-on time” was high at 32975 hours, but the “start / stop count” in particular seems extremely high at 84281. It would mean that the drive was on for less than 30 minutes on average. Isn't that abnormal ? Or is it related to the regular operation of that range of drives, if they spin down automatically when idle or something like that ?


{1} GParted error message :
Code:
Filesystem volume name:   <none>
Last mounted on:          /media/sdb4
Filesystem UUID:          1b5a9115-a28c-4f16-8e7d-c4c3d9d4cb09
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         unsigned_directory_hash
Default mount options:    user_xattr acl
Filesystem state:         clean with errors
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              243900416
Block count:              975575808
Reserved block count:     19511516
Free blocks:              830488873
Free inodes:              243819640
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      791
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Flex block group size:    16
Filesystem created:       Thu Dec 10 08:54:28 2015
Last mount time:          Mon Apr 15 13:26:30 2019
Last write time:          Mon Apr 15 13:59:43 2019
Mount count:              172
Maximum mount count:      -1
Last checked:             Thu Dec 10 08:54:28 2015
Check interval:           0 (<none>)
Lifetime writes:          26 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:             256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      e1c45096-875e-4d9c-a701-f8dcb039618e
Journal backup:           inode blocks
FS Error count:           2751
First error time:         Sun Apr 14 07:06:01 2019
First error function:     ext4_find_entry
First error line #:       935
First error inode #:      18219009
First error block #:      0
Last error time:          Mon Apr 15 13:59:43 2019
Last error function:      __ext4_get_inode_loc
Last error line #:        4199
Last error inode #:       18219197
Last error block #:       72876075</i>

<i>dumpe2fs 1.42.9 (4-Feb-2014)
Journal superblock magic number invalid!</i>

<i>Impossible de lire le contenu du système de fichiers.
Pour cette raison, certaines opérations peuvent être indisponibles.
La raison peut être l'absence d'un paquet logiciel.
Voici la liste des paquets logiciels nécessaires pour la prise en charge du système de fichiers ext4 : e2fsprogs v1.41+.






Attachment:
2019-05-23-024659_1022x760_scrot.png
2019-05-23-024659_1022x760_scrot.png [ 164.97 KiB | Viewed 25297 times ]

SMART status after the first transfer operations.

Attachment:
2019-05-23-190151_786x656_scrot.png
2019-05-23-190151_786x656_scrot.png [ 102.55 KiB | Viewed 25297 times ]

HDDSuperClone copying only 00s beyond ~12%, but at a decent rate of ~60MB/s.

Attachment:
2019-05-24-014637_786x656_scrot.png
2019-05-24-014637_786x656_scrot.png [ 103.7 KiB | Viewed 25297 times ]

Starting to seriously struggle...

Attachment:
2019-05-24-020414_786x656_scrot.png
2019-05-24-020414_786x656_scrot.png [ 105.75 KiB | Viewed 25297 times ]

...but then goes back to full speed...

Attachment:
2019-05-24-020426_786x656_scrot.png
2019-05-24-020426_786x656_scrot.png [ 106.44 KiB | Viewed 25297 times ]

...and struggles again...

Attachment:
2019-05-23-235000_1330x1024_scrot.png
2019-05-23-235000_1330x1024_scrot.png [ 39.89 KiB | Viewed 25297 times ]

HDDSCViewer, “stripes” pattern.

Attachment:
2019-05-24-020543_1330x1024_scrot.png
2019-05-24-020543_1330x1024_scrot.png [ 39.53 KiB | Viewed 25297 times ]

Same pattern much further down the road.

Attachment:
2019-05-24-024015_1023x633_scrot.png
2019-05-24-024015_1023x633_scrot.png [ 161.82 KiB | Viewed 25297 times ]

SMART status at that point.

Attachment:
2019-05-24-044525_777x530_scrot.png
2019-05-24-044525_777x530_scrot.png [ 70.41 KiB | Viewed 25297 times ]

GParted, source drive.

Attachment:
2019-05-24-044554_777x530_scrot.png
2019-05-24-044554_777x530_scrot.png [ 68.57 KiB | Viewed 25297 times ]

GParted, recovery drive.


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 24th, 2019, 23:38 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
« – when the whole drive is opened in WinHex, it appears totally empty beyond sector 918566948 / offset 470306277376 »
=> Here I meant the recovery drive as it should be obvious (or not).


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 25th, 2019, 13:18 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
The first thing I want to say is: Why are you using such an old version of HDDSuperClone? It is almost 2 years old and is a beta version! I fixed that bug with the skip size not saving 6 months ago, not to mention all of the other improvements, such as the skipping algorithm improvement so that you would likely not need to even adjust the skip size manually.

You are correct that the pattern indicates an issue with a head. I can’t tell if the head is dead or just weak from the image, maybe I could tell from the log file. It was fine up until a certain point, it is possible that is just how it is reacting to a weak head, or the head is suddenly getting worse. One way to tell is to start a new recovery with the destination of NULL (no destination) and let it run for a few minutes or so to see if it still reads fine at the beginning like it already has, or if you now see the same pattern starting from the beginning. But I wouldn’t mess with it too much since you are trying to get the data out of it.

As for targeting just the wanted data with free tools, good luck. If it were NTFS I would point you towards the discussion section of ddrutility, where someone figured out how to use the ntfs utilities along with ddrescue to manually target files. But I have nothing for the EXT filesystem. It is reasons like that why I made the virtual driver mode in the paid pro version, but even then you also need to purchase a Linux license for R-Studio (optionally can try DMDE, but R-Studio definitely worked best in testing). Plus you need space for the clone along with space for the recovered files at the same time. The next option up would be the expensive hardware imagers with data extraction. The next option down would be attempting to copy files directly with a file utility, likely without the ability to handle bad sectors or resume. And the last alternative option: write your own program to do it :)

I guess my suggestion would be that unless you are willing to fork out some money, then just upgrade to the latest version of HDDSuperClone and resume the cloning with the free version as you were. It will skip around the bad/weak head and get all the good sectors first. If you think you know where all the desired data is within reason, use the input offset for the start point, and the size to limit how far it will go (sorry, no stop offset, must do some math and use size). I don’t know why you had an issue with setting the input offset before (except for being an old beta version:)).

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 25th, 2019, 14:00 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Damn, this seems to spark a lot of interest... :? Is this too detailed or too convoluted or something ?

Another thing I noticed : before I ran HDDSuperClone, I made two short runs with ddrescue, copying to an image file. For the first run, I made a mistake in the command (forgot the name of the recovery partition) and the file was written to the live system volume, so it stopped at 3.5GB when the memory was full. Then I stopped the second run at 12GB because the average copy rate seemed abnormally low (~20MB/s for what seemed to be good areas), and I remembered that writing an image file (especially in “sparse” mode it would seem) with ddrescue to a NTFS partition was suspected to cause a significant drop in performance (although I couldn't get a definite confirmation when I asked about it), so it seemed like a bad idea to create a 4TB image. But I kept those partial images just in case, and I compared them with a dump of the first 15GB of the HDDSuperClone clone with HexWorkshop [*] : it turns out that there are a lot of differences between the clone and both images, and it's not consistent at all, i.e., in some areas there are empty sectors in the images and recovered sectors in the clone, but in other areas it's reversed. And in the data present in the images but missing in the clone, I can see what looks like metadata / file-system informations, including lists of pathnames in plain text (and it's weird because it's located before the begining of the Ext4 partition, there are many differences between 300 and 400MB, and again around 1.7GB, so it should be either in “linux-swap” or in the first Ext3 partition). Yet the contents of both ddrescue images are strictly identical up until the end of the shorter one, not a single sector is different (so even the skipped sectors were at the same spots). If the drive's condition had deteriorated between the ddrescue runs and the HDDSuperClone run, there would be only missing sectors in the clone, yet there are many recovered sectors in the clone which are empty in both images. And it's not former data that used to be on the recovery drive, because after I stopped the copy with HDDSuperClone, I ran a complete pass in “Fill zero” mode, which as the name implies is supposed to specifically fill the non-recovered sectors with 00s. That's really puzzling.
One of those differences starts at 343343104 with a size of 65533. But in the HDDSuperClone log file I find :
Code:
0xa3f014    0x22abef80    0x7f    0x0    0x0

Which should mean that the copy was fine between 10743828 and 581693311 : that interval contains the aforementioned difference, and many others. Could the “Fill zero” pass have overwritten actually recovered sectors ? Or could the source drive have been modified at some point ? (I mentioned that I briefly mounted the main partition, but that was later, after I stopped the HDDSuperClone copy which had become too erratic.) Could there be another explanation which eludes me right now ?
Would it be wise to merge those extra sectors from the partial images with the clone ? I tried to do that manually, but there are too many. So I'll try to do it with ddrescue, by using the “generate” mode to create a mapfile of empty / non-empty sectors in the clone, and then using that as a logfile to complete the empty sectors with the non-empty sectors from the larger image file. Does that sound safe and sound ?

Then, again, how could I proceed at this point to find out where the missing data is and recover the most of it as efficiently as possible ?


[*] HexWorkshop's comparison module is much more convenient than the one in WinHex, but apparently can't open whole volumes directly, and seems to have issues recognizing the size of 2TB+ storage devices, so neither of them can be conveniently used alone for that kind of task.


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 25th, 2019, 14:20 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
@maximus
Sorry, I hadn't seen your post when sending the one above.
Why using this version : it was on the Lubuntu live SD card which I had ready to run, which was made quite some time ago indeed (probably 2017_08_27) ; I downloaded a new version of that ISO recently but didn't update it, because it was working, and I believe in the “if it's not broken don't fix it” motto, and at first I thought I would use ddrescue, which I know quite well by now, and hasn't been updated recently as far as I know. (And I remember reading discussions about HDDSuperClone at that time which seemed to imply that it was already more efficient than ddrescue in most situations, even though the GUI was still in beta version. And considering how slowly things evolve around me generally, I usually tend to consider something from two years ago as recent enough for my purposes ! :D )

Quote:
You are correct that the pattern indicates an issue with a head. I can’t tell if the head is dead or just weak from the image, maybe I could tell from the log file. It was fine up until a certain point, it is possible that is just how it is reacting to a weak head, or the head is suddenly getting worse. One way to tell is to start a new recovery with the destination of NULL (no destination) and let it run for a few minutes or so to see if it still reads fine at the beginning like it already has, or if you now see the same pattern starting from the beginning.

I'll have to try again, but it seemed to be getting worse even at the begining.

Quote:
But I wouldn’t mess with it too much since you are trying to get the data out of it.

But the thing is, as I explained, the relevant data should already be recovered, if there's only about 550GB of it. Before I proceed any further I'm trying to understand why it doesn't seem to be the case, and why the partition isn't already operational on the recovery drive, considering that it's been copying zeroes for hours when that WD40EFRX drive started having serious issues.

Quote:
As for targeting just the wanted data with free tools, good luck. If it were NTFS I would point you towards the discussion section of ddrutility, where someone figured out how to use the ntfs utilities along with ddrescue to manually target files. But I have nothing for the EXT filesystem. It is reasons like that why I made the virtual driver mode in the paid pro version, but even then you also need to purchase a Linux license for R-Studio (optionally can try DMDE, but R-Studio definitely worked best in testing). Plus you need space for the clone along with space for the recovered files at the same time. The next option up would be the expensive hardware imagers with data extraction. The next option down would be attempting to copy files directly with a file utility, likely without the ability to handle bad sectors or resume. And the last alternative option: write your own program to do it :)

I didn't ask about targeting specific files (which is way more advanced), just the occupied sectors, and skipping what is considered as free space. I already did that successfully with ddrescue and ddru_ntfsbitmap. Apparently it's more complicated with Ext filesystems.

Quote:
I guess my suggestion would be that unless you are willing to fork out some money, then just upgrade to the latest version of HDDSuperClone and resume the cloning with the free version as you were. It will skip around the bad/weak head and get all the good sectors first. If you think you know where all the desired data is within reason, use the input offset for the start point, and the size to limit how far it will go (sorry, no stop offset, must do some math and use size). I don’t know why you had an issue with setting the input offset before (except for being an old beta version:)).

Alright, I'll try that. But, since the partition can still be accessed, is there a tool which could analyse it and return the intervals of occupied sectors, with as little stress as possible, just like ddru_ntfsbitmap does by analysing the $Bitmap file, even if I have then to manually set the copy intervals based on that information ?


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 25th, 2019, 14:48 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Quote:
But the thing is, as I explained, the relevant data should already be recovered, if there's only about 550GB of it. Before I proceed any further I'm trying to understand why it doesn't seem to be the case, and why the partition isn't already operational on the recovery drive, considering that it's been copying zeroes for hours when that WD40EFRX drive started having serious issues.
Maybe the data is not all recovered, you just think it is. Obviously something is still different between the source and destination. You could try examining the destination with DMDE, although I can't help with that, as I don't use it.

Quote:
Alright, I'll try that. But, since the partition can still be accessed, is there a tool which could analyse it and return the intervals of occupied sectors, with as little stress as possible, just like ddru_ntfsbitmap does by analysing the $Bitmap file, even if I have then to manually set the copy intervals based on that information ?
I do not know of a tool for that. On a healthy drive, I use Clonezilla which can copy only the used sectors, but it does not handle bad sectors very well at all. I don't know of a tool that will list the used space, especially without hammering on the drive.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 25th, 2019, 16:52 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Quote:
Maybe the data is not all recovered, you just think it is. Obviously something is still different between the source and destination. You could try examining the destination with DMDE, although I can't help with that, as I don't use it.

Of course there's something different because 1) there were some skipped sectors during the copy, and 2) I stopped the cloning process at about 47%, while it was still in “phase 1”. I didn't (yet) try to get everything that is still recoverable by software-only means, which I know can severely wear out an already failing drive. I told the owner that, at this point, to maximize the recovery rate, it would be necessary to replace the heads, which I can't do. If he unterstands the risk and is absolutely certain that he will not pay the fee of a full-blown recovery service, then I'll proceed, but I'd like to do it in the wisest and most efficient way possible with the available means. There's still a lot that could be recovered out of what's seemingly missing, even with 1 bad head out of 8 on that model, but there could be another one failing anytime soon if I try to scan everything that has not been scanned yet.
I say (most of) the relevant data “should be” recovered because I don't see why there would be a significant amount of data beyond 2TB (more like 1.7 actually) with that much space absolutely empty between 438GB and whatever there's left. But if GParted correctly identified the occupied space, then a large chunk is definitely missing.
DMDE displays the same mostly empty folders as R-Studio, but almost instantly (R-Studio takes 30-40 minutes just to open the partition, with no prior thorough scan, which normally involves no deep analysis – but I recently used R-Studio to recover data from a [healthy] formatted drive which had a similar structure, with an Ext4 partition starting at about 4.5GB, plus smaller partitions of unknown nature before that, which probably used to be in some kind of RAID enclosure, and it took about that long to display the recovery tree). If I do a “virtual filesystem reconstruction” it displays a gazillion of empty folders, but the ones containing something relevant (not much) seem to be the same as in R-Studio. The main 4TB / 3.63Tib partition is flagged as “E B C F”, I don't know what that means. It's a fairly complex piece of software despite its small size and I'm not familiar enough with it to know right from the bat where to look at.

Quote:
I do not know of a tool for that. On a healthy drive, I use Clonezilla which can copy only the used sectors, but it does not handle bad sectors very well at all. I don't know of a tool that will list the used space, especially without hammering on the drive.

And so... does anyone else know ?


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 25th, 2019, 18:09 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Quote:
I told the owner that, at this point, to maximize the recovery rate, it would be necessary to replace the heads, which I can't do. If he unterstands the risk and is absolutely certain that he will not pay the fee of a full-blown recovery service, then I'll proceed, but I'd like to do it in the wisest and most efficient way possible with the available means. There's still a lot that could be recovered out of what's seemingly missing, even with 1 bad head out of 8 on that model, but there could be another one failing anytime soon if I try to scan everything that has not been scanned yet.
Good call on making sure the owner understands that this requires professional recovery for the best results, and that you may not be able to recover it on your own, at least not with good results.

If you do get permission to continue, I would finish the cloning with hddsuperclone through phase 2, that will get the most good data from the good heads. After that, I can't say what to do, as every case is different. You want to target data, but you don't know how, and targeting data is not easy. If you get enough data from the initial clone and the file table information is all there, you may be able to figure something out while using the clone as to how to target the rest of the needed data. You are attempting to recover data from a drive that needs pro data recovery, and you will only get so far on your own.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 25th, 2019, 19:12 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Quote:
You are attempting to recover data from a drive that needs pro data recovery, and you will only get so far on your own.

Yes I'm well aware of that... It seemed definitely manageable when I started examining it (40-50 bad sectors is serious but not yet disastrous, I've recovered two drives of mine with upward of 200 bad sectors, one was a 100% success with only a NTFS system file corrupted and the other had 6 corrupted files out of 3TB, and those were from the dreaded Seagate STx000DM001 series !), but now I realize that 4TB is a lot to handle for a drive with even minor issues ; and also that what seems like minor issues at first can quickly evolve into serious trouble.
(Perhaps I should note that the guy already handed that drive out to a local association, with people proposing to fix computers for a cheap price, 20€ he told me, but they couldn't get anything at all, not a single byte was recovered, with, I suppose, more common methods of scanning the drive directly with a recovery software like Recuva. This of course may have worn the drive some more and made my own attempt more hazardous, perhaps the rescue cloning would have been completed without a hitch if it had been done right away at the first sign of trouble, or perhaps that head would have failed 75% in instead of 47% – impossible to know.)

Quote:
If you do get permission to continue, I would finish the cloning with hddsuperclone through phase 2, that will get the most good data from the good heads. After that, I can't say what to do, as every case is different. You want to target data, but you don't know how, and targeting data is not easy. If you get enough data from the initial clone and the file table information is all there, you may be able to figure something out while using the clone as to how to target the rest of the needed data.

Indeed, if I finish the cloning process I should see where the rest of the data is, but at this point I'm afraid that the drive will be too exhausted to then try to insist on a particular area of interest... Hence why I'm asking for some specific advice, to preemptively avoid such a scenario, if at all possible.

Regarding another question I asked, do you happen to know a Linux command equivalent to Robocopy /CREATE, which would copy the entire file tree with 0 byte files ? Or is this also more complicated on Linux partitions without “hammering” the drive ?


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 25th, 2019, 19:38 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Quote:
Regarding another question I asked, do you happen to know a Linux command equivalent to Robocopy /CREATE, which would copy the entire file tree with 0 byte files ? Or is this also more complicated on Linux partitions without “hammering” the drive ?
The Linux equivalent of robocopy is rsync. I can’t help with the options to say if there is an equivalent command to what you want.

Quote:
Indeed, if I finish the cloning process I should see where the rest of the data is, but at this point I'm afraid that the drive will be too exhausted to then try to insist on a particular area of interest... Hence why I'm asking for some specific advice, to preemptively avoid such a scenario, if at all possible.
With the weak/bad head, I would stop the cloning after phase 2 and assess the results. Phases 1 and 2 get the most good data from the good heads with the least effort, after that it can start to thrash the drive. It is all about understanding what you are dealing with. I offer to analyze hddsuperclone logs, as long as they are not messed up by multiple runs and changed settings.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 25th, 2019, 19:43 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Quote:
I've recovered two drives of mine with upward of 200 bad sectors, one was a 100% success with only a NTFS system file corrupted and the other had 6 corrupted files out of 3TB, and those were from the dreaded Seagate STx000DM001 series !), but now I realize that 4TB is a lot to handle for a drive with even minor issues ; and also that what seems like minor issues at first can quickly evolve into serious trouble.
And I have an older 2 head drive drive that has a weak head, and has survived many many testing hours with little change. And also a 6 head drive with a dying head that survived my recovery attempts to get the most out of it. But a head (or even heads) can die at any time if the drive is failing. That is the nature of DIY.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 25th, 2019, 21:52 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Quote:
With the weak/bad head, I would stop the cloning after phase 2 and assess the results. Phases 1 and 2 get the most good data from the good heads with the least effort, after that it can start to thrash the drive. It is all about understanding what you are dealing with. I offer to analyze hddsuperclone logs, as long as they are not messed up by multiple runs and changed settings.

It seems to me that even in phase 1 it spends too much time on problematic areas – but then again it's an outdated version I was using, I'll definitely update it next time I use it. (I'll run some tests until I get a reply from the owner on how he wants to proceed.)

Quote:
I offer to analyze hddsuperclone logs, as long as they are not messed up by multiple runs and changed settings.

Here it is :
Attachment:
WD40EFRX.log [192.18 KiB]
Downloaded 840 times

Normally it shouldn't be messed too much, I made two runs, the long one starting at the begining up until about 3.660.000.000 sectors, and a short one around 7.000.000.000 sectors ; when there were mistakes (as I tried to set the start position) I re-opened it and restored the log file from a backup copy.


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 26th, 2019, 7:57 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Quote:
It seems to me that even in phase 1 it spends too much time on problematic areas – but then again it's an outdated version I was using, I'll definitely update it next time I use it. (I'll run some tests until I get a reply from the owner on how he wants to proceed.)
There is something wrong with the skipping in the version you are using, and the skip size is not growing, possibly due to a bug with the skip resets. I don't remember that specific issue (too long ago), but that doesn't mean it didn't exist. It is supposed to adjust so that it only performs about 7 reads in each section of the bad head during phases 1 and 2.

The bad news is that it looks like the head died and is not reading any data. Make sure you don’t do a zero fill again so as not to possibly wipe out any data that has been recovered while the head was working, just to be safe. Also, there is always the chance that the head could cause platter damage, making it more difficult for professional recovery of that surface, so any further attempts should be done so with that knowledge. FYI according to search results that model has 4 platters 8 heads.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 26th, 2019, 9:11 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
So... noone else has any clue on that matter ? Or did I unknowingly say or do something incredibly stupid to elicit a “polite silence” ? :(

Regarding the high “start / stop count”, could it be related with the “Intellipark” feature ? Is this indeed known to reduce the longevity of concerned HDDs ? I've read contradictory statements on the subject, some knowledgeable users advise to disable this feature with “wdidle3”, but apparently the manufacturer advises against it.

So, the owner of the drive told me that I could go on and do whatever I could (since among the files found by R-Studio by way of “raw” analysis of the as-of-yet recovered data which I examined and described to him are already the ones he most wishes to recover).
I'm still looking for any method specific to the Ext4 filesystem which could help target the recovery to the allocated data. If there's no specific tool for such purposes, would it be a good idea to use a defragmentation program in analysis mode only, to get a graphic representation of allocated / unallocated areas ?
Also, is it possible, generally speaking, that a significant amount of files get written at the end of a volume with that much free space in the middle ? Aren't filesystems supposed to write files on the outermost cylinders whenever possible, where the read/write performance is better ?
And what could explain the discrepancies between the ddrescue images and the HDDSuperClone clone ?
Is it at all possible that a defective HDD would transfer only 00s at a high rate from areas which actually do contain data ?

@maximus :
A nice feature for HDDSuperClone would be the ability to verify the recovered portions of a clone / image against the source and report any discrepancy in an extra log file.


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 26th, 2019, 9:25 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
@maximus :
Again, I hadn't seen your post when I sent the one above -- but I'm surprised that noone else chimed in, since the thread has 219 views.

Quote:
The bad news is that it looks like the head died and is not reading any data. Make sure you don’t do a zero fill again so as not to possibly wipe out any data that has been recovered while the head was working, just to be safe. Also, there is always the chance that the head could cause platter damage, making it more difficult for professional recovery of that surface, so any further attempts should be done so with that knowledge. FYI according to search results that model has 4 platters 8 heads.

In what circumstances could the “zero fill” mode wipe duly recovered data ? Could it explain the discrepancies I observed, or is it unlikely ? In any case, it would have been safer to do the zero fill before any recovery attempt... :?
The owner specifically replied that he definitely wouldn't pay for a “HEPA room” recovery, and that I could proceed, even if it completely trashes the drive. As I wrote above, I described him just a few of the files I previewed in R-Studio and they were among the ones he most cares about (family pictures / videos), which are therefore among the ~430GB already recovered (but as I explained they should not be fragmented to be recovered correctly by this method, in case I don't manage to retrieve the complete metadata structures).
Yes I have found the same information about the number of heads.


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 26th, 2019, 9:59 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
I guess the only reason for not zero filling is if you start a different recovery and used that log. If you continue with the current one, it would be safe. Just make sure you are zero filling with the correct log, because if you mess up you can't get some of that data back again.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 27th, 2019, 0:20 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Quick update :
According to R-Studio, opening the main partition from the recovery drive in its current state, the $Journal file is located at sector 3900964864, or 1860GB (and of course is currently empty). How stupid is that... (For recovery purposes at least, but I can't think of a logical explanation ; considering that these files are accessed very often they should be in the fastest area, at the begining, as is the case with NTFS system files.)
R-Studio somehow (I'd be curious to know how – what file contains references to all the others ?) manages to retrieve the LBAs of many (all ?) “$INodeTable” files, which I suppose contain metadata information. It would be nice if the “Save names to file” feature could include the dates and size (like the freeware Recuva does) and also the number of the first sector : it would avoid the hassle of having to open the Hex viewer for each file just to read that information ; there could even be an optional column to display that information right away. (While I'm off topic, an optional checksum calculation during the scan would be very useful, and the checkums could included in the list of files. It would allow to detect large duplicated files without extracting them first, among other things.)
Those files are indeed all over the place apparently.
$INodeTable.8808.bin : 2306900224 / 1100GB
$INodeTable.11903.bin : 3116429568 / 1486GB
$INodeTable.28720.bin : 7528775936 / 3590GB
$INodeTable.29095.bin : 7625273600 / 3636GB
...
Since there are thousands of them, it wouldn't be practically possible to define a specific interval in order to hopefully recover each one with as few reads as possible... (It would be manageable, again, if R-Studio could dump that information in a single file, then I could use a similar method as this.)

I tried to resume the cloning (with the newer version of HDDSuperClone), but even the good reads were much slower than before (~3MB/s). Could it be related with the “slow issue”, regularly mentioned here ? Is this range of WD drives affected ? If so, would fixing that issue with HDDSuperTool significantly improve the copy rate, with a minimal risk of screwing it up for good in the process ?
As for changing the starting position before a copy run (which, I realize now, is different from the “input position” which affects the whole recovery), I can't find another way than editing the value in the log file. Is it how it's supposed to be done (I tried to RTFM but found nothing relevant) or am I missing something ? (Could be, I'm lacking sufficient sleep lately !)

I found an equivalent command to Robocopy /CREATE :
cp -R --attributes-only
But the partition on the source drive can no longer be mounted – perhaps because of that same “slow issue” ? Is it at least consistent with that possibility, considering that there are now more than 1100 pending sectors ?


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 27th, 2019, 8:19 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
Quote:
As for changing the starting position before a copy run (which, I realize now, is different from the “input position” which affects the whole recovery), I can't find another way than editing the value in the log file. Is it how it's supposed to be done (I tried to RTFM but found nothing relevant) or am I missing something ?
You are not missing anything, the option is not there. And the reason is that manually jumping around can mess with the algorithm, and make for a less efficient recovery.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 27th, 2019, 13:02 
Offline

Joined: February 16th, 2016, 21:07
Posts: 43
Location: Boston, USA
The GParted error message in your original post I suspect is showing the output of dumpe2fs -h ExtPartitionName. And it shows the meta data of a damaged Ext superblock with 553.46GB of space used out of 3.63TB, or about 15%. And viewing the HDDSuperClone log file you posted shows mostly a sea of green for the first 43% of the drive. So certainly at this point most user data has been imaged. The task now is to extract the data from the 43% partial image.

In my remote data recovery practice I use a technique I call drive hybridization which takes a partial image and joins it to the unimaged remainder of the source drive to create a hybrid drive that is both part image and part hard drive which can then be used for data recovery operations. In today's world of multi TB drives, performing full drive imaging can be very inefficient and places unneeded stress on the source drive. If we say that copying one sector one time equals one unit of drive stress then full drive imaging would earn the max stress score of 100% while the stress score for partial imaging can be considerably less. And really what's the point of imaging TB's of nothing but zeros?

Anyway if you would like help with your case let me know.

_________________
On-Line Data Recovery Consultant. RAID / NAS / Linux Specialist.
Serving clients worldwide since 2011
FreeDataRecovery.us


Top
 Profile  
 
 Post subject: Re: WD40EFRX, issues when cloning with HDDSuperClone
PostPosted: May 27th, 2019, 21:38 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
So... I resumed the recovery, it's still copying only zeroes as far as I can tell, the rate is correct on readable parts (20-50MB/s – previously I had forgotten that it was set to “reverse”, which might explain the much slower rate observed yesterday).
Without further insight on that matter, I applied both “slow issue” fixes with HDDSuperTool, but even then, the partition can no longer be mounted on the original drive, while GParted does no longer display its allocated size. And the “pending sector count” keeps increasing. Shouldn't those fixes completely disable the monitoring of bad sectors ?
(But today I can no longer get GSmartControl to work. Yesterday I was still running the older HDDLiveCD version and just updated HDDSuperClone – the shortcut was no longer working but launching it from the terminal did work. Today I updated the ISO on the SD card, installed GSmartControl as I did before, but it no longer appears in the start menu subfolder “System tools”, and if launched from the terminal, with “sudo” it shuts down almost immediately with the message “terminate called after throwing an instance of 'std::out_of_range'” preceded by various “<warn>” lines, without “sudo” it doesn't shut down but fails to detect any connected storage device ; removed and reinstalled, same result.)

Found this, after loading the partial 12GB ddrescue image in WinHex and clicking on “Technical Detail Reports” :
Code:
File system: Ext4
Total capacity: 3 995 958 509 568 bytes = 3,6 TB
Sector count: 7 804 606 464
Bytes per sector: 512
Bytes per cluster: 4 096
Free clusters: 830 488 873 = 85% free
Total clusters: 975 575 808
No. of Inodes: 243 900 416
No. of free Inodes: 243 819 640
No. of block groups: 29 773
Blocks per group: 32 768
Inodes per group: 8 192
Inode size: 256
[b]Uses sparse superblocks: Yes[/b]
Last mount time: 15/04/2019, 13:26:30
Last write time: 15/04/2019, 13:59:43

Could it mean that all files are written in “sparse” mode, and could it explain the fact that I only recovered 438GB worth of data, even though the actual size is 553GB ? Or is it something else entirely ?
If files are indeed written as “sparse”, will R-Studio or Photorec extract them correctly in “raw file carving” mode ? (For those which are not fragmented but do contain chunks of empty data.)

Also, it turns out that Ext4 uses a similar system as the $Bitmap in NTFS to indicate which blocks are allocated / unallocated. The difference is that this structure is divided in small $BlockBitmap.xxxx.bin files, 29773 it would seem, as many as there are “block groups”. If each “$BlockBitmap” file contains 4096 bytes and each byte codes the status of 8 blocks, it would require (7804606464 * 512) / (4096 * 8 * 4096) = 29772.21, that seems about right.
So, theoretically, it would be manageable to extract all these files, append them, and use the same algorithm as ddru_ntfsbitmap to generate a mapfile (or even that tool itself, if it can analyse a standalone $Bitmap file, don't remember).
But it wouldn't help much in my situation, since many of the $BlockBitmap files are beyond what's been recovered so far... ($BlockBitmap.29772.bin for instance is located at sector 7801405536 / 7804616464, at the very end of the partition and of the whole physical volume.)


Quote:
You are not missing anything, the option is not there. And the reason is that manually jumping around can mess with the algorithm, and make for a less efficient recovery.

In this particular case, what's very inefficient is spending hours copying hundreds of GB of 00s with 7 heads out of 8 when what I want is the missing blocks of metadata and (possibly) 115GB worth of actual data. But with the way those metadata files are organized it seems impossible at this point to do it another way. Had I known all that from the begining I may have been able to device a neat trick (what I wrote above seems to be on the right track), but now it most likely wouldn't be possible to dump all the required files from the source.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 40 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 56 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group