Damaged external HDD, how do I recover its data efficiently?

[Modinstaller]

Hello everyone,



I have an external HDD which was dropped several times and started acting up. By the looks of it, it's good for the trash bin. At this point I'm only trying to save the data that's on it.
My knowledge in the field is lacking, I'm backing up some files right now but it's very inefficient and time-consuming so I'd like to know if there's any better way of doing it.
This is my partner's first drive and she didn't know she had to treat it like a newborn baby. We don't really have 500-1000€ to shell out hence why I'm trying to recover as much data myself as possible. For sure next time I am teaching her about backups and everything.
Anyway,


About the state of the drive:
It works but very slowly. Parts of the disc are damaged since some files will read fine and others won't (and will freeze windows explorer and teracopy in an infinite loop). The drive sometimes disconnects, especially when trying to read bad files and looping for a long time.
It's very slow to show its files, copying takes a very long time. Some files are instant to copy, others take some time, others freeze.
Interestingly, for about 20 seconds after I plug it in, it works at a normal speed, though there's still the issue with files that are damaged beyond recovery freezing the drive.


About the backing up:
I can't just copy entire folders and wait. The problem is, when the drive gets to a bad file, it'll freeze and at some point eventually disconnect. I don't know how to tell the drive, or teracopy or whatever, to skip bad files and continue copying everything that's readable.
Hence if I have a folder with a thousand files in it and the 10th one is bad, I'm only going to copy 9 files out of potentially hundreds readable ones. Because of this, I've taken to manually going into sub-folders and copying sometimes file by file, plugging the drive out and in when I encounter a bad one and logging which one it is for reference.
On top of this, since the drive is really only efficient for a couple seconds after plugging it in, I've taken to plugging it in and out repeatedly and copying stuff very fast.
As you can imagine, this is insanely time-consuming. I just got to a folder with 155 files in it. I really need a better solution.


I'd like to know if there is any specialized tool that could parse through the entire drive, backing up files and folders, skipping the ones that take too long.
Or any other way of backing up the drive's data without too much human interaction - something that could save me hours of pain.

I haven't tried any of the regular tools (clonezilla etc..) because I figure when they get to a bad sector, the disk will act up, freeze or disconnect, and the program will miserably fail like all the scans I've used so far (including chkdsk). Are there any worth trying anyway?



Thank you for your time!

[Lardman]

Stop killing the drive trying to copy files off. Clone the drive onto a new one or into an image file with something that can handle failing drives. hddsuperclone is probably the easiest http://www.hddsuperclone.com/sitev1/ there's an iso to download and make a bootable usb stick, alternatively use ddrescue if you're comfortable with the linux command line.

Worry about file recovery after you have imaged it.

[Modinstaller]

Thanks! I was wondering how cloning would work better than copying given that both would attempt to read and since copying freezes, then why wouldn't cloning, and the user manual of the app you linked answers my question.

I ended up copying as much as I could, but had to skip a couple files. I may have damaged it further but, well, I didn't know. I don't think I damaged it further anyway, but in any case I'm cloning it first thing, following your advice.

I only have one problem: it's a 4TB drive. Does that mean even though only 150GB are used in it, I need 4TB free on another drive? Because that's gonna be hard to do unless I buy an entirely new one.

[Lardman]

I only have one problem: it's a 4TB drive. Does that mean even though only 150GB are used in it, I need 4TB free on another drive? Because that's gonna be hard to do unless I buy an entirely new one.

Yes, and another drive to extract the data onto from the clone. You'll probably need a new drive to replace the dead one any dont you?

[Modinstaller]

Ok. Once I've cloned it, will the data be readily usable (means we can copy whichever file is intact off of it), or will it require cleaning up before that?

When we've recovered everything we want from the clone and we're sure we don't want it anymore, can we simply format the recovery drive?

[Lardman]

Ok. Once I've cloned it, will the data be readily usable (means we can copy whichever file is intact off of it), or will it require cleaning up before that?

It depends on the level of damage - if you can currently still access them it's probable you will still be able to (without errors) on the clone.

When we've recovered everything we want from the clone and we're sure we don't want it anymore, can we simply format the recovery drive?

Yes.

[Modinstaller]

I've finally received the new hdd and tried to let hddsuperclone do its thing at night. I always wake up to an "unable to reopen source device" error, then the cloning stops.

Googling this does not show any relevant results.

I can just restart where it left off, but if I reboot (I used the flash drive iso option) the project file is gone and the cloning has to take over from the beginning. So I'm just going to let it run for 3 days straight and see if it completes.

Very sorry about the shitty screenshot but it's all I had, here's the extended analysis:

https://cdn.discordapp.com/attachments/740863506429247571/1020277806686543913/1663323565564.jpg

By the way, it shows a ton of sectors as good because the data actually takes less than 5% of the disk. It's 95% 0's. Is there really not a way to tell it to only clone the first 200GB and leave the rest?

[Lardman]

You can put another usb pen drive in and just save the log file over to it, it's a simple text file.

Personally I'd change your starting point to +15% into the drive and set the phase 1 timeout to 350ms rather than the default which I think is 1 second and continue the clone from there. Once the good sectors are copied you can pick up the rest on the other phases and dig deeper by increasing the timeout.

With the pro version you can create a virtual driver and then link that to dmde, r-studio et al and use that to clone based of mft if it isn't in the damaged area which it may well be.

[Modinstaller]

Hey, quick update just to say that I'm at the scraping phase, 14k non-scraped sectors remaining, 14k bad ones. That's between 7 and 14 megabytes of unrecoverable data which is really good for 150GB.

I ended up doing this on my laptop which has been running non-stop for some days, and I didn't limit the area read. There were things to read at the end.
It would appear the heads might be weak, but not damaged enough to harm the data. I suppose I got lucky because all my copying and cloning could've very well done big damage. The disk has shut down several times during the cloning. When that happens I just reconnect it and tell hddsuperclone to resume.
Anyway, I'm happy to say I now have a much better understanding of how hddsuperclone and disks/disk recovery work.

Couple of quick questions:

1. After scraping I'll attempt a retry on all the bad sectors. Is there anything I can do to maximize success rate? I'll monitor how many bad sectors are recovered that way, if any. Will there be any use attempting a retry twice?
2. If say a RAW image has a couple bad sectors inside it, and what we recover has a couple zeroes in the middle, what kind of effect can we expect on the image? Will it be readable at all? Will it simply be a couple black/wrong color pixels? Will it have more visible effects such as parts of the image repeating, or glitchy looking images?
3. From the info I gave, can we guess what happened to the disks/heads during the fall? I'm just curious!

Thanks again for your help, hddsuperclone is great

Edit: last question!
You know how windows can mark sectors as bad and replace them with new "replacement" sectors, with 'chkdsk /r' ?
How does that interact with a cloning? Does the cloning copy the "bad sector pointers"? (not sure how that works in the first place)
Or does the cloning skip the sectors marked as bad by chkdsk? Or are the "bad sector pointers" on another part of the disks or part of the controller, which isn't accessed by the cloning - so nothing will be skipped and no pointers will be copied?

[Modinstaller]

I answered some of my own questions:

- Retrying is, in my case, useful. The recovery ended with 18,432 bad sectors, after one retry it was 15,920, after two 14,920 (oddly) and after ten 12,336.
- The raw images are actually CR2 and I'm assuming there's some kind of lossless compression happening since the images with missing data look glitchy. Windows makes a preview and that preview is sometimes okay - I assume the preview skips the bad data by happenstance. Interestingly, in one case, an image looked glitchy in the preview, but not when fully displayed, which I can't explain and makes me very curious as to how that's even possible.

I was able to recover everything that had no problem copying, and a couple of files that did. Some I expected to recover (some folders were just loading so slowly I never bothered copying the files while I knew some of them were good), some I didn't (files that never copied in 30 minutes of waiting). I'm retrying 8 times by 8 times since the program won't let me queue more. It might kill the drive but at this point, given what it's been through and the fact that it seems to be mostly intact, tells me it likely won't, and if it does then what the hell - I have everything I need.

In other words thanks a ton and my s.o (whose drive it was) wants me to tell you she wants to kiss you on the forehead - that's one of her highest "thank you" tiers.

[fzabkar]

The raw images are actually CR2 and I'm assuming there's some kind of lossless compression happening since the images with missing data look glitchy. Windows makes a preview and that preview is sometimes okay - I assume the preview skips the bad data by happenstance. Interestingly, in one case, an image looked glitchy in the preview, but not when fully displayed, which I can't explain and makes me very curious as to how that's even possible.

CR2 files contain the raw image data plus a high (full?) resolution JPEG. Perhaps one is corrupt while the other is intact.

[Lardman]

I'm retrying 8 times by 8 times since the program won't let me queue more. It might kill the drive but at this point, given what it's been through and the fact that it seems to be mostly intact, tells me it likely won't, and if it does then what the hell.

It's surprising how many "bad" sectors you can pick up on multiple retries and passes and also when the drive changes it's thermal state.

In other words thanks a ton and my s.o (whose drive it was) wants me to tell you she wants to kiss you on the forehead - that's one of her highest "thank you" tiers.

I think making a regular backup would be more beneficial, but what do I know.

[Arch Stanton]

The raw images are actually CR2 and I'm assuming there's some kind of lossless compression happening since the images with missing data look glitchy. Windows makes a preview and that preview is sometimes okay - I assume the preview skips the bad data by happenstance. Interestingly, in one case, an image looked glitchy in the preview, but not when fully displayed, which I can't explain and makes me very curious as to how that's even possible.

CR2 files contain the raw image data plus a high (full?) resolution JPEG. Perhaps one is corrupt while the other is intact.

Correct!

RAW data is lossless JPEG in case of CR2. The embedded JPEG preview is full res medium quality.