HDD GURU FORUMS
http://forum.hddguru.com/

Corrupted File Finder
http://forum.hddguru.com/viewtopic.php?f=15&t=37662
Page 2 of 3

Author:  fzabkar [ November 20th, 2018, 16:17 ]
Post subject:  Re: Corrupted File Finder

I was thinking that @DRUG might be able to make sense of DFL's map file, in which case he could write code specifically for you. AISI, all one would need to do would be to convert DFL's maps to ddrescue's log format, then run ddrutility against the log. BTW, the author of ddrutility is @maximus.

Author:  Spildit [ November 20th, 2018, 16:20 ]
Post subject:  Re: Corrupted File Finder

Try to compress the map file with something like winrar ....

If the pattern on the file is almost the same for the entire file it will compress nicely ....

Author:  LarrySabo [ November 20th, 2018, 18:09 ]
Post subject:  Re: Corrupted File Finder

Here are the only two files that may be useful, as far as I know. I don't know how useful they will be; they boggle my (easily boggled) mind. :?

Attachments:
SATA_0.zip [185.83 KiB]
Downloaded 76 times
DDP Drive Image Task.zip [4.43 MiB]
Downloaded 77 times

Author:  fzabkar [ November 20th, 2018, 19:07 ]
Post subject:  Re: Corrupted File Finder

ISTM that it shouldn't be too difficult to add DFL's skipped sectors to a ddrescue log as bad sectors (-).

excerpt from a ddrescue log

Code:
#      pos        size  status

0x00000000  0x00018000  +
0x00018E00  0x00000200  -
0x00019000  0x01061000  +

DFL skipped sectors example

Code:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00002400  00 00 00 00 B8 32 30 31 38 2F 31 31 2F 31 37 20  ....¸2018/11/17
00002410  31 36 3A 30 32 3A 32 38 20 20 20 20 3D 3D 3E 3E  16:02:28    ==>>
00002420  20 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E   ...............
00002430  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002440  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002450  20 4C 42 41 3A 20 5B 39 36 34 30 37 30 35 2D 2D   LBA: [9640705--
00002460  2D 2D 39 36 34 30 37 31 30 5D 20 53 6B 69 70 70  --9640710] Skipp
00002470  65 64 20 53 74 65 70 3A 36 0A 00 00 00 00 00 00  ed Step:6.......

Author:  DRUG [ November 21st, 2018, 6:52 ]
Post subject:  Re: Corrupted File Finder

LarrySabo wrote:
The drive in question is a 2TB drive from a My Book containing TV show recordings, videos, music and pictures. Opening a random assortment of videos yielded no errors so I'm hopeful overall results will be "good enough." There's a limit to how much time and effort I'm able to dedicate to this job, given it's cost to the client. Plus, I'm lazy. :shock:


This is what made me write this code. I kept having drives that had some % of zeroed / blank files (FF,BB,00) and I always like to provide the client with the best information possible about the percentage of files recovered.

I'll pm you soon @LarrySabo.

Author:  lcoughey [ November 21st, 2018, 10:12 ]
Post subject:  Re: Corrupted File Finder

This tool might be useful to those who are limited to RapidSpar. If you don't make the search pattern editable, are you able to add <unreadable> as one of the search pattern options?

Author:  DRUG [ November 21st, 2018, 12:42 ]
Post subject:  Re: Corrupted File Finder

fzabkar wrote:
Code:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00002400  00 00 00 00 B8 32 30 31 38 2F 31 31 2F 31 37 20  ....¸2018/11/17
00002410  31 36 3A 30 32 3A 32 38 20 20 20 20 3D 3D 3E 3E  16:02:28    ==>>
00002420  20 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E   ...............
00002430  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002440  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002450  20 4C 42 41 3A 20 5B 39 36 34 30 37 30 35 2D 2D   LBA: [9640705--
00002460  2D 2D 39 36 34 30 37 31 30 5D 20 53 6B 69 70 70  --9640710] Skipp
00002470  65 64 20 53 74 65 70 3A 36 0A 00 00 00 00 00 00  ed Step:6.......


@fzabkar

What is DFL default fill on unreadbale sectors? 2E?

Author:  LarrySabo [ November 21st, 2018, 13:59 ]
Post subject:  Re: Corrupted File Finder

@DRUG, fill characters for DFL should be 42h according to my results in the Spildit Imager Challenge: "The DDP program writes a sector full of 42h ("B") in place of unreadable sectors." However, the recovery job just competed shows 00h as fill characters, so I'm confused and need to do more research/testing.

Author:  Spildit [ November 21st, 2018, 15:23 ]
Post subject:  Re: Corrupted File Finder

Can't you just configure DFL to fill the un-readable sectors with a pattern of your choice ?

This is my HRT-DRE (data recovery edition) data extraction / cloning&imaging tool :

Attachment:
1.jpg
1.jpg [ 83.84 KiB | Viewed 2827 times ]


You chose the pattern that you want to use to fill the LBA space on the image/clone for the sectors that can't be read and the ones that are skipped from the original drive.

Author:  LarrySabo [ November 21st, 2018, 15:48 ]
Post subject:  Re: Corrupted File Finder

Spildit wrote:
Can't you just configure DFL to fill the un-readable sectors with a pattern of your choice ? ...


None that I can find in the settings.

Author:  DRUG [ November 23rd, 2018, 13:54 ]
Post subject:  Re: Corrupted File Finder

Ok so.

Some bugs* were fixed and:

added suport for DFL default <unreadable>
added support for MRT fill !bad option

bugs*:
0KB files we're not being marked as corrupted.
Maintaining directory structure on corrupted files folder

Still on work:

Displaying file hex content on a box
Creating a Snap2HTML report of the corrupted files (still being considered)

I'm also open to sugestions from users as to other functionalities of this software so please be my guest and ask for it.

Author:  LarrySabo [ November 23rd, 2018, 16:01 ]
Post subject:  Re: Corrupted File Finder

DRUG wrote:
added suport for DFL default <unreadable>


Actually, I think <unreadable> is the Rapidspar fill pattern, but no matter.

Author:  DRUG [ November 23rd, 2018, 18:06 ]
Post subject:  Re: Corrupted File Finder

LarrySabo wrote:
DRUG wrote:
added suport for DFL default <unreadable>


Actually, I think <unreadable> is the Rapidspar fill pattern, but no matter.


You are right, my typo. Fixed :)

Author:  DRUG [ November 25th, 2018, 20:49 ]
Post subject:  Re: Corrupted File Finder

So, weekend updates:

Two kinds of operations:

Check for 00 or FF or BB percentage in a group of files

or

Do a string check:

User will input is string in hex and the software will run and marks the ones found with such string.

Working now on a auto-update feature so any changes will be made to all users.

Attachments:
1.png
1.png [ 209 KiB | Viewed 2662 times ]
2.png
2.png [ 204.88 KiB | Viewed 2662 times ]

Author:  LarrySabo [ November 25th, 2018, 20:54 ]
Post subject:  Re: Corrupted File Finder

Does it scan folders and sub-folders recursively or does the user need to select each folder individually?

Author:  pclab [ November 26th, 2018, 4:39 ]
Post subject:  Re: Corrupted File Finder

It does folder and sub-folder scan.

Author:  lcoughey [ November 26th, 2018, 13:01 ]
Post subject:  Re: Corrupted File Finder

Not to put a wrench in this project, it surely is something that is needed. But, wouldn't it be nice if R-TT were to add the ability to search and select files within R-Studio Tech based on cross-referencing file contents with a pattern entered by the user? It kind of sucks that R-Studio already shows a file's condition by cross-referencing the DeepSpar sector map on the clone, yet after a few years of my requesting, still haven't provided any method for using that information to filter the files.

Author:  maximus [ November 27th, 2018, 18:49 ]
Post subject:  Re: Corrupted File Finder

I tried to see if there was any other tool or even Linux command that could do this usefully for hex strings, and I came up short, so this could definitely be a useful tool. Don’t be surprised if I write one for Linux at some point in time, maybe a future pro feature of hddsuperclone ;)

But I would like to point out that if you have some sort of map file that can be converted to ddrescue format, then there is a way to use the pro version of HDDSuperClone along with a Linux licensed copy of R-Studio to know exactly which files have the bad sectors. It involves using the virtual disk feature to present the recovery as a virtual block device to the system, which requires the clone/image and a map file (called a progress log in hddsuperclone, which can be an imported ddrescue map file). You then use R-Studio to extract the files from the virtual disk, which will give I/O errors for the bad files (according to the map) so they can be sorted by R-Studio. But that does require converting whatever map file format the imager created to a ddrescue one. And I don’t plan on adding any other import format into hddsuperclone at this time, so someone would have to either write a converter, or convince me that there would actually be enough demand for me to do it. I just wanted to point out that it could be possible, although not as easy as one would like.

Author:  fzabkar [ November 27th, 2018, 19:01 ]
Post subject:  Re: Corrupted File Finder

maximus wrote:
I tried to see if there was any other tool or even Linux command that could do this usefully for hex strings, and I came up short, ...

If you were to search for text strings, would grep be suitable? :?

maximus wrote:
I don’t plan on adding any other import format into hddsuperclone at this time, so someone would have to either write a converter, or convince me that there would actually be enough demand for me to do it.

I've written some simple code to extract the bad sectors from a DFL DE map and convert them to ddrescue format. Would the following map be sufficient for HDDSuperClone to process, or do I need to include the ranges for the good sectors?

Code:
# Tool:  DFDE
# Model:  WDC WD20EZRX-00D8PB0                   
# Serial:  WD-WCC4M5KHJNKA JNKA
# Firmware:  80.00A80
# Total LBAs:  3906963633 / 0xE8DF88B1
# Logical / Physical Sector Size:  512 / 4096 bytes

#      pos        size  status

0x2000  0x200  -
0x10D2E000  0x200  -
0x10D2EA00  0x200  -
0x10EBF000  0x200  -
0x10EBF200  0x200  -
0x10EBF400  0x200  -
0x10EBF600  0x200  -
0x10EBF800  0x200  -
0x10EBFA00  0x200  -
0x10EBFC00  0x200  -
0x10EBFE00  0x200  -
0x121C4400  0x200  -
0x12355A00  0x200  -
0x1280AE00  0x200  -

Author:  maximus [ November 27th, 2018, 19:36 ]
Post subject:  Re: Corrupted File Finder

fzabkar wrote:
If you were to search for text strings, would grep be suitable? :?

Sort of, but grep would be looking for that string anywhere in the file. In the event of a bad sector, the pattern would be in 512 byte blocks (typical logical sector size), so you would really only need to check the first part of every 512 byte offset for the pattern, and then if it matches, optionally make sure the whole 512 byte block continues to match the pattern. Only checking on 512 byte boundaries would speed it up. Grep can't do that.

fzabkar wrote:
I've written some simple code to extract the bad sectors from a DFL DE map and convert them to ddrescue format. Would the following map be sufficient for HDDSuperClone to process, or do I need to include the ranges for the good sectors?

Actually, the good ranges are more important than the bad in this case. The virtual driver will present any area that was not successfully read as an IO error. So the answer is yes, the good ranges would need to be included.

Page 2 of 3 All times are UTC - 5 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/