All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 46 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
 Post subject: Re: Corrupted File Finder
PostPosted: November 20th, 2018, 16:17 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 11010
Location: Australia
I was thinking that @DRUG might be able to make sense of DFL's map file, in which case he could write code specifically for you. AISI, all one would need to do would be to convert DFL's maps to ddrescue's log format, then run ddrutility against the log. BTW, the author of ddrutility is @maximus.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 20th, 2018, 16:20 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 9981
Location: Portugal
Try to compress the map file with something like winrar ....

If the pattern on the file is almost the same for the entire file it will compress nicely ....

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 20th, 2018, 18:09 
Offline

Joined: December 8th, 2010, 11:37
Posts: 596
Location: Ottawa, Canada
Here are the only two files that may be useful, as far as I know. I don't know how useful they will be; they boggle my (easily boggled) mind. :?


Attachments:
SATA_0.zip [185.83 KiB]
Downloaded 25 times
DDP Drive Image Task.zip [4.43 MiB]
Downloaded 24 times

_________________
Sabo Computer Repairs & Data Recovery
Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 20th, 2018, 19:07 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 11010
Location: Australia
ISTM that it shouldn't be too difficult to add DFL's skipped sectors to a ddrescue log as bad sectors (-).

excerpt from a ddrescue log

Code:
#      pos        size  status

0x00000000  0x00018000  +
0x00018E00  0x00000200  -
0x00019000  0x01061000  +

DFL skipped sectors example

Code:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00002400  00 00 00 00 B8 32 30 31 38 2F 31 31 2F 31 37 20  ....¸2018/11/17
00002410  31 36 3A 30 32 3A 32 38 20 20 20 20 3D 3D 3E 3E  16:02:28    ==>>
00002420  20 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E   ...............
00002430  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002440  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002450  20 4C 42 41 3A 20 5B 39 36 34 30 37 30 35 2D 2D   LBA: [9640705--
00002460  2D 2D 39 36 34 30 37 31 30 5D 20 53 6B 69 70 70  --9640710] Skipp
00002470  65 64 20 53 74 65 70 3A 36 0A 00 00 00 00 00 00  ed Step:6.......

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 6:52 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 199
Location: Portugal
LarrySabo wrote:
The drive in question is a 2TB drive from a My Book containing TV show recordings, videos, music and pictures. Opening a random assortment of videos yielded no errors so I'm hopeful overall results will be "good enough." There's a limit to how much time and effort I'm able to dedicate to this job, given it's cost to the client. Plus, I'm lazy. :shock:


This is what made me write this code. I kept having drives that had some % of zeroed / blank files (FF,BB,00) and I always like to provide the client with the best information possible about the percentage of files recovered.

I'll pm you soon @LarrySabo.

_________________
BTC Wallet - 16S1yq41ehJr9Kh8GeA2cgSYLt397XKU4Q


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 10:12 
Offline
User avatar

Joined: February 9th, 2009, 16:13
Posts: 2105
Location: Ontario, Canada
This tool might be useful to those who are limited to RapidSpar. If you don't make the search pattern editable, are you able to add <unreadable> as one of the search pattern options?

_________________
Luke
RAID Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 12:42 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 199
Location: Portugal
fzabkar wrote:
Code:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00002400  00 00 00 00 B8 32 30 31 38 2F 31 31 2F 31 37 20  ....¸2018/11/17
00002410  31 36 3A 30 32 3A 32 38 20 20 20 20 3D 3D 3E 3E  16:02:28    ==>>
00002420  20 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E   ...............
00002430  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002440  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002450  20 4C 42 41 3A 20 5B 39 36 34 30 37 30 35 2D 2D   LBA: [9640705--
00002460  2D 2D 39 36 34 30 37 31 30 5D 20 53 6B 69 70 70  --9640710] Skipp
00002470  65 64 20 53 74 65 70 3A 36 0A 00 00 00 00 00 00  ed Step:6.......


@fzabkar

What is DFL default fill on unreadbale sectors? 2E?

_________________
BTC Wallet - 16S1yq41ehJr9Kh8GeA2cgSYLt397XKU4Q


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 13:59 
Offline

Joined: December 8th, 2010, 11:37
Posts: 596
Location: Ottawa, Canada
@DRUG, fill characters for DFL should be 42h according to my results in the Spildit Imager Challenge: "The DDP program writes a sector full of 42h ("B") in place of unreadable sectors." However, the recovery job just competed shows 00h as fill characters, so I'm confused and need to do more research/testing.

_________________
Sabo Computer Repairs & Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 15:23 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 9981
Location: Portugal
Can't you just configure DFL to fill the un-readable sectors with a pattern of your choice ?

This is my HRT-DRE (data recovery edition) data extraction / cloning&imaging tool :

Attachment:
1.jpg
1.jpg [ 83.84 KiB | Viewed 723 times ]


You chose the pattern that you want to use to fill the LBA space on the image/clone for the sectors that can't be read and the ones that are skipped from the original drive.

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 15:48 
Offline

Joined: December 8th, 2010, 11:37
Posts: 596
Location: Ottawa, Canada
Spildit wrote:
Can't you just configure DFL to fill the un-readable sectors with a pattern of your choice ? ...


None that I can find in the settings.

_________________
Sabo Computer Repairs & Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 23rd, 2018, 13:54 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 199
Location: Portugal
Ok so.

Some bugs* were fixed and:

added suport for DFL default <unreadable>
added support for MRT fill !bad option

bugs*:
0KB files we're not being marked as corrupted.
Maintaining directory structure on corrupted files folder

Still on work:

Displaying file hex content on a box
Creating a Snap2HTML report of the corrupted files (still being considered)

I'm also open to sugestions from users as to other functionalities of this software so please be my guest and ask for it.

_________________
BTC Wallet - 16S1yq41ehJr9Kh8GeA2cgSYLt397XKU4Q


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 23rd, 2018, 16:01 
Offline

Joined: December 8th, 2010, 11:37
Posts: 596
Location: Ottawa, Canada
DRUG wrote:
added suport for DFL default <unreadable>


Actually, I think <unreadable> is the Rapidspar fill pattern, but no matter.

_________________
Sabo Computer Repairs & Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 23rd, 2018, 18:06 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 199
Location: Portugal
LarrySabo wrote:
DRUG wrote:
added suport for DFL default <unreadable>


Actually, I think <unreadable> is the Rapidspar fill pattern, but no matter.


You are right, my typo. Fixed :)

_________________
BTC Wallet - 16S1yq41ehJr9Kh8GeA2cgSYLt397XKU4Q


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 25th, 2018, 20:49 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 199
Location: Portugal
So, weekend updates:

Two kinds of operations:

Check for 00 or FF or BB percentage in a group of files

or

Do a string check:

User will input is string in hex and the software will run and marks the ones found with such string.

Working now on a auto-update feature so any changes will be made to all users.


Attachments:
1.png
1.png [ 209 KiB | Viewed 558 times ]
2.png
2.png [ 204.88 KiB | Viewed 558 times ]

_________________
BTC Wallet - 16S1yq41ehJr9Kh8GeA2cgSYLt397XKU4Q
Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 25th, 2018, 20:54 
Offline

Joined: December 8th, 2010, 11:37
Posts: 596
Location: Ottawa, Canada
Does it scan folders and sub-folders recursively or does the user need to select each folder individually?

_________________
Sabo Computer Repairs & Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 26th, 2018, 4:39 
Offline
User avatar

Joined: July 12th, 2010, 4:38
Posts: 1103
Location: Portugal
It does folder and sub-folder scan.

_________________
http://www.facebook.com/PCLAB.Assistencia.Tecnica
ACELab partner


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 26th, 2018, 13:01 
Offline
User avatar

Joined: February 9th, 2009, 16:13
Posts: 2105
Location: Ontario, Canada
Not to put a wrench in this project, it surely is something that is needed. But, wouldn't it be nice if R-TT were to add the ability to search and select files within R-Studio Tech based on cross-referencing file contents with a pattern entered by the user? It kind of sucks that R-Studio already shows a file's condition by cross-referencing the DeepSpar sector map on the clone, yet after a few years of my requesting, still haven't provided any method for using that information to filter the files.

_________________
Luke
RAID Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 27th, 2018, 18:49 
Offline

Joined: January 29th, 2012, 1:43
Posts: 473
Location: United States
I tried to see if there was any other tool or even Linux command that could do this usefully for hex strings, and I came up short, so this could definitely be a useful tool. Don’t be surprised if I write one for Linux at some point in time, maybe a future pro feature of hddsuperclone ;)

But I would like to point out that if you have some sort of map file that can be converted to ddrescue format, then there is a way to use the pro version of HDDSuperClone along with a Linux licensed copy of R-Studio to know exactly which files have the bad sectors. It involves using the virtual disk feature to present the recovery as a virtual block device to the system, which requires the clone/image and a map file (called a progress log in hddsuperclone, which can be an imported ddrescue map file). You then use R-Studio to extract the files from the virtual disk, which will give I/O errors for the bad files (according to the map) so they can be sorted by R-Studio. But that does require converting whatever map file format the imager created to a ddrescue one. And I don’t plan on adding any other import format into hddsuperclone at this time, so someone would have to either write a converter, or convince me that there would actually be enough demand for me to do it. I just wanted to point out that it could be possible, although not as easy as one would like.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 27th, 2018, 19:01 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 11010
Location: Australia
maximus wrote:
I tried to see if there was any other tool or even Linux command that could do this usefully for hex strings, and I came up short, ...

If you were to search for text strings, would grep be suitable? :?

maximus wrote:
I don’t plan on adding any other import format into hddsuperclone at this time, so someone would have to either write a converter, or convince me that there would actually be enough demand for me to do it.

I've written some simple code to extract the bad sectors from a DFL DE map and convert them to ddrescue format. Would the following map be sufficient for HDDSuperClone to process, or do I need to include the ranges for the good sectors?

Code:
# Tool:  DFDE
# Model:  WDC WD20EZRX-00D8PB0                   
# Serial:  WD-WCC4M5KHJNKA JNKA
# Firmware:  80.00A80
# Total LBAs:  3906963633 / 0xE8DF88B1
# Logical / Physical Sector Size:  512 / 4096 bytes

#      pos        size  status

0x2000  0x200  -
0x10D2E000  0x200  -
0x10D2EA00  0x200  -
0x10EBF000  0x200  -
0x10EBF200  0x200  -
0x10EBF400  0x200  -
0x10EBF600  0x200  -
0x10EBF800  0x200  -
0x10EBFA00  0x200  -
0x10EBFC00  0x200  -
0x10EBFE00  0x200  -
0x121C4400  0x200  -
0x12355A00  0x200  -
0x1280AE00  0x200  -

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 27th, 2018, 19:36 
Offline

Joined: January 29th, 2012, 1:43
Posts: 473
Location: United States
fzabkar wrote:
If you were to search for text strings, would grep be suitable? :?

Sort of, but grep would be looking for that string anywhere in the file. In the event of a bad sector, the pattern would be in 512 byte blocks (typical logical sector size), so you would really only need to check the first part of every 512 byte offset for the pattern, and then if it matches, optionally make sure the whole 512 byte block continues to match the pattern. Only checking on 512 byte boundaries would speed it up. Grep can't do that.

fzabkar wrote:
I've written some simple code to extract the bad sectors from a DFL DE map and convert them to ddrescue format. Would the following map be sufficient for HDDSuperClone to process, or do I need to include the ranges for the good sectors?

Actually, the good ranges are more important than the bad in this case. The virtual driver will present any area that was not successfully read as an IO error. So the answer is yes, the good ranges would need to be included.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 46 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group