All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 44 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
 Post subject: Re: Corrupted File Finder
PostPosted: November 20th, 2018, 18:09 
Offline

Joined: December 8th, 2010, 11:37
Posts: 738
Location: Ottawa, Canada
Here are the only two files that may be useful, as far as I know. I don't know how useful they will be; they boggle my (easily boggled) mind. :?


Attachments:
SATA_0.zip [185.83 KiB]
Downloaded 602 times
DDP Drive Image Task.zip [4.43 MiB]
Downloaded 575 times

_________________
Sabo Computer Repairs & Data Recovery
Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 20th, 2018, 19:07 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15440
Location: Australia
ISTM that it shouldn't be too difficult to add DFL's skipped sectors to a ddrescue log as bad sectors (-).

excerpt from a ddrescue log

Code:
#      pos        size  status

0x00000000  0x00018000  +
0x00018E00  0x00000200  -
0x00019000  0x01061000  +

DFL skipped sectors example

Code:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00002400  00 00 00 00 B8 32 30 31 38 2F 31 31 2F 31 37 20  ....¸2018/11/17
00002410  31 36 3A 30 32 3A 32 38 20 20 20 20 3D 3D 3E 3E  16:02:28    ==>>
00002420  20 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E   ...............
00002430  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002440  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002450  20 4C 42 41 3A 20 5B 39 36 34 30 37 30 35 2D 2D   LBA: [9640705--
00002460  2D 2D 39 36 34 30 37 31 30 5D 20 53 6B 69 70 70  --9640710] Skipp
00002470  65 64 20 53 74 65 70 3A 36 0A 00 00 00 00 00 00  ed Step:6.......

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 6:52 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 413
Location: Portugal
LarrySabo wrote:
The drive in question is a 2TB drive from a My Book containing TV show recordings, videos, music and pictures. Opening a random assortment of videos yielded no errors so I'm hopeful overall results will be "good enough." There's a limit to how much time and effort I'm able to dedicate to this job, given it's cost to the client. Plus, I'm lazy. :shock:


This is what made me write this code. I kept having drives that had some % of zeroed / blank files (FF,BB,00) and I always like to provide the client with the best information possible about the percentage of files recovered.

I'll pm you soon @LarrySabo.

_________________
BTC Wallet - 3AoQPTBsz9PbfoanCx44Lw76Y2TwtKa1x5
Instagram https://www.instagram.com/datarecovery_morde.pt/


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 10:12 
Offline
User avatar

Joined: February 9th, 2009, 16:13
Posts: 2520
Location: Ontario, Canada
This tool might be useful to those who are limited to RapidSpar. If you don't make the search pattern editable, are you able to add <unreadable> as one of the search pattern options?

_________________
Luke
Recovery Force Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 12:42 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 413
Location: Portugal
fzabkar wrote:
Code:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00002400  00 00 00 00 B8 32 30 31 38 2F 31 31 2F 31 37 20  ....¸2018/11/17
00002410  31 36 3A 30 32 3A 32 38 20 20 20 20 3D 3D 3E 3E  16:02:28    ==>>
00002420  20 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E   ...............
00002430  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002440  2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E  ................
00002450  20 4C 42 41 3A 20 5B 39 36 34 30 37 30 35 2D 2D   LBA: [9640705--
00002460  2D 2D 39 36 34 30 37 31 30 5D 20 53 6B 69 70 70  --9640710] Skipp
00002470  65 64 20 53 74 65 70 3A 36 0A 00 00 00 00 00 00  ed Step:6.......


@fzabkar

What is DFL default fill on unreadbale sectors? 2E?

_________________
BTC Wallet - 3AoQPTBsz9PbfoanCx44Lw76Y2TwtKa1x5
Instagram https://www.instagram.com/datarecovery_morde.pt/


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 13:59 
Offline

Joined: December 8th, 2010, 11:37
Posts: 738
Location: Ottawa, Canada
@DRUG, fill characters for DFL should be 42h according to my results in the Spildit Imager Challenge: "The DDP program writes a sector full of 42h ("B") in place of unreadable sectors." However, the recovery job just competed shows 00h as fill characters, so I'm confused and need to do more research/testing.

_________________
Sabo Computer Repairs & Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 21st, 2018, 15:48 
Offline

Joined: December 8th, 2010, 11:37
Posts: 738
Location: Ottawa, Canada
Spildit wrote:
Can't you just configure DFL to fill the un-readable sectors with a pattern of your choice ? ...


None that I can find in the settings.

_________________
Sabo Computer Repairs & Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 23rd, 2018, 13:54 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 413
Location: Portugal
Ok so.

Some bugs* were fixed and:

added suport for DFL default <unreadable>
added support for MRT fill !bad option

bugs*:
0KB files we're not being marked as corrupted.
Maintaining directory structure on corrupted files folder

Still on work:

Displaying file hex content on a box
Creating a Snap2HTML report of the corrupted files (still being considered)

I'm also open to sugestions from users as to other functionalities of this software so please be my guest and ask for it.

_________________
BTC Wallet - 3AoQPTBsz9PbfoanCx44Lw76Y2TwtKa1x5
Instagram https://www.instagram.com/datarecovery_morde.pt/


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 23rd, 2018, 16:01 
Offline

Joined: December 8th, 2010, 11:37
Posts: 738
Location: Ottawa, Canada
DRUG wrote:
added suport for DFL default <unreadable>


Actually, I think <unreadable> is the Rapidspar fill pattern, but no matter.

_________________
Sabo Computer Repairs & Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 23rd, 2018, 18:06 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 413
Location: Portugal
LarrySabo wrote:
DRUG wrote:
added suport for DFL default <unreadable>


Actually, I think <unreadable> is the Rapidspar fill pattern, but no matter.


You are right, my typo. Fixed :)

_________________
BTC Wallet - 3AoQPTBsz9PbfoanCx44Lw76Y2TwtKa1x5
Instagram https://www.instagram.com/datarecovery_morde.pt/


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 25th, 2018, 20:49 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 413
Location: Portugal
So, weekend updates:

Two kinds of operations:

Check for 00 or FF or BB percentage in a group of files

or

Do a string check:

User will input is string in hex and the software will run and marks the ones found with such string.

Working now on a auto-update feature so any changes will be made to all users.


Attachments:
1.png
1.png [ 209 KiB | Viewed 14846 times ]
2.png
2.png [ 204.88 KiB | Viewed 14846 times ]

_________________
BTC Wallet - 3AoQPTBsz9PbfoanCx44Lw76Y2TwtKa1x5
Instagram https://www.instagram.com/datarecovery_morde.pt/
Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 25th, 2018, 20:54 
Offline

Joined: December 8th, 2010, 11:37
Posts: 738
Location: Ottawa, Canada
Does it scan folders and sub-folders recursively or does the user need to select each folder individually?

_________________
Sabo Computer Repairs & Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 26th, 2018, 4:39 
Offline
User avatar

Joined: July 12th, 2010, 4:38
Posts: 1418
Location: Portugal
It does folder and sub-folder scan.

_________________
http://www.pclab.com.pt facebook.com/PCLAB.A.T
ACELab partner


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 26th, 2018, 13:01 
Offline
User avatar

Joined: February 9th, 2009, 16:13
Posts: 2520
Location: Ontario, Canada
Not to put a wrench in this project, it surely is something that is needed. But, wouldn't it be nice if R-TT were to add the ability to search and select files within R-Studio Tech based on cross-referencing file contents with a pattern entered by the user? It kind of sucks that R-Studio already shows a file's condition by cross-referencing the DeepSpar sector map on the clone, yet after a few years of my requesting, still haven't provided any method for using that information to filter the files.

_________________
Luke
Recovery Force Data Recovery


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 27th, 2018, 18:49 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
I tried to see if there was any other tool or even Linux command that could do this usefully for hex strings, and I came up short, so this could definitely be a useful tool. Don’t be surprised if I write one for Linux at some point in time, maybe a future pro feature of hddsuperclone ;)

But I would like to point out that if you have some sort of map file that can be converted to ddrescue format, then there is a way to use the pro version of HDDSuperClone along with a Linux licensed copy of R-Studio to know exactly which files have the bad sectors. It involves using the virtual disk feature to present the recovery as a virtual block device to the system, which requires the clone/image and a map file (called a progress log in hddsuperclone, which can be an imported ddrescue map file). You then use R-Studio to extract the files from the virtual disk, which will give I/O errors for the bad files (according to the map) so they can be sorted by R-Studio. But that does require converting whatever map file format the imager created to a ddrescue one. And I don’t plan on adding any other import format into hddsuperclone at this time, so someone would have to either write a converter, or convince me that there would actually be enough demand for me to do it. I just wanted to point out that it could be possible, although not as easy as one would like.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 27th, 2018, 19:01 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15440
Location: Australia
maximus wrote:
I tried to see if there was any other tool or even Linux command that could do this usefully for hex strings, and I came up short, ...

If you were to search for text strings, would grep be suitable? :?

maximus wrote:
I don’t plan on adding any other import format into hddsuperclone at this time, so someone would have to either write a converter, or convince me that there would actually be enough demand for me to do it.

I've written some simple code to extract the bad sectors from a DFL DE map and convert them to ddrescue format. Would the following map be sufficient for HDDSuperClone to process, or do I need to include the ranges for the good sectors?

Code:
# Tool:  DFDE
# Model:  WDC WD20EZRX-00D8PB0                   
# Serial:  WD-WCC4M5KHJNKA JNKA
# Firmware:  80.00A80
# Total LBAs:  3906963633 / 0xE8DF88B1
# Logical / Physical Sector Size:  512 / 4096 bytes

#      pos        size  status

0x2000  0x200  -
0x10D2E000  0x200  -
0x10D2EA00  0x200  -
0x10EBF000  0x200  -
0x10EBF200  0x200  -
0x10EBF400  0x200  -
0x10EBF600  0x200  -
0x10EBF800  0x200  -
0x10EBFA00  0x200  -
0x10EBFC00  0x200  -
0x10EBFE00  0x200  -
0x121C4400  0x200  -
0x12355A00  0x200  -
0x1280AE00  0x200  -

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 27th, 2018, 19:36 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
fzabkar wrote:
If you were to search for text strings, would grep be suitable? :?

Sort of, but grep would be looking for that string anywhere in the file. In the event of a bad sector, the pattern would be in 512 byte blocks (typical logical sector size), so you would really only need to check the first part of every 512 byte offset for the pattern, and then if it matches, optionally make sure the whole 512 byte block continues to match the pattern. Only checking on 512 byte boundaries would speed it up. Grep can't do that.

fzabkar wrote:
I've written some simple code to extract the bad sectors from a DFL DE map and convert them to ddrescue format. Would the following map be sufficient for HDDSuperClone to process, or do I need to include the ranges for the good sectors?

Actually, the good ranges are more important than the bad in this case. The virtual driver will present any area that was not successfully read as an IO error. So the answer is yes, the good ranges would need to be included.

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 28th, 2018, 14:17 
Offline
User avatar

Joined: April 22nd, 2015, 20:32
Posts: 413
Location: Portugal
Guys sorry for all the waiting but I've been debbuging some nasty bugs with some types of files.

I hope eveything will be ready in a weeks time.

_________________
BTC Wallet - 3AoQPTBsz9PbfoanCx44Lw76Y2TwtKa1x5
Instagram https://www.instagram.com/datarecovery_morde.pt/


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 28th, 2018, 16:09 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15440
Location: Australia
maximus wrote:
... there is a way to use the pro version of HDDSuperClone along with a Linux licensed copy of R-Studio to know exactly which files have the bad sectors. It involves using the virtual disk feature to present the recovery as a virtual block device to the system ... You then use R-Studio to extract the files from the virtual disk ...

Why do you need R-Studio for this? Why not just use "cat" to recursively copy each virtual file to dev/nul and record the results in a log file?

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Corrupted File Finder
PostPosted: November 28th, 2018, 18:22 
Offline

Joined: January 29th, 2012, 1:43
Posts: 982
Location: United States
fzabkar wrote:
maximus wrote:
... there is a way to use the pro version of HDDSuperClone along with a Linux licensed copy of R-Studio to know exactly which files have the bad sectors. It involves using the virtual disk feature to present the recovery as a virtual block device to the system ... You then use R-Studio to extract the files from the virtual disk ...

Why do you need R-Studio for this? Why not just use "cat" to recursively copy each virtual file to dev/nul and record the results in a log file?

That sounds good in theory, but what if there are errors in the file table? The OS can't handle that very well at all, and may not even list whole folders that R-Studio can find with ease. Besides, this is really meant for the actual file extraction from the image/clone. On a side note, DMDE does also work okay for this, but I found it clunky with too many pop-ups that I did not know how to answer when it encountered errors. I also tried TestDisk and it failed horribly (kept crashing). UFS Explorer wanted to scan the whole image with no option to do otherwise, so I don't consider it useful for file extraction through hddsuperclone.

If the issue is not having a good pattern to search for, but you can convert a map file to ddrescue to use with hddsuperclone, then another possibility would be to use the Fill Mark option. You would need to be sure the map conversion was correct (or you would overwrite good data), but the idea is that you image/clone the drive, convert the map to ddrescue, import that into hddsuperclone, then run the Fill Mark on the image/clone. That will make a very definite pattern you could find in the files after they are extracted with whatever tool you use. And that can be done with the free version of hddsuperclone, assuming you can convert the map file. And I guess you could even use ddrescue as it has a fill feature. Of course you would then need the nice tool this thread is about to search for the pattern :)

_________________
http://www.hddsuperclone.com
Home of HDDSuperClone


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 44 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 15 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group