Detecting corrupted & duplicate documents, photos mp3 files

[jag17]

The hard disk was overwritten with new operating system. After doing recovery I find lot of corrupted and duplicate documents, photos and mp3. The total number of files is will over 100,000. So it is impossible to manually open and test each and every file. Is there any software or technique to find all the corrupted and duplicate documents, photos and mp3 files and move it to a separate folder?.

Please help

Cheers
-Jag

[XanderSholtz]

The hard disk was overwritten with new operating system. After doing recovery I find lot of corrupted and duplicate documents, photos and mp3. The total number of files is will over 100,000. So it is impossible to manually open and test each and every file. Is there any software or technique to find all the corrupted and duplicate documents, photos and mp3 files and move it to a separate folder?.

Please help

Cheers
-Jag

I'm pretty sure this doesn't address your corruption issues (I have some ideas on this one, but have not made it that far yet) but which structure modification of the recovered data, I'm pretty sure this should help you on the issue of duplicates and sorting:

http://builtbackwards.com/projects/photorec-sorter/

If I understand correctly, documents are very hard to verify as Docs, XLS, etc, do not have a footer, just a header

Please let me know your opinions, I'm not trying to supply hollow responses on topics, I want to learn.

[jag17]

The hard disk was overwritten with new operating system. After doing recovery I find lot of corrupted and duplicate documents, photos and mp3. The total number of files is will over 100,000. So it is impossible to manually open and test each and every file. Is there any software or technique to find all the corrupted and duplicate documents, photos and mp3 files and move it to a separate folder?.

Please help

Cheers
-Jag

I'm pretty sure this doesn't address your corruption issues (I have some ideas on this one, but have not made it that far yet) but which structure modification of the recovered data, I'm pretty sure this should help you on the issue of duplicates and sorting:

http://builtbackwards.com/projects/photorec-sorter/

If I understand correctly, documents are very hard to verify as Docs, XLS, etc, do not have a footer, just a header

Please let me know your opinions, I'm not trying to supply hollow responses on topics, I want to learn.

Thank you for the link

But as you noted I am looking a magical software that can identify and move corrupted docs, xls, pdf, jpef, mp3 to a different folder. If a doc cannot be opened by ms word then it should be in a format that is not understood by MS word and a software will be able to identify that kind of corrupted files right? no?

Cheers
-Jag

[pepe]

I wrote a raw rec prog that restore dates of office docs and many others.
Basically i can be almost sure if it could not extract the date, the file is corrupted and can be distinguished this way. But it is not a perfect way I know.
Also if the date could be restored, there is a chance that the file is corrupted, but it is less than 5% according to my experience

pepe

[XanderSholtz]

I wrote a raw rec prog that restore dates of office docs and many others.
Basically i can be almost sure if it could not extract the date, the file is corrupted and can be distinguished this way. But it is not a perfect way I know.
Also if the date could be restored, there is a chance that the file is corrupted, but it is less than 5% according to my experience

pepe

You just turned on a light bulb in my head, Thanks Pepe!

[dick]

The hard disk was overwritten with new operating system.

But as you noted I am looking a magical software that can identify and move corrupted docs, xls, pdf, jpef, mp3 to a different folder.

Also if the date could be restored, there is a chance that the file is corrupted, but it is less than 5% according to my experience

There is no magical software! Once the data is destroyed its gone forever. Ok there might be a chance of repairing the odd file manually but in the end 5% is all you are gonna get. Put it down to experience!

[pepe]

No, the state of things is not so bad, there might be many files recoverable in RAW...
But the method cannot be automated i am afraid. Of course I mean sorting the files...

pepe

[jag17]

There is no magical software! Once the data is destroyed its gone forever. Ok there might be a chance of repairing the odd file manually but in the end 5% is all you are gonna get. Put it down to experience!

I am not looking software to repair or recover data. I need to only separate out the corrupted files from the good files. With 100,000 files manually sorting out the corrupted files becomes impossible. No recovery needed just detecting corrupted jpeg, doc, pdf, mp3 etc