All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 29 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Forensic Image Hash
PostPosted: June 25th, 2018, 21:39 
Offline

Joined: January 29th, 2012, 1:43
Posts: 414
Location: United States
I have had some inquiries about adding an option to HDDSuperClone for forensic imaging with a hash. My question is about the hash. Do the sectors need to be read and hashed sequentially, or can they be hashed out of order and still come up with the same hash?

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 25th, 2018, 21:44 
Offline

Joined: October 16th, 2013, 13:21
Posts: 683
Location: Brazil
It depends on the method for hashing/calculating the crc you use.


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 25th, 2018, 21:49 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 9405
Location: Portugal
rogfanther wrote:
It depends on the method for hashing/calculating the crc you use.


It shouldn't "depend" ...

If you have a MD5 hash of a binary file, let's say a jpg, that MD5 hash for the exact same file should be exactly the same no matter what tool you do use to calculate the MD5 hash, has long as the "hash" is MD5 and not any other like SHA-1, CRC-32, etc ...

If you compute one MD5 (or any other sort of hash) for your "image" file the produced "hash" has to be the same for ANY tool that does compute the same sort of hash for the same file !!!

Try it out.

I would say you will have to do a sector by sector implementation but you can test it out !!!

Grab a very small HDD like 1 or 2 GB and image it. Now create one hash using your own method. Now get something on the net that can calculate "hash". Use that software and compare the result to the hash that you have generated.

They will have to be the same.

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 25th, 2018, 22:31 
Offline

Joined: January 29th, 2012, 1:43
Posts: 414
Location: United States
rogfanther wrote:
It depends on the method for hashing/calculating the crc you use.

How about the standard MD5 and SHA1?

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 25th, 2018, 22:34 
Offline

Joined: January 29th, 2012, 1:43
Posts: 414
Location: United States
Spildit wrote:
rogfanther wrote:
It depends on the method for hashing/calculating the crc you use.


It shouldn't "depend" ...

If you have a MD5 hash of a binary file, let's say a jpg, that MD5 hash for the exact same file should be exactly the same no matter what tool you do use to calculate the MD5 hash, has long as the "hash" is MD5 and not any other like SHA-1, CRC-32, etc ...

If you compute one MD5 (or any other sort of hash) for your "image" file the produced "hash" has to be the same for ANY tool that does compute the same sort of hash for the same file !!!

Try it out.

I would say you will have to do a sector by sector implementation but you can test it out !!!

Grab a very small HDD like 1 or 2 GB and image it. Now create one hash using your own method. Now get something on the net that can calculate "hash". Use that software and compare the result to the hash that you have generated.

They will have to be the same.

I think you missed my question. I am asking if it is possible to hash data out of order and get the same hash. It is obvious that any program should come up with the same hash (same hash type) for any given data file.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 26th, 2018, 0:03 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 9405
Location: Portugal
maximus wrote:
Spildit wrote:
rogfanther wrote:
It depends on the method for hashing/calculating the crc you use.


It shouldn't "depend" ...

If you have a MD5 hash of a binary file, let's say a jpg, that MD5 hash for the exact same file should be exactly the same no matter what tool you do use to calculate the MD5 hash, has long as the "hash" is MD5 and not any other like SHA-1, CRC-32, etc ...

If you compute one MD5 (or any other sort of hash) for your "image" file the produced "hash" has to be the same for ANY tool that does compute the same sort of hash for the same file !!!

Try it out.

I would say you will have to do a sector by sector implementation but you can test it out !!!

Grab a very small HDD like 1 or 2 GB and image it. Now create one hash using your own method. Now get something on the net that can calculate "hash". Use that software and compare the result to the hash that you have generated.

They will have to be the same.

I think you missed my question. I am asking if it is possible to hash data out of order and get the same hash. It is obvious that any program should come up with the same hash (same hash type) for any given data file.


I don't know. Should be easy to test.

Make your own output / hash a file by taking random sectors of it and compare the final hash with the hash for the exact same file taken sector by sector by a known working hashing tool.

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 26th, 2018, 9:02 
Offline

Joined: October 16th, 2013, 13:21
Posts: 683
Location: Brazil
maximus wrote:
rogfanther wrote:
It depends on the method for hashing/calculating the crc you use.

How about the standard MD5 and SHA1?



I do not believe these will result in the same codes for out-of-order groups of bytes.
After all, if you change the position of two 512-byte groups ( even if you change position of just two bytes ) , the files will be different, so by design their MD5, SHA, etc, should be different also.

I may be wrong here, but from what I have read and seen, for forensic work you would need a way of checksumming that is strong and easy to explain to people. Think about trying to explain to a room with a bunch of lawyers that only knows hard disks as "those things that go inside a computer and get destroyed by viruses" about out-of-order checksums, polinomials and the like.

I don´t know how Encase/FTK does it, but I would venture that a checksum is calculed after the image is taken, and appended to it, so that it travels with the file.


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 26th, 2018, 10:41 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 9405
Location: Portugal
rogfanther wrote:

I do not believe these will result in the same codes for out-of-order groups of bytes.
After all, if you change the position of two 512-byte groups ( even if you change position of just two bytes ) , the files will be different, so by design their MD5, SHA, etc, should be different also.


AGREE !!!

I think it's like a block chain and you do need to hash in order from the first sector to the last.

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 26th, 2018, 10:43 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 9405
Location: Portugal
rogfanther wrote:
I don´t know how Encase/FTK does it, but I would venture that a checksum is calculed after the image is taken, and appended to it, so that it travels with the file.


You can do a drive image with anything that you want (any software) and then calculate the hash of that image with any other available software.

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 26th, 2018, 17:21 
Offline

Joined: January 29th, 2012, 1:43
Posts: 414
Location: United States
+1 to calculating the hash after imaging, as that would be the ideal way. But the inquiries were about calculating the hash on the fly. I have never written any code to do any popular hashes, so I had to ask. But my instinct was that for any valuable hash, it needed to be done in perfect sequence.

I am not even sure if I will even add that feature, too many things about it that I don't like. I am just exploring it because it was asked by about three different people within a few months or so.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 26th, 2018, 19:16 
Offline

Joined: October 16th, 2013, 13:21
Posts: 683
Location: Brazil
My modest opinion is that the people who asked for it are misplacing the function with the tool. Do not want to offend anyone, more so if they are from this forum, but there are tools for that kind of thing, and they do not need to mix the functions of hddsuperclone in it.

About doing it on the fly. How would we define "on the fly" ? Hash any sector read ? Hash all of the sectors read until now ? And, before the cloning finishes, for what purpose ? What good would hashing/crc´ing something do in the middle of the cloning , and how would that be verifiable ?

The idea for CRCs/Hashes would for to assure the image has not been tampered with after being acquired. For that there are write blockers and Trusted labs/places that do the forensic imaging. After the imaging, a hash of the image can be generated and sent to the involved, so anyone that goes to work in that image later can verify if it has been changed or not. And even so, try to explain to people who do not understand, or worse, people that know how to put some tricky questions, how the process worked and if was possible to such and such evidence having been removed/inserted....


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 26th, 2018, 19:36 
Offline

Joined: January 29th, 2012, 1:43
Posts: 414
Location: United States
rogfanther wrote:
My modest opinion is that the people who asked for it are misplacing the function with the tool. Do not want to offend anyone, more so if they are from this forum, but there are tools for that kind of thing, and they do not need to mix the functions of hddsuperclone in it.

About doing it on the fly. How would we define "on the fly" ? Hash any sector read ? Hash all of the sectors read until now ? And, before the cloning finishes, for what purpose ? What good would hashing/crc´ing something do in the middle of the cloning , and how would that be verifiable ?

The idea for CRCs/Hashes would for to assure the image has not been tampered with after being acquired. For that there are write blockers and Trusted labs/places that do the forensic imaging. After the imaging, a hash of the image can be generated and sent to the involved, so anyone that goes to work in that image later can verify if it has been changed or not. And even so, try to explain to people who do not understand, or worse, people that know how to put some tricky questions, how the process worked and if was possible to such and such evidence having been removed/inserted....

That about sums up how I feel about it. I am not even sure why it was a requested feature. Your response helps me think that I just won't do it (which was my initial thought). It is not at all what the software is intended for. But I figured I would at least run the idea by others before condemning it.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 26th, 2018, 22:09 
Offline
User avatar

Joined: February 9th, 2009, 16:13
Posts: 2029
Location: Ontario, Canada
DeepSpar calculates on the fly, to the best of my knowledge. That is likely the reason for the question.

_________________
Luke
RAID Data Recovery


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 26th, 2018, 23:41 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 9405
Location: Portugal
lcoughey wrote:
DeepSpar calculates on the fly, to the best of my knowledge. That is likely the reason for the question.


What about if you do a reverse cloning or if you jump over some sectors / head and you decide to get that data back later on ?

If the "hash" have to be calculated in "chains" from the start of the binary data to the end as soon as you do add data on the middle or if you are imaging in reverse for each sector added the entire hash have to be re-calculated and that can take a long time if you do have a big binary file ....

Also it doesn't make any sense to have the "hash" unless you do finish your image and you do want to make sure that the image file is binary equal to other copies of that same file that you might produce.

It would make sense to calculate the hash at the end only. That would save a huge amount of processor power as you wouldn't be hashing the entire content of the binary image for each time you would add a single sector to it.

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 27th, 2018, 10:25 
Offline

Joined: January 29th, 2012, 1:43
Posts: 414
Location: United States
lcoughey wrote:
DeepSpar calculates on the fly, to the best of my knowledge. That is likely the reason for the question.

Does anyone have any real information on this (like if they have used it), other than what can be found on DeepSpar's website? http://www.deepspar.com/forensics-ds-disk-imager.html

My thought would be that to hash on the fly, it would need to be done in a linear motion with no going back. It would require its own algorithm that was not the best for data recovery. Read in larger chunks for speed, when an error is encountered read the chunk sector by sector. Bad sectors are marked as such, and are never tried again. If the errors were too extensive, the forensic mode would need to be aborted and then a data recovery algorithm used, and hashed when completed.

It's not like I couldn't do it, it is a matter if I want to, and if it would even be that much of a selling point. But then we come back to the fact that three separate people asked me about it as a pro feature they would like to see...

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 27th, 2018, 11:56 
Offline

Joined: October 16th, 2013, 13:21
Posts: 683
Location: Brazil
Well, maybe you could get this people to explain exactly why/how they imagine this feature to be implemented/work. Maybe they are ok with the very long extra time it will take, or the uncomplete cloning. Or maybe they are thinking about something very different from what we are all thinking...


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 27th, 2018, 13:47 
Offline

Joined: January 29th, 2012, 1:43
Posts: 414
Location: United States
rogfanther wrote:
Well, maybe you could get this people to explain exactly why/how they imagine this feature to be implemented/work. Maybe they are ok with the very long extra time it will take, or the uncomplete cloning. Or maybe they are thinking about something very different from what we are all thinking...

I think the point of hashing on the fly is that it would be much faster. I think the CPU usage for hashing is nothing compared to the time it takes to read the data, and doing it on the fly would not add much time to the cloning. Anyone can hash it afterwards using simple Linux commands, but the whole image must be read, and it takes time to read a large amount of data, considering you just took the time to process that data in the first place when imaging.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 27th, 2018, 13:51 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 9405
Location: Portugal
The problem is that you wouldn't be able to skip sectors or hash on reverse imaging, etc ...

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 27th, 2018, 14:20 
Offline

Joined: January 29th, 2012, 1:43
Posts: 414
Location: United States
Spildit wrote:
The problem is that you wouldn't be able to skip sectors or hash on reverse imaging, etc ...

Yes, but I think that when someone wants to forensically image a drive, they are most likely assuming it is in good shape. It would not be hard to handle a few bad sectors. But in the event that imaging starts to go bad, it would need to be stopped and switched to a recovery mode, which would no longer provide a running hash. The plus side would be that at least it would resumable and not have to start over. But it would need to be hashed after it is done.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Forensic Image Hash
PostPosted: June 27th, 2018, 14:45 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 9405
Location: Portugal
maximus wrote:
Spildit wrote:
The problem is that you wouldn't be able to skip sectors or hash on reverse imaging, etc ...

Yes, but I think that when someone wants to forensically image a drive, they are most likely assuming it is in good shape. It would not be hard to handle a few bad sectors. But in the event that imaging starts to go bad, it would need to be stopped and switched to a recovery mode, which would no longer provide a running hash. The plus side would be that at least it would resumable and not have to start over. But it would need to be hashed after it is done.


Agree.

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
paypal.me/Spildit - (PayPal Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 29 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group