All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 19 posts ] 
Author Message
 Post subject: Raw recovery from a physical dump
PostPosted: May 5th, 2017, 3:49 
Offline

Joined: May 5th, 2017, 3:36
Posts: 3
Location: Netherlands
Hi guys

I have been researching couple of days and I have a few questions to the community. First I tell some short background.

I received a physical dump of a USB memory stick (DT100G3 8GB with PS2251-07 controller). At least that I am aware, the controller is what I just wrote.

The problem is of course that the blocks/pages are in incorrect order and I need to rearrange them in order to read the underlying FAT file system.

So the questions:

1) How do I determine the page and block size from the hex dump?
2) Is there a standard for the spare area? I know that the spare area stores the physical to logical block address translation.
3) Are the blocks OR the pages in the wrong order when a physical image is read? Is this a question about granularity?
4) Is there any free software to rearrange the FAT structure? I have tried a lot of different software.

I am happy to give more information related to the reconstruction of the FAT file system in question.

Thanks!


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 5th, 2017, 3:53 
Offline

Joined: November 29th, 2006, 10:08
Posts: 7843
Location: UK
You'll need some specialist s/w like this....

http://www.flash-extractor.com/manual/

_________________
PC Image Data Recovery
http://www.pcimage.co.uk

New!! HDD-PCB.COM for all your PCB and donor HDD requirements!


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 5th, 2017, 5:22 
Offline

Joined: May 5th, 2017, 3:36
Posts: 3
Location: Netherlands
I see that is commercial software. I need free software and preferably for Linux.

The most important question is that how do I determine the page and block size with hex editor? Is there some magic numbers from which I recognize the spare area like 55 AA in the MBR?

I have found a spare area format for 512 byte page size + 16 byte spare area but as I'm not sure which page size this dump resembles I can't move forward.

In case there are no free software available to do this simple job I will write my own script for it. It needs only to rearrange the blocks/pages.

Thank you for your help.


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 5th, 2017, 7:47 
Offline

Joined: March 19th, 2015, 15:01
Posts: 1387
Location: isreal
neitherNandNorXor wrote:
In case there are no free software available to do this simple job I will write my own script for it.

i dont say its not possible
but i'm really interested to know when you finished and got recovered the data


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 5th, 2017, 9:55 
Offline

Joined: October 24th, 2009, 15:22
Posts: 872
Location: Poland
neitherNandNorXor wrote:
In case there are no free software available to do this simple job I will write my own script for it. It needs only to rearrange the blocks/pages.


I do not know free software for it. But if you write own script/software share it for free :)

PS. Outsource this case will be much cheaper, time saving etc.

_________________
Flash Killer - everyday new resources (pinout, XOR, ECC,config) for flash devices


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 5th, 2017, 10:46 
Offline
User avatar

Joined: December 4th, 2012, 1:35
Posts: 3844
Location: Adelaide, Australia
PS .. do you know about XOR, also block size may change. start with 0x200000, do some operation, now relevant block size must be seen as 0x400000 for next step. do you know how to recognise different operations on data, how to interpret SA etc? simply writing a script is a massive simplification IMHO.


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 5th, 2017, 14:00 
Offline
User avatar

Joined: August 15th, 2006, 3:01
Posts: 3464
Location: CDRLabs @ Chandigarh [ India ]
neitherNandNorXor wrote:
Hi guys

I have been researching couple of days and I have a few questions to the community. First I tell some short background.

I received a physical dump of a USB memory stick (DT100G3 8GB with PS2251-07 controller). At least that I am aware, the controller is what I just wrote.

The problem is of course that the blocks/pages are in incorrect order and I need to rearrange them in order to read the underlying FAT file system.

So the questions:

1) How do I determine the page and block size from the hex dump?
2) Is there a standard for the spare area? I know that the spare area stores the physical to logical block address translation.
3) Are the blocks OR the pages in the wrong order when a physical image is read? Is this a question about granularity?
4) Is there any free software to rearrange the FAT structure? I have tried a lot of different software.

I am happy to give more information related to the reconstruction of the FAT file system in question.

Thanks!


Well,
it seems easy but its not ,it takes time experience and then you can handle it boss .I agree when you write your own script/tool share it free ,i hope you are getting the message loud and clear

_________________
Regards
Amarbir S Dhillon , Chandigarh Data Recovery Labs [India]
Logical,Semi Physical And Physical Data Recovery
Website-> http://www.chandigarhdatarecovery.com


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 5th, 2017, 14:14 
Offline

Joined: November 29th, 2006, 10:08
Posts: 7843
Location: UK
neitherNandNorXor wrote:
I see that is commercial software. I need free software and preferably for Linux.

The most important question is that how do I determine the page and block size with hex editor? Is there some magic numbers from which I recognize the spare area like 55 AA in the MBR?

I have found a spare area format for 512 byte page size + 16 byte spare area but as I'm not sure which page size this dump resembles I can't move forward.

In case there are no free software available to do this simple job I will write my own script for it. It needs only to rearrange the blocks/pages.

Thank you for your help.


With respect, I think you're underestimating how difficult this is.

In the "olden days" with real small capacity devices it may well have been as simple as re-arranging the blocks.

Now you have to consider (amongst other things)....

ECC
XOR
Block cuts
Block rotates
Block pair mixes
Sector updates
Etc......

_________________
PC Image Data Recovery
http://www.pcimage.co.uk

New!! HDD-PCB.COM for all your PCB and donor HDD requirements!


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 5th, 2017, 15:57 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
I would try to obtain a similar donor, write test data to each LBA (eg "LBA 0" to LBA 0, "LBA 1" to LBA 1, etc), and then examine the raw dump. ISTR that some of the tool vendors supply free utilities to do this.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 6th, 2017, 7:50 
Offline
User avatar

Joined: December 4th, 2012, 1:35
Posts: 3844
Location: Adelaide, Australia
You will also need to know something about the chip. What protocol is it and was it read with this in mind? What sizes are the pages and blocks and what layout are the pages in regard to the way they are chopped up into data and SA?

Also it is imperative to get as many details out as possible, and accurate. is the controller PS2251-07-V or PS2251-07-6 or other.


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 8th, 2017, 2:29 
Offline

Joined: May 5th, 2017, 3:36
Posts: 3
Location: Netherlands
Thanks for the replies guys.

As this is a single case I am not aware of all the general attributes of physical nand flash images.

This case turned out to be simple. There was a 16 byte area between every page which included the logical sector number of the underlying FAT file system.

So it was easy to write a python script which splits the image in parts, size of the page sizes and assembles them back together in the right order.

Thank you for your replies.


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 8th, 2017, 2:56 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
Nice work! Did you manage to work out the ECC algorithm? Are we to understand that there was no XOR, etc? BTW, I did something similar with a DiskOnChip in an earlier thread, but that was a very simple device. I'm surprised that your flash drive is essentially no more complicated than the DOC was.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 8th, 2017, 7:37 
Offline
User avatar

Joined: December 4th, 2012, 1:35
Posts: 3844
Location: Adelaide, Australia
well done. It is pretty unusual for a PS2251-07

Can you say what the full controller number was? also interested to know page size. XOR is almost certainly used, but common. Or maybe dump already had some work done on it?

many times pages will be numbered in SA and be consecutive in the block. pretty unusual for the whole image to be consecutive blocks for this controller. Also ECC is usually fairly important for these. But then again never surprised by flash. I wonder if it was a genuine Phison?


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 10th, 2017, 4:45 
Offline
User avatar

Joined: August 15th, 2006, 3:01
Posts: 3464
Location: CDRLabs @ Chandigarh [ India ]
HaQue wrote:
well done. It is pretty unusual for a PS2251-07

Can you say what the full controller number was? also interested to know page size. XOR is almost certainly used, but common. Or maybe dump already had some work done on it?

many times pages will be numbered in SA and be consecutive in the block. pretty unusual for the whole image to be consecutive blocks for this controller. Also ECC is usually fairly important for these. But then again never surprised by flash. I wonder if it was a genuine Phison?


Well,
i do not agree he has been able to crack it so easily ,This is not that simple a controller to work .For Me This Case And His Comments Carry No Meaning i am considering it to be a bagful of lies .This is The Page Of The controller -> http://www.pc-3000flash.com/solbase/tas ... 1&lang=eng

_________________
Regards
Amarbir S Dhillon , Chandigarh Data Recovery Labs [India]
Logical,Semi Physical And Physical Data Recovery
Website-> http://www.chandigarhdatarecovery.com


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 10th, 2017, 6:18 
Offline
User avatar

Joined: December 4th, 2012, 1:35
Posts: 3844
Location: Adelaide, Australia
Well Amarbir, while I can understand your view, I wouldn't go that far. Often see variations in flash controller/nand combo's that make no sense at all.

I just find the combination of absence of mentioning ECC and XOR, with sector numbers instead of page/block numbers strange.

The "space" between page numbers is usually padding, or SA itself including ECC.

Unless the dump was from something like an embedded system, it just isn't like anything I have seen before out of 20+ recoveries of same controller. Sometimes this controller can be a real PITA as well.


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 10th, 2017, 8:27 
Offline
User avatar

Joined: August 15th, 2006, 3:01
Posts: 3464
Location: CDRLabs @ Chandigarh [ India ]
HaQue wrote:
Well Amarbir, while I can understand your view, I wouldn't go that far. Often see variations in flash controller/nand combo's that make no sense at all.

I just find the combination of absence of mentioning ECC and XOR, with sector numbers instead of page/block numbers strange.

The "space" between page numbers is usually padding, or SA itself including ECC.

Unless the dump was from something like an embedded system, it just isn't like anything I have seen before out of 20+ recoveries of same controller. Sometimes this controller can be a real PITA as well.



Well,
I like to be on the knifes edge .This controller has many complex steps to follow,then only it gives anything out

_________________
Regards
Amarbir S Dhillon , Chandigarh Data Recovery Labs [India]
Logical,Semi Physical And Physical Data Recovery
Website-> http://www.chandigarhdatarecovery.com


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 10th, 2017, 10:15 
Offline
User avatar

Joined: December 4th, 2012, 1:35
Posts: 3844
Location: Adelaide, Australia
I thought it might be prudent to answer some of the questions properly, Instead of a quick scan and poo-ppooing the OP with negativity..even if a little late.


1) How do I determine the page and block size from the hex dump?

Usually you will know the page and block size from the config you used to read the chip, but this is what I call the starting page and block size. Generally you need to do operations on the dump and this changes the size reference.
example, if you need to pair the blocks, you might assume a block is 0x200000 and a page 0x2000. You get first page from first block, then first page from second block and keep going to make one block of 0x400000. If you need to do operations still, page size now is probably 0x4000, but could be different.

if you only have the dump, you start looking at the data and not where a set of bytes look the same/similar at certain size points. Rusoluts VNR is excellent for this with the bitmap view. otherwise if you can find a hex viewer that allows you to stretch out rows of bytes to as many as 9216, 17664, then try common page sizes and you will see distinct vertical rows of bytes the same, this is the SA between the data.


2) Is there a standard for the spare area? I know that the spare area stores the physical to logical block address translation.
No standard, different firmware versions of same controller, different controller models/manufacturers mix up data/spare area like crazy. It is almost as if they think if they made one the same as another they would get fired!

3) Are the blocks OR the pages in the wrong order when a physical image is read? Is this a question about granularity?
yes no maybe. some are a complete list consecutive (early ones and fake ones with little or no concern to wear levelling) remember, the controller does not "read" an image. WE need to construct the image, reverse engineering what the controller does. But the controller only throws data around the nand chip based on its algorithms.

4) Is there any free software to rearrange the FAT structure? I have tried a lot of different software.
No, remember the FAT structure isnt un-arranged, it was never arranged to begin with. There are thousands of different (albeit some slightly) algo's in play. I only know of 5 softwares in existence. 1st (SD) is a stinking pile of crap, where if bytes were apples, they would rotten. 2nd (nand team or similar?)is gone, I think Devs moved on to current other software.
3 (VNR), 4 (SC)& 5 (ACE Labs)are current and varying degrees of maturity and support for devices. any other software such as hex editors, file manipulators, disk editors and forensic tools are just not designed to play with the dump data.
you might get somewhere but it will be a hard slog.



The most important question is that how do I determine the page and block size with hex editor?

answered above (1)

Is there some magic numbers from which I recognize the spare area like 55 AA in the MBR?
yes and no. usually you will see at the same place in each page like or same hex code
example, this is fro a controller that doesnt use block numbers, and SA is at same place in every page. This is the SA for first page of every block:

00 b8 d3 07
00 b8 d3 07
00 b8 d3 07
00 b8 d3 07
00 b8 d3 07
00 b8 d3 07
00 b8 d3 07
00 b8 d3 07
..etc

one with block and page numbers(simplified) might look like:

FF FF 00 00 FF 00
FF FF 00 01 FF 00
FF FF 00 02 FF 00
FF FF 00 03 FF 00
FF FF 00 04 FF 00
FF FF 00 05 FF 00
FF FF 00 06 FF 00

some SA's will contain bytes for block renumbering, sector updates, page numbers, bank numbers, unused or "test" bytes and bytes that a vendor may know what it is for but we dont.


I have found a spare area format for 512 byte page size + 16 byte spare area but as I'm not sure which page size this dump resembles I can't move forward.
What this is is actually referred to as a DATA+SA+DATA+SA format, not page. a page is made up of a whole group of these to a certain length (page size).


In case there are no free software available to do this simple job I will write my own script for it. It needs only to rearrange the blocks/pages.
maybe in this case, but generally it would need to do a LOT more.

On the surface you would think you could just write some scripts to put things back together, but you would find these would keep blowing out until you have essentially written VNR or another commercial tool! So it is just common sense to buy it from the get go

This is all very general, but in practice it is even more complex by an order of magnitude as you need to also take into account reading from the physical chip, errors in the dump, ECC, bad columns, XOR, encryption, etc etc


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 10th, 2017, 10:49 
Offline
User avatar

Joined: August 15th, 2006, 3:01
Posts: 3464
Location: CDRLabs @ Chandigarh [ India ]
Well,
Haque ,Thanks you went all the way to explain to us and the op .Your way is different but mine is different .I have no idea how simple python scripts can solve complex nand mixes .BTW you never talked about dcc and XOR which to is complex work .

_________________
Regards
Amarbir S Dhillon , Chandigarh Data Recovery Labs [India]
Logical,Semi Physical And Physical Data Recovery
Website-> http://www.chandigarhdatarecovery.com


Top
 Profile  
 
 Post subject: Re: Raw recovery from a physical dump
PostPosted: May 10th, 2017, 14:51 
Offline
User avatar

Joined: October 21st, 2014, 1:39
Posts: 131
Location: Ellijay, GA
HaQue wrote:
I thought it might be prudent to answer some of the questions properly, Instead of a quick scan and poo-ppooing the OP with negativity..even if a little late.


Thanks for taking the time to answer point by point. Members like you have made this forum worth visiting on a daily basis.

_________________
Blizzard Data Recovery


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ] 

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: Google [Bot] and 47 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group