Hi,
I have this older 1Gb flash drive in to recover. The Chip dumped fine



The controller is one Ive never seen, and cant seem to find any ifo - probably a fly by night rebranded one:


There is definately no XOR applied:


I tried creating an image with a few different parameters, GDB on the image gives pretty good filenames but files corrupt.. pictures are quite close though.

Anyone have an idea what controller it could have cloved or seen one of these?
cheers

Whenever I have needed to reverse engineer an unknown part (eg one that has had its markings ground off, or has been rebranded with an OEM in-house part number), I have determined its pinout by way of its connections to known components.

In your case you could trace each of the Hynix NAND flash pins back to the controller.

You could then determine the Ground and 3.3V VCC pins with reference to the 3-pin linear LDO regulator IC and the USB ground.

Locating the USB Data+ and Data- pins would be straighforward.

The 12MHz crystal appears to be associated with 2 capacitors and a resistor (~1Mohm ?) near the "-02" marking on the controller.

The date code on the controller (820) suggests that it was manufactured during the 20th week of 2008, so that might help narrow down the candidates.


HY27UG088G(5/D)M Series 8Gbit (1Gx8bit) NAND Flash, hynix:
http://www.hynix.com/datasheet/pdf/flash/HY27UG088G(5_D)B%20(Rev0.2).pdf

_________________
A backup a day keeps DR away.

Hi Franc,
The Controller itself would probably be impossible to track down, and likely no information exists to help figure out the mix of data. The only reason knowing a controller helps is to narrow down likely candidates for how it stores the data on the NAND, and a single controller number could have 30 different possible data mixes. There are common attributes that can be used and common operations done to the data that can sometimes be easily identified, but it is extremely time consuming if the actual data mix you have in your hand is not supported in the recovery software.

What you are looking at in the other thread is a very basic and probably old data mix.

what do I mean by data mix? It may pay to explain that. When the data is sent to the flash drive for any reason, formatting, saving files, deleting etc., the controller does operations to it to save the NAND wearing blocks out and to save ECC data. It does this depending on the firmware. It also does this knowing a certain structure is on the NAND.. block sizes, page sizes, where the ECC and other OOB Data(SA, Spare area, Service area or whatever you want to call it) is stored..
the way it is structured has many variations. The OOB data could be stored at the end of each page in a block, a few pages and then all the OOB data, start of pages, or even in the page itself. The number of ECC bytes can be different, the markers for the block different, inverted, rotated, operations on the Spare area can be different to the data, There are literally hundreds of ways to store data on a NAND.

Add to that the data could be XORed with some string of bytes, of varying sizes, the SA might or might not be. The blocks are not in order. The pages in a block may need to be shuffled around, or even need to be striped with other blocks, data inverted etc. Different numbers of chips and banks add more complexity.

Also, a computer expects sector sizes, whereas a Flash drive may not care so you would need to take the "sectors" and transform them to match.

Another thing the controller could do is to allow for shitty chips, such as cut a certain number of blocks from use from the start, the end of the image, the bank or after a certain number of blocks. You may have a 16GB chip in a 8GB Flash drive, with the data stored say the first half of 2 banks, or the last half, or the middle half.

Then you have errors in the drive such as a block that was copied but the old block exist with the same block number, or even a block with a few bit errors. It does not take a very big mistake in putting all this back together to have a corrupt or unusable result, sometimes a single correction can be make or break.

There are other factors as well, different things like a block being logically rotated by some bytes
Imagine going into a library and putting together a shredded book where you don't know how big the pages are, the pages in a chapter, or how many pages it has...

The other thread example, I am well on the way to figuring it out, but had many interruptions. It is an extremely simple flash drive, no XOR.

Take this as part of the answer for the other thread.

cheers, thanks for taking an interest in these issues.

Update:

I've recovered all files. 980 Files, 79 Folders. Only 26 corrupted files, so pretty happy with the result.

Never ID'd the controller, but in this case it is a non issue.

cheers

Pretty good outcome for a Hynix.

_________________
Hard Disk Drive, SSD, USB Drive and RAID Data Recovery Specialist in Massachusetts

I agree, They can be pretty crappy. This was a schoolteachers drive and a lot of the stuff was quite old.. she just needed the recent reports that are due. pretty funny to see about 50 MS Word Temp files on there.. I can only imagine the work process just waiting for disaster. I could have maybe got a few more with a bit of work, It was actually quite fun to work in core v5, but other jobs are waiting.

@HaQue, thanks for the explanation.

Am I right in assuming that not all the firmware is embedded within the flash controller and that at least some of it exists in NAND flash? I'm referring to code, not the FTL or bad block map. If so, then couldn't you examine the code and come up with a likely candidate by comparison with other firmware? If it's a generic flash drive, then ISTM that its manufacturer would be using generic OEM firmware.

_________________
A backup a day keeps DR away.

I don't think many store any code in the NAND Flash. There are some firmware version strings and controller number strings, but without context, it is hard to know what use they would be.
I know what you mean about comparing something known to other firmware, yes It is possible.

a drive I am working on now is a SM321Q CC and in the dump of the NAND was the string "SM325AB"

The Mass Production Utilities provided to the end user generally sets up the controller with everything,
The controller has flash on it, and apparently hard coded routines for things like ECC and encryption (such as AES). I am working on studying the controller more deeply to figure out XOR without having to get another working device to reverse engineer. Ive just got a Beaglebone Black to use as a USB Proxy to capture USB packets and look at them in wireshark, so we will see how that goes.

Flash controllers in most of cases store 2-10 or more blocks with FW code in NAND. It depends on device/controller vendor.
Here's as example screenshots of bloсks with FW of Sigmatel STMP3770 controller which used in DVR/MP3 players (was forensics job).

If you take a look to vendor's FW updater for this controller you'll find a same data (FW code) in it.

Every used block in NAND array has it's own marker of block header besides Logical block number and Logical page number markers (not all controllers have it all). Normally controllers mark all blocks with specific header:
- main blocks
- replacement/log blocks
- bad blocks
- TT blocks
- FW blocks
- obsolete blocks

Each brand has own format for every controller family. Usually it's ricky but possible to understand all bock headers ans define which belong to FW blocks.
E.g. Sigmatel assigns same LBN 5453h to all FW blocks and header FFh.
TT blocks marked with header 4Dh and LBN 424C.


This is typical for "CC" version of this controller.

The STMP3770 is a multimedia SoC that is designed to boot from several sources, including NAND flash, I2C EEPROM, and MMC/SD. Surely that makes it a lot different to a USB flash controller?

http://www.rockbox.org/wiki/pub/Main/Da ... s-1-04.pdf

BTW, I notice that 5453h and 424C correspond to ASCII "TS" and "BL", or "ST" and "LB". Perhaps "ST" = SigmaTel?

_________________
A backup a day keeps DR away.

The controller core is probably not all that much different to a USB controller in the core functions but implementation would be another thing.Sigmatel probably bought much of the controller IP to use along with design guides and tools.

USB controllers also mark block headers, in this way, at least for things like Bad Blocks, Update Blocks and block numbers and markers. Are you able to say what the firmware that is stored on the NAND is? IE the function of it? I would love it if it was the XOR string used to XOR pages or blocks.

BTW, what is the HEX editor you are using? Ive been looking for one that a user can define arbitrary blocks of bytes in that way. Winhex probably does it, but I don't particularly like it, and haven't used it a great deal to know.

HaQue, it is screen from new recovery tool.

_________________
Odzyskiwanie danych

Honestly I don't do FW disassembling, so cannot tell you what they store in Nand what in controller. But it would be nice to extract scrambling polynom from FW
However Xor can be extracted either from patterned flash or in many cases from user data, through visual analysis if Bitmap.
This is XOR of Phison PS2251, that looks like diagonal pattern (xor key equal to one page, and every page program operation it's shifted for 8 bits withing one blocks, so-called cyclic scrambler):



You can find some funny stuff in xor key, like Pacmans, ducks, rabbits, sometimes even human faces
This visual signature can help you to identify Noname controllers, e.g. if you found block with Xor pattern and see the duck it's PS2251.

_________________
VISUAL NAND RECONSTRUCTOR. A big revolution in chip-off data recovery

Excellent explanation Sasha The best and simple.

_________________
Odzyskiwanie danych

Very interesting. I played with Bitmap visualisation for a few other reverse engineering tasks. I did load a firmware into one but I cant remember how useful it was, I was concentrating on finding 8051 code to disassemble. I will definitely have to revisit the visualisation tools I was looking at a while ago, Thanks Sasha!

The purpose of the XOR is for what do you think Harder to reverse engineer their wear Levelling algo's? Makes the WL Algos more effective? I tend to doubt it is for security, being only XOR.

The other area I am hoping to start putting more time into is extracting from controllers themselves. It is a good model to supply customers of controllers vendors with a simple UI to configure them and not have to give out a whole set of data sheets such as you would get with an AVR or ARM based SoC. makes it hard to get leaked Docs when there aren't any

It is a good idea to look at things like STMP3770 where you can get docs, as the general principles can be applied, even if not totally the same.

It is how the SD Card hacking, or the NAND bad block research(I forget specifics - I read too much) was moved along. They noticed a cell phone(IIRC) contained a NAND firmware update to fix some block bug, reversed that and was able to understand a lot about the controller.

At this point I have no idea if I am on topic but it is great to see some new and exciting work, the level at how far along it is very surprising and impressive!

The more you learn, the more you discover how little you know!

I dug out an old MC6800 Emulator code I was trying to write while at Uni, thinking to write a 8051 emulator. Purpose to try and run parts of the controller firmware - well that showed me I had around 1% resources/knowledge to do that and just needed to know pretty much everything else to even start.

It is Saturday, and family wants to "get out and do something" but all I want to do is play around with my Beaglebone Black & USBProxy and try to intercept some firmware configs using MP Tools

I believe controller vendors don't care much about protecting their technology from DR industry (at least not with scrambling), it's all for data protection.
The latest TLC chips are very sensitive to data patterns (thank to mother nature and parasitic capacitance effect), so they randomize all incoming data with scrambling (XOR).

_________________
VISUAL NAND RECONSTRUCTOR. A big revolution in chip-off data recovery

When I was at Uni, the 6800 was state-of-the-art. :-)

_________________
A backup a day keeps DR away.