Winbond W25N01GV Off chip ECC

[og0]

Hi there

I've been looking at an off chip read of a Winbond W25N01GV SPI NAND 128Mb chip for a few days. (yes my life is that exciting).

It's from an IP Camera, and I'm looking to extract an lzma file from the first partition around 0x9300 of the dump. This contains the proper bootloader I need to further research.

I've removed the OOB (the layout of which is weird below), but when trying to decompress the lzma file it is clearly corrupted, and fails after producing only a little valid data.

Comparison with a dump from the same camera with the same bootloader version shows 420 bit errors between them for the data area I am interested in.

I do not have access to either device - just the dumps.

I did write a program to navigate these 420 bits errors that allowed me to interactively try both options for each of the 420 bits the 2 dumps disagreed on, previewing the decompression results, and backtracking to earlier choices as needed. That's allowed me to get about 200 bits in with valid results but now I'm struggling to make further progress.

I did look at using the ECC data in the dump to correct the bits, but short of spending $1000s for VNR there don't seem to be many tools that understand either the OOB layout or the ECC scheme used.

Page: 2048 + 64 OOB

OOB layout:

0x00 - 0x40F user data
0x410 - 0x40D 14 bytes (assumed ECC1)
0x40E - 0x7FF user data
0x800 - 0x801 Bad block marker (0xFF 0xFF)
0x802 - 0x80F user data
0x810 - 0x81D 14 bytes (assumed ECC2)
0x81E - 0x83F 0xFFs (except 0x83A there are always 2 0x00s)

So 0x800 user data in all and 0x40 OOB

On a blank page
ECC1 is always 0A 3A E9 39 43 DE 09 AC 83 22 D0 E1 7F F3
ECC2 is always A5 BB 1D F0 B8 A2 03 70 69 78 36 69 84 B1

So I'm thinking BCH-8 with a sector size of 1024 each having it's own 14 byte ECC code. 2 sectors in each 2048 + 64 page.

Despite this, the bootlog shows:

From bootlog of the kernel:
[ 1.602149] SPI Nand(cs 0) ID: 0xef 0xaa 0x21
[ 1.606622] nand: device found, Manufacturer ID: 0xef, Chip ID: 0xaa
[ 1.612980] nand: Winbond W25N01GV
[ 1.616460] nand: 128MiB, SLC, page size: 2048
[ 1.620901] Nand(Auto): OOB:64B ECC:4bit/512

The last line of this seems to disagree with my guess above, but it that because I'm looking at a physical off chip dump rather than the logical OOB view the OS gets? Data sheets on manufacturer web just talk about the logical layout view not what a chip reader sees.

Can anyone recommend a sensible way forward? Especially tools where I just press a button and magic happens lol :p

I know going too deep into BCH will get very mathematically complex likely beyond my ability for what is just a fun project for me. I can try different schemes, rearrange the OOB areas of the dump in to 1 64 byte area, try different polys for example, but if I need to work at a university and smoke a pipe with a whiteboard in the background I might give up

Any help appreciated.

[DRUG]

Can anyone recommend a sensible way forward? Especially tools where I just press a button and magic happens lol :p

Despite this being a fetish of you imagination, you could try reading:

https://www.blackhat.com/docs/us-14/mat ... fit-WP.pdf
https://www.blackhat.com/docs/us-14/mat ... enefit.pdf
http://recon.cx/2014/slides/Reverse%20E ... enefit.pdf

This is a begining, where you can make and adapter for you case, and then you just have to mod the OOB page in the python software.

I've been there, i've done that, it will make you wish to bang your head against a wall many times, but good luck.

p.s: Look at me being retarded trying to read nand with that homemade programmer 5 years ago: https://forum.hddguru.com/viewtopic.php?t=31253

[og0]

Thanks for the reply and the links

I do know the position and length of the 2nd stage u-boot by looking at the 1st stage in IDA, and don't care about the filesystems later on in the dump so that simplifies thing a bit at least.

I have found nand-dump-tools which implements a BCH decoder which I'm looking at with interest even though it will need some changes to the nand structures it knows about (or simply programmatically change of the dump I feed it which is somewhat easier now I've written a program to do that already).

I do know there are proprietary tools that already seem to have understanding of the underlying OOB/ECC scheme of this chip, and thought I'd check here in case I'm missing more effective ways of proceeding, or someone with experience of this specific chip which seems to be used with increasing frequency.

Thanks again.

[HaQue]

Have you looked at FlashcatUSB? I am not sure if it needs to do the ECC when dumping, or you can just open the dump in the software.
https://www.embeddedcomputers.net/products/FlashcatUSB_Mach1/

[fzabkar]

I do know there are proprietary tools that already seem to have understanding of the underlying OOB/ECC scheme of this chip, ...

I have no real experience in this area, but my reading of the datasheet suggests that the chip defaults to using its own, internal, transparent ECC algorithm, with a standard spare area. Your application, however, appears to override this default configuration and instead implements its own "soft" algorithm. If I've understood this correctly, then I can't imagine that any tool would understand this strange, proprietary (application specific?) structure.

[og0]

Have you looked at FlashcatUSB? I am not sure if it needs to do the ECC when dumping, or you can just open the dump in the software.
https://www.embeddedcomputers.net/products/FlashcatUSB_Mach1/

Funnily enough I posted on their forum yesterday - I wondered the same thing.

But, in the last few hours I think I may have it. I think I've figured out enough to write a program that will use the ECC to correct the dump. But it's late here and time for bed

[lolkek]

Hi,

I'm struggling with the same ecc on micron nand chip. Our ECC are the same for the blank pages. Can you share the ecc algo?

[HaQue]

depending on your situation, $250 Commercial Licence appears to enable integrateded ECC for the Flashcat software
https://www.embeddedcomputers.net/software/

Code:

Software features enabled:
Multi-device programming support
Boundary-scan programming for NOR
Command-line / Console mode
Integrated ECC support for NAND
LABview plugin

[hayn]

I have a very similar dump, also from a NAND chip with 2048+64 byte pages, and definitely took some effort to figure it out. The ECC bytes for the zero page are as follows, ECC1 matches yours exactly, but ECC2 doesn't, and I can explain why.

ECC1: 0A3AE93943DE09AC8322D0E17FF3 (0A 3A E9 39 43 DE 09 AC 83 22 D0 E1 7F F3)
ECC2: 79E89434A2D6B8411195934B6B5B (79 E8 94 34 A2 D6 B8 41 11 95 93 4B 6B 5B)

It's not easy to figure out, since the implementation is entirely in hardware, and you don't know you are on the right track until you have the exact algorithm and the ECC matches.

Indeed, just looking at some pages full of non-zero data and pages full of zero data makes it relatively simple to figure out the layout. Mine was the same as yours, 0x800 bytes of data is split into [0x0, 0x410), [0x41E, 0x800), [0x802, 0x810) with ECC bytes at [0x410, 0x41E) and [0x810, 0x81E).

Like you note, this suggests two BCH(8304, 8192, 8) codes in GF(2^14), but there's one more hint about the encoding. Zero data bytes trivially produce zero ECC bytes which clearly isn't the case here, so something more must be going on.

The first clue is to realize that it's possible that erased pages (full of FF bytes) also contain valid ECC bytes. The simplest way to achieve that would be to perform the error correction on a bitwise negated version of the page. This would also explain why the zero data page has non-zero ECC bytes, so I always did this as the first step before trying anything else.

I tried to find ECC1 using the zero data page (negated, so all FF) by trying all primitive polynomials of degree 14 in GF(2)[x] to see if anything matches the negated ECC1 bytes. It's not clear how the codeword bits are mapped to the ECC bytes, so I matched on 4 possibilities (normal and reversed byte order, normal and reversed bit order in each byte). But nothing matches, so this seems like a dead end.

The second clue is that the ECC bytes start at 0x410, not 0x400 for some reason. I realized it's possible that ECC1 is actually BCH(8432, 8320, 8) for the first 0x410 data bytes, and ECC2 is BCH(8176, 8064, 8) for the remaining 0x3F0 bytes. And amazingly, this finds a solution, namely x^14+x^10+x^6+x+1 (usually encoded as 0x4443) for ECC1, but nothing matches for ECC2, so we're not done yet.

The third clue is that there's still a lot of unused OOB bytes (all FF), so it's possible that ECC2 also uses 0x410 bytes. And when I added [0x81E, 0x83E) to the data to make it 0x820 bytes long, the ECC2 bytes are now correct using the same algorithm. If I set [0x83C, 0x83E) to 00 with the rest remaining FF, then it matches your ECC2 value. Maybe in your case there's some additional shuffling going if they really are at 0x83A, and not 0x83C.

Sorry for posting this so late, I already had this figured out a few years ago, hopefully it will still be of some help to someone out there.

TL;DR:

It's BCH(8432, 8320, 8) in GF(16384) on negated bytes. ECC1 operates on [0x0, 0x410) plus [0x410, 0x41E). ECC2 operates on [0x41E, 0x800), [0x802, 0x810), [0x81E, 0x83E) plus [0x810, 0x81E).

For an existing algorithm, you can use bch_init(14, 8, 0x4443, true) from https://gitlab.com/linux-kernel/linux/-/blob/master/lib/bch.c, and call bch_decode on the negated bytes in the ranges above to get the location of the errors.

[HaQue]

I do know there are proprietary tools that already seem to have understanding of the underlying OOB/ECC scheme of this chip, ...

I have no real experience in this area, but my reading of the datasheet suggests that the chip defaults to using its own, internal, transparent ECC algorithm, with a standard spare area. Your application, however, appears to override this default configuration and instead implements its own "soft" algorithm. If I've understood this correctly, then I can't imagine that any tool would understand this strange, proprietary (application specific?) structure.

On the contrary, Pro tools exist mainly for removing, reversing proprietary schemes of XOR, whitening, ECC, Byte, page, block shenanigans. The challenge is that most have to be added manually on a case by case basis and vendors don't usually give out datasheets that document it, so it all has to be reverse engineered such as @hayn's excellent detailed reply.

[hayn]

Thanks. I'll add that you don't have to extract bch.c from Linux yourself, there's a Python wrapper: https://github.com/jkent/python-bchlib. For example, this prints the value of ECC1:

Code:

import bchlib

bch = bchlib.BCH(8, 0x4443, swap_bits=True)
ecc = bch.encode(b'\xff' * 0x410)
print((bytes(b ^ 0xff for b in ecc)).hex())


Also, the internal on-chip ECC has to be disabled, it's not used at all in this case. In retrospect, this scheme does make a lot of sense. There's 0x840 page bytes (including OOB), and the ECC scheme can cover more than just the main area, so why stop at 0x400? And the first OOB byte serves as a bad block marker, that's why it's skipped.

But I had no prior experience with this, so it took me multiple weeks just to figure out the ECC algorithm. For example, I didn't realize there could be data in the OOB area, so I was stuck on that for a while because I dumped only the main area. But I was familiar with the mathematics of Galois fields, so I was able to understand the relevant Wikipedia page: https://en.wikipedia.org/wiki/BCH_code.

Then I got stuck trying to extract the UBI volumes, there clearly was some data, but I just couldn't get it to parse properly. It seemed hopeless again, and I didn't find time to return to it until recently. It turned out that some of the blocks in my dump were corrupt in a way that passed the ECC checks (duplicated pages). After I fixed that and redid the dump, the standard UBI tools worked and I was finally able extract the filesystem.