All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 19 posts ] 
Author Message
 Post subject: Barefoot logical to virtual mapping details (M225)
PostPosted: June 1st, 2017, 16:59 
Offline

Joined: June 1st, 2017, 15:09
Posts: 9
Location: London
Hi

I have a Crucial CT256M225 which uses the barefoot IDX110M00-LC together with 32 K9HCG08U1M-PCB0 NANDs. I had a power failure and it died, this is a couple of years ago. I put it down because I was too busy and have recently come back to it to try to get 3000 photos of my daughters first 3 years off it. I had this returned from a recovery company I specifically checked would use PC3000 SSD on it. They claim this was unsuccessful. So being a programmer with time on hands, I decided to see how far I could get. Having ARM microcontroller hardware and software experience I was prepared to do a chip off myself (build some hardware to facilitate it) and then send the dump off to someone, but getting the virtual image proved easier than that.

So the drive does not show in normal mode but it has a switch on it for 'factory mode'. In this it reports at yatapdong barefoot etc, as expected for this chip, so the real firmware can't load and its booting to rom firmware. It happens that there is an open source project for this controller, Jasmine openSSD, which has some limited documentation and several useful bits of code as part of it. There are a few firmwares which are of no use to me because I don't want to overwrite the firmware of the drive obviously, but there is also an 'installer' program source which passes commands over SATA to the drive and can read, write etc data. And the firmwares can give me clues about the drive too.

On running the installer program it sends its config (that you set in the source) to the controller, which tells it the NAND configuration. I've hacked it to then dump the virtual image of the drive which it seems to do. I have 256GB in 16kb pages of data which contain working gifs, and text, contiguous 16kb chunks, which leads me to think that I have the block structure right (NAND page size is 4kb +128 spare, but in 2 plane 2 chip per die mode this seems to be translated to 16kb pages).

The controller reports ECC status per page read and in one particular configuration I get success for every read. IE I believe that is the correct config because the ECC reports as no error. There are some all FF pages 128 rows long, which corresponds to the NAND block size. These are empty blocks I guess. I haven't bothered trying to get the actual bad block info by scanning the chips bad block bit, because since I'm not interested in writing to this SSD I don't think I need to.

There is also a forensics paper on the web from an author who probed this drive for recovery purposes. He found 8x4 byte XOR values which the controller apparently uses to scramble the data, and although I can find these values as a pattern (as described) in the firmware section of my image, I have not implemented the descrambling function myself. I don't seem to need to, the fact that I have recovered unscrambled pages makes me think that on this version of the firmware, scrambling was not implemented, or somehow its happening transparently to me by the controller ROM firmware.

So my problems are now these. Firstly, maybe not significant, I don't understand the use of the spare area. There are 128 bytes spare per 4k page on the NAND. I guess some of this is ECC, but I wonder if the rest might be storing backup LPNs. If so obviously I am home and dry, reassembling the logical image will be easy. I doubt this is the case though, but I wondered if anyone had any info on this. I don't seem to be able to read the spare area very well. If I try to read 4096*4 bytes per row, I get ECC success. If I try to read (4096+128)*4, ie page plus spare times the factor of 4 for 2 dies, 2 planes, I get ecc fail. The data seems OK still, but without ECC it can't be trusted (some diffs between data read this way and data read with ECC success differ, some don't). Perhaps I can access the spare area this way albeit without ecc, but whether its useful I don't know. It might just be ecc data here with the remaining unused bytes not used for anything else.

Secondly and the only important question really, has anybody any idea how to proceed from here to try to work out where any logical to virtual mapping info might be found? After reading about the methods likely to be used on a drive of this period (2009ish), I've taken a look in the last page in each block and I can see interesting repetition, which makes me think perhaps theres something there. I guess if the log-hybrid technique is used in this drive, the full data blocks might be easier to piece together than the ones still not merged. I might not get all the drive back but if I can get the older stuff back that will be fine.

I think I know why the drive won't come ready, of the 32 chips, 4 of them have first blocks all FF. This is the bad block record I believe. The actual blocks referred to seem fine, they are reading with no ECC errors, but the firmware probably chokes when it tries to load that list of bad blocks. I guess the bad block record blocks were erased or something, when the power failure happened. Its tempting to download the first block of each chip, 'correct' it, and write it back, but I don't want to risk my photos. Alternatively if I can get my hands on another of these exact drives, I can try to transfer the entire image, after correcting those back block list records, but so far I have not been able to find another of these drives.

So firstly has anyone any actual experience with this controller, who might have any clues to where I can find the logical to virtual mapping? And if not has anyone any idea from other drives which might be comparable what to try now?

Thanks, Pete


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 2nd, 2017, 4:15 
Offline
User avatar

Joined: December 4th, 2012, 1:35
Posts: 2895
Location: Adelaide, Australia
Hi, for info on the spare area, and much other useful info, go to Rusolut.com and read the explanations there. You will have a deeper understanding. http://rusolut.com/nand-recovery-case-samples/

I wouldnt write anything to the drive/chips. do everything in software/dumps.

The mix probably isnt too complex, but may not be as simple as you hope. There could be versions of pages to deal with. have a look at similar recoveries http://www.flash-extractor.com/library/SSD/IDX110M00/

you could try pcimage forum member, highly recommended, to perform recovery if he wants to take on the case, or help you out somehow.

looks like most of these have a join by byte step, though if you didnt read nand "normal" way, this might not apply.

done very well so far, thanks for detailing it!


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 2nd, 2017, 12:03 
Offline

Joined: June 1st, 2017, 15:09
Posts: 9
Location: London
Thanks that Rusolut site was very interesting. Taking a few points from there I did some more today and got surprising results I think. I seem to have blocks of contiguous text (like html pages) which are larger than 16k, much larger, and they 'work' ie they are not corrupt. So this makes me wonder if I have the basic lbn-vpn scheme figured out.

As I mentioned last time the last page in each block had a pattern. I wrote a bit of code to present it like they did in that tool on Rusolut and the patterns were clear, here is an example:

Code:
e7 03 7f 00 7f 00 00 00 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 7f 1e 81 2c 00 a3 74 70 00 a4 74 70 00 c5 0d 55 00 8d 04 d2 00 b5 ff ae 00 12 00 af 00 40 f9 7a 00 41 f9 7a 00 56 9e 5e 00 15 ec 2d 00 f3 59 0a 00 ec 49 5d 00 d3 81 88 00 eb 81 88 00 47 af 18 00 58 af 18 00 72 af 18 00 78 af 18 00 3b 97 96 00 43 97 96 00 64 97 96 00 73 97 96 00 94 97 96 00 a2 97 96 00 b6 97 96 00 da 97 96 00 96 79 6c 00 98 50 0b 00 ad 50 0b 00 b8 11 46 00 2d e8 48 00 5e 1c 29 00 00 7c 83 00 07 7c 83 00 b7 99 68 00 89 52 4b 00 b5 09 80 00 e8 de 48 00 f1 31 69 00 f2 31 69 00 09 32 69 00 0a 32 69 00 0b 32 69 00 0c 32 69 00 0d 32 69 00 0e 32 69 00 4a 32 1f 00 13 25 64 00 1b 25 64 00 33 25 64 00 5f 25 64 00 8f 25 64 00 97 25 64 00 b0 25 64 00 c7 25 64 00 20 b5 68 00 67 f2 3f 00 68 f2 3f 00 69 f2 3f 00 6a f2 3f 00 4d f6 4a 00 3d e4 77 00 7d e4 77 00 b7 e4 77 00 15 e5 77 00 8b e5 77 00 70 94 52 00 56 b7 51 00 66 b7 51 00 78 b7 51 00 43 d1 54 00 44 d1 54 00 45 d1 54 00 40 a3 5e 00 59 e6 2a 00 94 3a 2d 00 b0 3a 2d 00 d7 3a 2d 00 44 6f 0e 00 7b e9 3d 00 2d 3e 03 00 7d e9 3d 00 77 69 5c 00 08 b3 15 00 ff fe 05 00 ae a6 72 00 cb 55 50 00 0d 25 2c 00 30 3e 2b 00 8c 46 33 00 38 67 65 00 c5 11 dc 00 5e 14 1b 00 4e 16 12 00 86 16 12 00 97 16 12 00 cb 16 12 00 e5 16 12 00 91 34 7f 00 94 34 7f 00 95 34 7f 00 0f 5a 36 00 97 34 7f 00 98 34 7f 00 99 34 7f 00 9a 34 7f 00 16 fa b1 00 38 fa b1 00 4a fa b1 00 61 fa b1 00 6b fa b1 00 72 fa b1 00 bd e7 b4 00 92 fa b1 00 b4 fa b1 00 bc fa b1 00 e0 fa b1 00 e1 fa b1 00 e2 fa b1 00 e3 fa b1 00 e4 fa b1 00 e5 fa b1 00 e6 fa b1 00 e7 fa b1 00 e8 fa b1 00 e9 fa b1 00 f0 02 09 00 09 00 00 00 ff 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f8 98 43 00 33 99 43 00 51 99 43 00 ab 99 43 00 ce 99 43 00 0a 9a 43 00 65 9a 43 00 6c 9a 43 00 6e 9a 43 00 ff ff ff ff
e7 03 7f 00 7f 00 00 00 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 7f 66 4e 57 00 bf a9 47 00 c0 a9 47 00 c0 2b 06 00 4e 5e 58 00 76 30 06 00 8c 30 06 00 72 1e ba 00 38 5e b1 00 51 5e b1 00 5c 5e b1 00 9f 5e b1 00 b7 5e b1 00 eb 5e b1 00 24 5f b1 00 39 5f b1 00 5a 5f b1 00 69 5f b1 00 72 5f b1 00 83 5f b1 00 c2 5f b1 00 fa 5f b1 00 03 60 b1 00 04 60 b1 00 0b 60 b1 00 a4 9a 13 00 af 9a 13 00 80 91 83 00 c8 91 83 00 78 c2 54 00 99 c2 54 00 df 60 49 00 ae 80 83 00 af 80 83 00 8e 17 6d 00 c2 52 63 00 f5 52 63 00 12 10 ae 00 98 1a 84 00 8d 1e 71 00 8e 1e 71 00 ea 3f 07 00 b3 0c c8 00 5d 7b 78 00 cd 0c c8 00 af cf 6d 00 11 9b 19 00 a5 99 45 00 7a f8 16 00 7b f8 16 00 51 95 0d 00 63 95 0d 00 74 95 0d 00 7e 95 0d 00 a7 38 4f 00 e2 38 4f 00 fe 38 4f 00 e7 c3 06 00 55 3e 79 00 56 3e 79 00 b4 52 42 00 be f8 2f 00 bf f8 2f 00 c2 f8 2f 00 ae 2f 29 00 3b 71 43 00 c6 f8 2f 00 c7 f8 2f 00 ab d0 9f 00 d4 d0 9f 00 1e d1 9f 00 35 d1 9f 00 97 d1 9f 00 08 d2 9f 00 81 0f 56 00 85 0f 56 00 87 0f 56 00 88 0f 56 00 89 0f 56 00 8a 0f 56 00 8e 0f 56 00 90 0f 56 00 91 0f 56 00 92 0f 56 00 94 0f 56 00 9a 34 5c 00 9b 34 5c 00 a4 2c 63 00 11 c0 a2 00 37 c0 a2 00 55 c0 a2 00 7d c0 a2 00 85 c0 a2 00 a1 c0 a2 00 c2 c0 a2 00 e6 c0 a2 00 8d 40 4f 00 8e 40 4f 00 3a ba 26 00 41 ba 26 00 fe bf 24 00 1a 3c 84 00 3d 3c 84 00 de 5c 02 00 7b 57 81 00 7c 57 81 00 7e 57 81 00 7f 57 81 00 76 33 b5 00 cc 34 b5 00 32 35 b5 00 81 6f 6f 00 32 64 0f 00 89 d6 06 00 e8 ea 25 00 ec ea 25 00 4a 41 96 00 b4 b7 9f 00 b7 b7 9f 00 b8 b7 9f 00 b9 b7 9f 00 bc 01 de 00 f7 38 4a 00 a9 f8 1c 00 11 4d 21 00 97 19 0b 00 d2 97 56 00 5e 00 7d 00 7d 00 00 00 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 1f 0d de bc 00 46 e5 2e 00 4f de bc 00 93 de bc 00 a2 de bc 00 db e0 af 00 dc ff 97 00 a4 29 b5 00 18 39 00 00 2f 39 00 00
e7 03 7f 00 7f 00 00 00 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 7f f6 7f a8 00 07 80 a8 00 4a aa 18 00 51 aa 18 00 5b aa 18 00 ad aa 18 00 d7 aa 18 00 e2 aa 18 00 f6 30 89 00 13 7b 3f 00 14 7b 3f 00 8a 5a 43 00 93 fc 7d 00 b8 85 1d 00 cb ea 3d 00 26 62 ac 00 60 4a b1 00 77 4a b1 00 8e 4a b1 00 ac 4a b1 00 ae 4a b1 00 b7 4a b1 00 a1 4d b1 00 85 cb 64 00 9e cb 64 00 22 d8 03 00 3b d8 03 00 52 d8 03 00 6e d8 03 00 29 77 04 00 25 04 41 00 3d 04 41 00 c7 b9 3c 00 d6 b9 3c 00 2c 4f 1e 00 09 bb 23 00 36 bb 23 00 50 fc 88 00 53 fc 88 00 54 fc 88 00 55 fc 88 00 56 fc 88 00 64 fc 88 00 8a f6 1f 00 a8 f6 1f 00 70 40 0e 00 75 40 0e 00 c0 73 0a 00 09 74 0a 00 21 74 0a 00 ce 87 69 00 01 88 69 00 9b 0f 65 00 e3 0f 65 00 00 10 65 00 10 c7 04 00 52 c7 04 00 b2 73 1d 00 cf 73 1d 00 f8 73 1d 00 51 7b 1d 00 7d ad 03 00 c9 cd 62 00 da cd 62 00 ed b7 a5 00 ff b7 a5 00 53 b8 a5 00 6b b8 a5 00 76 b8 a5 00 7c b8 a5 00 af b8 a5 00 fb b8 a5 00 63 b9 a5 00 6b b9 a5 00 7c b9 a5 00 b2 b9 a5 00 ba b9 a5 00 d6 b9 a5 00 ee b9 a5 00 10 ba a5 00 18 ba a5 00 3b ba a5 00 58 ba a5 00 78 ba a5 00 80 ba a5 00 b1 ba a5 00 b4 03 b0 00 48 02 b0 00 69 ff af 00 bb 37 ba 00 ff 37 ba 00 bf dc 86 00 dc 5d 54 00 0b 67 27 00 9d 61 54 00 c4 61 54 00 ca 61 54 00 d1 dc 86 00 d6 dc 86 00 1b 5a 70 00 9a 47 9a 00 09 48 9a 00 6e 49 9a 00 1f f7 04 00 32 7e 67 00 7b 82 2f 00 46 fd b9 00 9c d5 56 00 82 cb 63 00 83 cb 63 00 ed f4 2d 00 04 f5 2d 00 c5 da 26 00 6b 9f 48 00 01 a7 3f 00 c6 77 0c 00 39 07 16 00 58 07 16 00 ab 56 69 00 6e 51 65 00 28 78 46 00 2a 78 46 00 66 8b 05 00 99 ee 0d 00 54 eb 40 00 50 69 35 00 51 69 35 00 0f 09 60 00 60 00 00 00 ff ff ff ff ff ff ff ff ff ff ff ff 00 00 00 00 11 87 99 00 90 1d ba 00 2a 99 66 00 77 2a 4c 00 16 e8 97 00 23 a0 9a 00 2b a0 9a 00 55 17 14 00 00 8f 50 00 61 10 37 00


Unfortunately I don't know how to apply a horizontal scroll bar to that code above so its not easy to see. There are 3 lines there. Each line is the last page in its virtual block.

The first 24 bytes have a pattern with little variation, then there are 128 * 4 byte numbers which seemed to be the right kind of value for lbn's, then more 'different' stuff. So I tried using the 128 * 4 byte numbers as LBNs. I created an array to map them etc, then wrote out a bit of the disk this way. Its still running, but I have a few GB copied, and in that file there are clearly chunks of data longer than the page size (16kb) which fit together correctly. To me this seems like its so unlikely to be from chance that these must be the LBNs.

So for instance at byte 25-26-27-28 I have '1e 81 2c 00' which is LPA 0x2c811e = 2916638 decimal. Next is LPA 'a3 74 70 00' = 0x0774a3 = 488611. So the first page of that block is LPA 2916638, the second page = LPA 488611, etc. And the last page, page 128, is the one I get the above data from (IE where I get the line e7 03 7f 00 7f 00 00 00 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 7f 1e 81 2c 00 a3 74 70 00 a4 74 70 00 c5 0d 55 00 8d 04 d2 00 etc).

I am absolutely astounded that getting this far was so easy, can't really believe it, although there is much duplication in the LBNs (about 800k duplicates in 16M page addresses). This must be from expired pages. How to work out which are the up to date pages will be next.

I think I will make a full image of the disk from this map I have already, since I need to get on with some other stuff today. Then try to run photorec on it to see if there is anything good in it. I'll try to work out the page expired / valid stuff soon though, as I doubt I will get that much from this currently with all those expired pages in there.

In each last page (IE the LPA record block) after the 128 bytes of LPA addresses, there is more stuff. I suspect this might have something more, perhaps even a version number for instance to allow me to work out which is the latest version of this page.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 4th, 2017, 13:57 
Offline
User avatar

Joined: August 15th, 2006, 3:01
Posts: 2294
Location: CDRLabs @ Chandigarh [ India ]
Hi,
The 128 bytes you refer to have ecc also for the initial 4096 you are reading and hence if you add that 128 to 4096 of course you will not get any ecc .Also the 128 byte spare area has other interesting info .But if this controller is supported PCImage will do a good job he is one of the oldest users of soft center :D

_________________
Regards
Amarbir S Dhillon
CDRLabs Data Recovery - [ Chandigarh ,India ]
My Data Recovery Experiments - > https://www.facebook.com/chandigarhdatarecovery


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 4th, 2017, 16:41 
Offline

Joined: June 1st, 2017, 15:09
Posts: 9
Location: London
Hmm so how do you read the 128 bytes spare area? If I try to set column to 4096 the controller ignores it and reads from the byte 0. Should you read the whole 4096+128 and ignore the ecc error perhaps, or perhaps I am using the wrong method to set column (but it works <4096).

Thanks, Pete


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 5th, 2017, 3:24 
Offline
User avatar

Joined: August 15th, 2006, 3:01
Posts: 2294
Location: CDRLabs @ Chandigarh [ India ]
peterpion wrote:
Hmm so how do you read the 128 bytes spare area? If I try to set column to 4096 the controller ignores it and reads from the byte 0. Should you read the whole 4096+128 and ignore the ecc error perhaps, or perhaps I am using the wrong method to set column (but it works <4096).

Thanks, Pete


Well,
If you do a chipoff recovery of this the nand is read completely ,including This 128 Bytes Service/spare Area ,You Are in London ,PCImage is also from your country

_________________
Regards
Amarbir S Dhillon
CDRLabs Data Recovery - [ Chandigarh ,India ]
My Data Recovery Experiments - > https://www.facebook.com/chandigarhdatarecovery


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 5th, 2017, 16:36 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 9607
Location: Australia
peterpion wrote:
Hmm so how do you read the 128 bytes spare area? If I try to set column to 4096 the controller ignores it and reads from the byte 0. Should you read the whole 4096+128 and ignore the ecc error perhaps, or perhaps I am using the wrong method to set column (but it works <4096).

What happens if you set the column to 4095? Does the address still wrap to 0?

ISTM that reading the data + spare area is akin to a READ LONG command, in which case it makes no sense for the output to be error-corrected. After all, you are requesting raw data, not error-corrected data. Of course the controller must have some [internal] way to correct any errors in the spare area which is obviously your current impasse.

In fact I'm not sure if anything would be gained by reading the spare area on its own. Can you be certain that any bit errors in the SA could be corrected without reference to the data area? I mean, is there an ECC for the data area and a separate ECC for the SA, or is the ECC computed over the entire page?

As for resolving the duplication in the LBNs, would anything be gained by examining the duplicate pages for one particular LBN and looking for patterns or differences in the SA?

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 5th, 2017, 20:17 
Offline
User avatar

Joined: December 4th, 2012, 1:35
Posts: 2895
Location: Adelaide, Australia
Quote:
I've hacked it to then dump the virtual image of the drive which it seems to do


Seems to me you are not reading the raw NAND, but the virtual image already. This implies that any XOR operations and ECC have been done.
These controllers have always used a join by byte first step, generally followed by pairing blocks with page sizes of these blocks 0x2000.. resulting in a page size of 0x4000. the ECC should have been done.

if you were reading raw nand, you should be seeing data with an obvious interleave such as take 2 dumps and in hex:

dump (chip1 plane1): RSLIGET
dump (chip2 plane2): EUTNTX

virtual Image : RESULTINGTEXT

I am really not 100% sure on your reading method, could you expand on it?

for chip off, we would:

1. read each plane, resulting in (if you say 32 chips 2 planes each) 64 dumps
2. for each pair of dumps, join by interleaving bytes from each dump resulting in 32 dumps
3. for each dump, pair the pages (of size 0x2000) of block 0 and 1, then of block 2 and 3... etc till end, resulting in 32 dumps of new page size of 0x4000
---I think you are here ----
4. stitch all dumps together one after the other in correct order.
5. figure out logical image from this virtual image

I don't know if any numbering is available to help with step 5.
I don't know if you have done everything in correct dump order, but step 2 at least must have been right.

The VNR software, flash extractor and PC3K does all this for you, or helps to do it with raw dumps.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 6th, 2017, 5:45 
Offline

Joined: June 1st, 2017, 15:09
Posts: 9
Location: London
Hi guys thanks for your posts.

Basically I am using a project called Jasmine which is part of something called OpenSSD. It just happens luckily that the makers of the controller released some source code for the chip, tutorial firmwares and an installer program which can upload firmware to the SSD. It was made for a different set of NANDs than I have but similar enough that I believe I have adjusted the parameters to suit, although I am missing stuff like how to read the bad block info from each spare area. This is a PITA but not seeming like a show stopper.

You can look at the source of 'installer.c' from OpenSSD jasmine on the web. It sets up the controller via a function called reset_fctrl, which is one place I have spent a lot of time at. Because the documentation is incomplete I don't quite know how to set the barefoot up for my nands, but I am good enough to get a dump (tiral and error with NAND config).

In this function you set info like nand page size, mlc / slc, planes, dies, block size, speed etc. This is passed over SATA to the controller which is running its inbuilt rom firmware. The controller then accepts commands to read blocks from the nand and pass them back over SATA. The way it reads the nands depends on how you set it up in reset_fctrl. So when you get it right, the interleaving and bank distribution is done by the controller, leaving you with large chunks of data. I have it now returning 16kb pages which seems right, since the nands are 4kb pages, 2 die, 2 plane. The SSD reports as having 32 banks, and looking through the code I am pretty convinced this is set up right now.

So then you have 'bank_read' function. Here you give it a bank/row address and number of bytes to read. After a read you can query the ecc status of the read. I can read 16k with successful ecc just about all the time apart from some blocks which are probably bad blocks (the bad block info at each banks block 0 is partially corrupt, and I can't scan the end of data block bad block markers because I don't have the documentation to adjust the code to suit my nands - so far).

So hence the join by byte, and that stuff (I'm not 100% up with the terminology), seems to be not needed. The XORing, to my surprise, seems not to be needed. I have code to set the XOR registers in the Jasmine source but its not implemented AFAIK. I also have the keys from another source. However, with html pages from the disk readable as things are, I feel its not required. Its possible that the ROM has implemented it, but I don't think so. This drive has original very old firmware on it and I suspect they never implemented the XORing on this firmware release (I got the drive 2009 IIRC).

When you use bank_read, you set certain parameters during each read. You can ask for ECC or not, set bank, column, set bytes to read, set 2 plane interleave etc. There is reasonable documentation for this operation.

So anyway I use bank_read and read 16k from each bank in sequence. This gives me what I believe is a virtual image. I see whole 16k pages which I can tell are written at once. Whether the original OS was writing 16kb or this is 2 8kb pages written sequentially I don't know and probably don't care, as thats for the file system to sort out.

Anyway that nearly brings me up to date, but for one thing. Now I have some logical page mapping info. The last 16kb page in each block of 128 pages in each bank holds data which I noticed a pattern in, and it looked a lot like LPAs. It looked a lot like 32 bit words and the first 6 words looked different to the next 128. So I tried using the 128 words which look like LPAs. I tried using the first word of the 128 list as the LPA of the first page in that block in that bank, word 2 as the LPA of the second page etc. It assembled to an image which has pages of html and other text longer than 16kb, and there are many of them. With a html page you can of course see if the pages do belong together and they do. Once I had seen several long pages reassembled together like this I believed that these 32 bit words were indeed the LPAs.

However the problem is as I said before, there are duplicate LPAs. Yesterday I did more coding to try to reduce the duplicates. I looked at the first 6 words of the mapping blocks (the ones I skipped, before the LPA words). They look like this:

Code:
  7f0018       7f ffffffff ffffffff ffffffff 7fffffff
  7f001f       7f ffffffff ffffffff ffffffff 7fffffff


Looking at the second word, 7f above, it looked interesting, so I examined what values were put in that place and how common they were. On the left here is the value from word 2, on the right is how many times I found it scanning several GB of the disk:

Code:
       0 31
      79 121
      70 2
      63 1
      75 1
      77 9
      7f 27427
      72 1
      73 1
      5c 1
1dcf32af 31
      76 4
      7c 512
      5d 1
      6c 1
      7a 144
ffffffff 103
      78 42
      6e 1
      7d 1037
      7e 2234
      7b 292
      55 1
      74 2


7f was very common so I wondered if that marked data blocks. Some of those others might be erased blocks, bad blocks, log blocks, other deliberate blocks and whatever else I don't know. But at the moment I am going ahead assuming the 7f marks a good block, and will try looking at whats in the other value marked blocks another time. Take the low lying fruit first.

Still I have a proportion of duplicate LPNs. Not many, but the question is how to resolve them. I have been looking for a sequence number, for instance. I thought maybe the word which would be used for LPN 128, which is useless since page 128 is the map page in each block, might be a sequence number, but there are duplicates of this number on the disk (IE its not unique so it can't be a sequence I imagine).

I think I will continue in another post.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 6th, 2017, 6:09 
Offline

Joined: June 1st, 2017, 15:09
Posts: 9
Location: London
So continuing on. I am at a point where maybe someone can suggest methods that might have been used to sequence the LPNs. I have some LPNs which come up more than once as I scan through the map blocks. I guess thinking about how the mapping system must work, the whole block becomes invalid, but it can't be marked invalid itself because it can't be written to once fully written. The logical way to do this would be to give each data block a sequence number, IE total_block_write_count or something, and then when mapping table restoration is required, as LBNs are read and put into a logical-virtual array to create a mapping array, if a duplicate is encountered, whichever LPN is associated with the later data block sequence number is kept and the older one discarded.

So there should be a sequence number in each block but I can't seem to find a unique number which looks right yet. Here is a couple of example mapping blocks with one particular LPN in both:

Code:
  7f0018       7f ffffffff ffffffff ffffffff 7fffffff   1e7508 (...126 more LPNs)   30864        3        7        0        0        0   9a79f3   a00ba5   a00bb9 ffffffff ffffffff
  7f001f        7f ffffffff ffffffff ffffffff 7fffffff   10917b (...126 more LPNs)  49046f       49 ffffffff ffffffff      1ff        0   9aa4cd   9aa56f  (...66 more 32 bit words)  954f10 ffffffff ffffffff ffffffff


Not shown (in the bit I cut for visibility), one LPN (4759441) exists in both map blocks. So the same logical page is in both blocks, one must be newer.

So in each line, the first word seems to be a virtual row number, the same for each bank per row. The second I am thinking might mark a data block type. Word 3, 4 and 5 seem mostly ffffffff or close, so they don't seem to hold much info. Then there are 128 words, of which the first 127 certainly seem to be LPNs as they work when used as such. Then there is one more word which looks like a LPN but would be useless if used as such since it corresponds to the block it is in - this is a mapping block so it should not have a LPN AFAIK. What purpose it serves I wonder, it is not unique across the disk. Then there are 5 more words which don't seem to have enough variation, or the values seem too small, to be sequence numbers. Then, most curiously, there are a variable quantity of words which look similar to LPNs. Sometimes 3 of them (as in the first line), sometimes less or more (69 of them in the second line).

Can anyone think how any of these unidentified words might be a squence number, or otherwise mark a block as being newer than another block, if they are not unique?

Cheers, Pete


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 7th, 2017, 13:06 
Offline

Joined: June 1st, 2017, 15:09
Posts: 9
Location: London
Just a little update, have worked out what the first 6 words do. Heres an example of the first 6 words:

Code:
7f0018       7f ffffffff ffffffff ffffffff 7fffffff


The first number is (row number / 128) + 0x7f0000. That could also be seen as block number + 0x7f0000, but in the physical sense, IE its the same for all banks.

The 3rd, 4th, 5th and 6th are bit fields showing valid LPNs in the following 128 words. However like everything in this drive they are all reversed, so a 3rd word of 0xfffffffe would mean the first LPN in the list is invalid (IE its zero in the binary representation of 0xfffffffe).

Word 6 is never greater than 0x7fffffff because the very last LPN field (number 128) is always invalid, being the map page for the block.

And the second word is the number of set bits in the 3rd, 4th, 5th and 6th fields added together. So it shows how many total valid LPNs are in the block of 127 pages. Once again because its never higher than 127, word 2 is never greater than 7f (127).

The LPNs marked as invalid always exist later in the physical block. So what the controller is obviously doing is writing a logical page, then if it writes the same logical page again in the same physical block (IE it updates the LPN with new data before the firmware has moved on to the next block, meaning it has to write the new data to a new page - I presume this is log block type workings), it marks the original LPN as invalid in the blocks map page. This has allowed me to eliminate some duplicate LPNs from my LPN-PPN map table.

Still no closer to finding a way of sequencing the duplicates which are not in the same physical block, but work continues.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 7th, 2017, 18:34 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 9607
Location: Australia
I would think that some blocks would contain firmware modules. The drive's model number, serial number, firmware version, SMART data, etc must live somewhere (is there a serial flash memory chip on the PCB?). I would think that there must also be a Flash Translation Layer in the firmware area. The wear levelling count must also live somewhere.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 8th, 2017, 3:27 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 9607
Location: Australia
Code:
       0 31
      55 1
      5c 1
      5d 1
      63 1
      6c 1
      6e 1
      70 2
      72 1
      73 1
      74 2
      75 1
      76 4
      77 9
      78 42
      79 121
      7a 144
      7b 292
      7c 512
      7d 1037
      7e 2234
      7f 27427
1dcf32af 31
ffffffff 103

    (0x1DCF32AF + 1) x 0x200 bytes = 256.06 GB

    0 = min LBA
    0x1DCF32AF = max LBA

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 9th, 2017, 10:06 
Offline

Joined: June 1st, 2017, 15:09
Posts: 9
Location: London
Yes the first few rows (couple of block actually) contain firmware for sure, I think bad block info in the very first rows, and other stuff which dosent match the template for the bulk of the rows which I believe are data rows (or rather blocks).

The FTL data itself might be cached in the first few blocks too, I have not searched for it there yet. But the last page in each block (IE for each bank, every 128th row) is definately LPNs, plus some other info I have not worked out yet. There should, logically, be some way of marking a previous lpn invalid here. IE once you change the data in an logical page, you need to write it to a new physical location, and then label that physical page with its logical number. But how to indicate that this is now the most up to date page for that lpn if you can't go back and erase the old one? It would be pointless to erase the whole block that the old page was in. So a sequence number for this lpn would be appropriate. But the rest of the data in the map page does not seem to look like that. Perhaps its got a more clever format since there is not enough memory on the device to store both lpn and version number as you build the map table in ram, if this is the master copy of the lpns.

I have been mulling the possible significance of the 0x1DCF32AF value alluded to by fzabkar but I can't figure it out yet. Do you have something in mind? It must be of significance. There are 32 instances of that value in that place which probably correspond to one per chip.

Anyway I thought I would write out a logical image and try to fsck it. Fsck did not like it much, it sees it but does not try to repair it, although the header to the HFS+ partition does seem intact and correct. I thought I would run a data recovery program on it (hfsprescue) to see if it turned anything up and it did to my surprise. It seems to have the majority of the structure of the fs and there are many images. The large images are the ones I am interested in (Canon raw, ~13MB). I thought there was little chance of many of these being intact since many blocks would be used for such large files, and the chance of a wrong block being in the mix would increase with the number of blocks in the file, but many of these files are intact (perhaps half). So its nice to have some of the data I was after back already, but I want to resolve this page duplication issue (about 400k duplicate pages in 16M total pages).

Messing round with writing a a whole image each time I want to try a different map is tedious so I did a little looking and there is an interesting project 'stackbd' which is a module which you can load to stack a loop device on top of another (ie pass requests to a virtual loop device it creates through it and on to another loop device). It could be hacked to translate the page number on its way through the module. So I should be able to load the logical->virtual map file I created into it and directly translate lpns to vpns on the fly which will allow me to test different map creation techniques much more easily. I think Ill work on that next and then see if I can figure out this duplicate lpn problem once I have a tool to make it easier.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 11th, 2017, 5:31 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 9607
Location: Australia
Is it possible that the information needed to resolve the LPN duplication is in the spare area? For example, could the SA of the newly written LPN contain a pointer to the previous physical page that was mapped to that LPN?

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 13th, 2017, 5:41 
Offline

Joined: June 1st, 2017, 15:09
Posts: 9
Location: London
@fzabkar thats exactly what I thought. Theres an interesting comment in the firmware of the jasmine project which says something like 'because the controller does not allow access to the spare area, we write lpn information to the last page of each block'. This is what I find in the image too, as I described above. Remember the jasmine firmwares are very basic tutorials so they are not the same as a production firmware. However in this aspect they are the same. They write the lpns of each page one by one into the last page of the block. There is also an identifier ('key'), 128 bits long which identifies lpns which are duplicated in that block - ie if a lpn is written twice in the same block of 128 pages, the first instance of it is marked in the map page key with a zero instead of a 1 to indicate its a duplicate. This seems redundant to me but anyhows.

So anyway if its correct that the controller does not permit reading the spare area, then all the info must be elsewhere, and I have found much of it in the map pages at the end of each block.

As I said before, in the map page (IE page 128 of each banks block of 128), after the lpn list, there is another array of numbers which look like lpns (or ppns). They are the right range etc. Logically these should be duplicate identifiers. However there are 2 problems. So far I have not been able to identify a correlation between the info in this section and duplicated lpns. And also, every 32nd map page (IE all of the map pages in bank 32, the last map page in each block of 128x32) do not have the extra info. They just have the lpn list, and then numbers which are very different to the other 32 banks. Its all 8080808, 2020202, 8020802, that kind of thing. Mainly 2s, 8s and 0s. Range far exceeds the lpn or ppn range. Perhaps block stats like erase count, that kind of thing I wonder.

So if bank 32 does not have the data I thought was duplicate info, how could that bank identify duplicates? Surely this means the extra info I thought was duplicate info is not actually duplicate info. Or, the controller makes sure duplicates are never put in bank 32 - seems impossible. Or, the info in bank 0 to 30 contains duplicate info for bank 0 to 31 - unlikely again, because you want to write duplicate IDs at the time you write the block LPNs. Or, the duplicate info is stored elsewhere - stupid if so, because theres stacks of wasted space in the map pages, they are hardly utilised, and this means you can't write duplicate info at the time you make it (IE you risk losing it or writing far more than you have to).

I am wondering if the element that would store the redundant (not used) lpn for page 128 might be significant. Something is in it, and it can't be a lpn. A sequence would be logical, although I can see a problem with just sequencing the whole 128 page block rather than sequencing individual pages. But its the only element which is present in all blocks, and has not already been identified as being used for something else. Modifying the stackbd driver to allow me to dynamically map lpns went well so I might go with that idea for a bit and test some different map pages created using this theory.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 15th, 2017, 7:58 
Offline

Joined: June 1st, 2017, 15:09
Posts: 9
Location: London
Having hit a bit of a wall with this duplication issue I did a bit of searching for info about the barefoot software since its understanding the ftl method thats my problem now. It turns out that theres someone on hddguru who seems to be involved in the openssd project and posted some info about it a few years ago, Jeremyb, and looking through his posts he also mentions that he went over to Korea to see Indilinx and get some info on their software, so I will pm him and see if he can add anything to this.

I'm wondering if the data in page 128 of each block is not everything that you would need to resolve the duplicates - perhaps its just there for garbage collection. Seems a waste to write 512 bytes into a 16k page with nothing else in it and not put the physical page numbers you are making obsolete in the process but perhaps thats what they did. I guess there is probably a cached mapping table somewhere on the disk but I imagine it moves around since it would be written a lot if any attention to power failure safety has been paid, but it would only be 64 meg if it was a direct page mapping, which would seem likely from the random lpn's I have in each block. Perhaps all they did is keep a few old versions as a backup in case the current mapping table block goes bad. I might try searching for it but if its some kind of hash table it might be tricky to identify and understand without the source.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 15th, 2017, 19:28 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 9607
Location: Australia
Where do the SMART data live?

Assuming that the SMART attributes are updated in RAM and then periodically written to NAND, perhaps you could locate all the corresponding pages and compare those attributes which are known to increase monotonically, eg Power On Hours, LBAs written/read. This should allow you to arrange these SMART pages in chronological order. Then you could examine the associated "metadata". Admittedly these will be firmware pages rather than user data (assuming they exist), but perhaps something might jump out at you.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Barefoot logical to virtual mapping details (M225)
PostPosted: June 15th, 2017, 19:38 
Offline
User avatar

Joined: December 4th, 2012, 1:35
Posts: 2895
Location: Adelaide, Australia
Jeremy also played around with Jasmine. Pretty sure he had the hardware kits as well. I think there was some firmware around that you could load in at boot (using the ROM Jumper) and it would act as an image dumper. Jeremy may have created it, or he acquired it. my memory of the conversation is a bit hazy.

basically I believe it does what you have done.

he runs Recovermyflashdrive, so you could call him if he doesn't visit HDDGuru soon.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 19 posts ] 

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group