All times are UTC - 5 hours [ DST ]


Forum rules


Please do not post questions about data recovery cases here (use this forum instead). This forum is for topics on finding new ways to recover data. Accessing firmware, writing programs, reading bits off the platter, recovering data from dust...



Post new topic Reply to topic  [ 27 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: LDPC research
PostPosted: August 21st, 2023, 6:29 
Offline

Joined: August 13th, 2016, 17:10
Posts: 197
Location: Vienna, Austria
I did some LDPC research lately and developed an algorithm to extract the LDPC parameters from a donor or a patient controller and to fix the bit errors on patient dumps. I could need some help from people with a logic analyzer / VNR / FE / PC3K-Flash to test my solution.


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 21st, 2023, 10:43 
Offline
User avatar

Joined: May 13th, 2019, 7:50
Posts: 913
Location: Nederland
While I try to catch my breath: can you explain how someone with FE could help?

_________________
Joep - http://www.disktuna.com - video & photo repair & recovery service


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 21st, 2023, 12:14 
Offline
User avatar

Joined: August 15th, 2006, 3:01
Posts: 3471
Location: CDRLabs @ Chandigarh [ India ]
Well,
It was a pleasure speaking to you today ,I would be contributing a lot of cases and access to VNR/PC3K .

_________________
Regards
Amarbir S Dhillon , Chandigarh Data Recovery Labs [India]
Logical,Semi Physical And Physical Data Recovery
Website-> http://www.chandigarhdatarecovery.com


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 21st, 2023, 14:40 
Offline
User avatar

Joined: May 13th, 2019, 7:50
Posts: 913
Location: Nederland
Well,
It was a pleasure speaking to you today, thanks for your explanation.

_________________
Joep - http://www.disktuna.com - video & photo repair & recovery service


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 22nd, 2023, 1:43 
Offline
User avatar

Joined: July 8th, 2019, 12:27
Posts: 148
Location: 中国大陆浙江省湖州市
Based on my personal experience, developers of 3K and VNR tend to be more reserved and less active within the community. Despite their strong capabilities, staying updated with the latest and comprehensive information can be challenging without community interaction.
Regarding LDPC, what are the specific tasks?

_________________
Auxiliary Tool Used For MonoLith Data Recovery, featuring the industry's most extensive Monolith pinouts
http://flash-matrix.com/


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 22nd, 2023, 15:59 
Offline

Joined: September 23rd, 2019, 2:55
Posts: 60
Location: Poland
sourcerer wrote:
I did some LDPC research lately and developed an algorithm to extract the LDPC parameters from a donor or a patient controller and to fix the bit errors on patient dumps. I could need some help from people with a logic analyzer / VNR / FE / PC3K-Flash to test my solution.



Is this yours?
https://futureware.at/CrowdFunding/flashextractor.php

This is related to your github?

If yes, why is this doing behind sergey back?

https://futureware.at/CrowdFunding/flashextractor.php

Thanks for answer.


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 22nd, 2023, 16:27 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15538
Location: Australia
Gregory wrote:
sourcerer wrote:
I did some LDPC research lately and developed an algorithm to extract the LDPC parameters from a donor or a patient controller and to fix the bit errors on patient dumps. I could need some help from people with a logic analyzer / VNR / FE / PC3K-Flash to test my solution.



Is this yours?
https://futureware.at/CrowdFunding/flashextractor.php

This is related to your github?

If yes, why is this doing behind sergey back?

https://futureware.at/CrowdFunding/flashextractor.php

Thanks for answer.

What's your problem, Greg?

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 22nd, 2023, 19:20 
Offline

Joined: August 13th, 2016, 17:10
Posts: 197
Location: Vienna, Austria
Hi,

Is this yours?
https://futureware.at/CrowdFunding/flashextractor.php

This is related to your github?

If yes, why is this doing behind sergey back?

https://futureware.at/CrowdFunding/flashextractor.php

Thanks for answer.[/quote]

Yes, that's mine. And it is related to my github, absolutely.
It's a draft for a CrowdFunding campaign that I am currently planning together with Sergey, and Sergey approved a slightly older version of it already, but we wanted to continue working and refining it before making it public. There are still a few details to be sorted out, but the big picture of the plan is there.

Basically, I heard that Sergey wanted to stop the development and the support of Flash Extractor, and I thought that this would a big loss for the data recovery industry, so I discussed it with Sergey and several other people, and proposed to help with the development and to make it opensource. Some feedback I got was that raising money for FE without having LDPC support would be futile in the long run, so I started to work on LDPC and developed a solution. The idea now is to add the LDPC solution into the offer, to also make the LDPC solution opensource and available to public when the campaign is successful. But the long-term goal is to preserve the investments Sergey and everyone else did into FE, to not just maintain it, but to make it far better, by working on it and with it together as a community!

So I would suggest the following: If anyone has any feedback on the campaign draft there, please let me know so that I can work it in. I will further improve it and we will likely launch the campaign in a few days. If anyone wants to join and help the effort, please let me know.

PS: Thanks to everyone who helped me so far, and thanks for pushing me to finally make this plan public, sometimes I overdo drafting :-)


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 23rd, 2023, 1:23 
Offline

Joined: September 23rd, 2019, 2:55
Posts: 60
Location: Poland
fzabkar wrote:
What's your problem, Greg?


Wrong second link, I want ask for this http://flash-extractor.com/forum/viewtopic.php?t=12967

In Sergey opinion this is fraud, so i'm wondering what's going on here?


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 23rd, 2023, 4:54 
Offline

Joined: November 7th, 2020, 5:31
Posts: 1092
Location: The_UK
LDPC development work is going to be needed but it doesn't appear to be coming from anywhere else at the moment. Anything is better than nothing, and opensource is even better, there are plenty who could assist that are not involved directly in DR so why waste that resource.

With hardware running out for FE that will become a closed environment entirely and we're left with a duopoly with pc3K and VNR that isn't going to drive development forward which is not good for any of us.

As to "fraud" I think Sergey was probably looking for word "scam" expressing his concerns the project will turn into vapourware from general complexity or from feature creep. Which having read the page, it does make a lot of promises. The LDPC trials will go some way to answer credibility and the future development side of things.

My personal opinion is that if you're going to work on more complex and current requirements then you need to have the correct foundations and building onto FE may not be the best place to start from, it is after all a legacy application. Perhaps there's some secret sauce in there that you can't recreate or you're trying to preserve the workflow for the end user?

_________________
Data Recovery Services in the UK.
https://www.usbrecovery.co.uk/


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 23rd, 2023, 14:34 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15538
Location: Australia
Gregory wrote:
fzabkar wrote:
What's your problem, Greg?


Wrong second link, I want ask for this http://flash-extractor.com/forum/viewtopic.php?t=12967

In Sergey opinion this is fraud, so i'm wondering what's going on here?

To me, the obvious answer is to provide @sourcerer with a raw NAND dump from a working drive and then see whether he can provide the files or file list. Of course, the controller should be one that @sourcerer chooses, as long as it is one which is regarded as unsolvable. In this way @sourcerer doesn't need to disclose his algorithm.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 23rd, 2023, 17:04 
Offline

Joined: September 23rd, 2019, 2:55
Posts: 60
Location: Poland
@sourcerer With what page structure your LDPC algorithm was testing: 4576, 4568_4632, 4628_4692, 4564_4644, 2280, 2324?


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 24th, 2023, 8:51 
Offline

Joined: August 13th, 2016, 17:10
Posts: 197
Location: Vienna, Austria
At the moment I am simulating it with a simple 4000 byte page structure which contains 1024-DATA 476-ECC 1024-DATA 476-ECC 8-SA. My simulator can easily be parameterized for different geometries. Can you provide a few blocks of dump and all the details about the geometry of the case you have?


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 24th, 2023, 14:35 
Offline

Joined: September 23rd, 2019, 2:55
Posts: 60
Location: Poland
sourcerer wrote:
Can you provide a few blocks of dump and all the details about the geometry of the case you have?

I have access to few donors, if I find lite time read few blocks with 0x77 pattern.

btw. 1024bytes protect by 476bytes ?? for example NAND with LDPC have 4096byte data, protect by 472 bytes ecc - 4 times more data.
Do your algo use only hard decision decoding technique or also soft decision?


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 26th, 2023, 9:06 
Offline

Joined: August 13th, 2016, 17:10
Posts: 197
Location: Vienna, Austria
Thank you, I will test it with 4096 and 472 too. From my point of view, choosing a good LDPC length not just depends on the size of the data area, but especially on the number of bit-errors that are to be expected. The more noise you have there, the larger ECC area I would suggest to use. I have experienced quite noisy and quite stable NAND flash in the past. The minimum sum algorithm I am using at the moment is using only hard decisions as far as I know, but from what I have seen we should be able to use various different decoders, both hard-decision and soft-decision. I think it would make sense to combine soft-decision decoders with Read-Retry. Not all decoders will be suitable for us, but from the decoder market overview I have at the moment I think that we will find about 10 decoders that we can use, and I hope that one of them will be fast enough for our needs.


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: August 26th, 2023, 23:30 
Offline
User avatar

Joined: July 8th, 2019, 12:27
Posts: 148
Location: 中国大陆浙江省湖州市
I believe that the feasibility of LDPC soft decision at the current stage is relatively low, as it requires obtaining accurate soft information from NAND chips. If the acquired soft information is incorrect, it could lead to catastrophic consequences. Currently, almost all developers cannot guarantee that their utilized ReadRetry implementations are fully compliant with the specifications, as those instructions are often derived from reverse engineering. Perhaps certain values do not belong to the addresses of those registers. However, we have no way of knowing the potential repercussions if those erroneous values are used; they could potentially disrupt the entire dump.

_________________
Auxiliary Tool Used For MonoLith Data Recovery, featuring the industry's most extensive Monolith pinouts
http://flash-matrix.com/


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: October 4th, 2023, 14:56 
Offline

Joined: August 13th, 2016, 17:10
Posts: 197
Location: Vienna, Austria
I am happy to present you my solution for LDPC now:
The necessary tools are available on https://github.com/thesourcerer/drresearch/
and I made a first video that shows how to use it on https://www.youtube.com/watch?v=z93vdLnfoDs


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: October 4th, 2023, 17:09 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15538
Location: Australia
Quote:
The necessary tools are available on https://github.com/thesourcerer/drresearch/


404 - "This is not the web page you are looking for"

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: October 5th, 2023, 2:20 
Offline

Joined: August 13th, 2016, 17:10
Posts: 197
Location: Vienna, Austria
Sorry, the is a typo, the correct link is https://github.com/thesourcerer8/drresearch/


Top
 Profile  
 
 Post subject: Re: LDPC research
PostPosted: October 5th, 2023, 3:16 
Offline

Joined: October 24th, 2009, 15:22
Posts: 875
Location: Poland
How you plan to find read parity check matrix? So far we can see patter written on video.

_________________
Flash Killer - everyday new resources (pinout, XOR, ECC,config) for flash devices


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 27 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 45 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group