All times are UTC - 5 hours [ DST ]


Forum rules


Please do not post questions about data recovery cases here (use this forum instead). This forum is for topics on finding new ways to recover data. Accessing firmware, writing programs, reading bits off the platter, recovering data from dust...



Post new topic Reply to topic  [ 101 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next
Author Message
 Post subject: Re: Head Map Parser
PostPosted: January 24th, 2021, 15:15 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
A new useful update is, today i got a case of a samsung drive, 1tb ST xxx LM series (ie seagate samsung)
Parsed the map and imaged it on ddi smoothly ..
Also, the zone average skip size readings are pretty helpful....i programmed ddi to skip sectors that are mentioned in summary as per zone average skip size and i could see the Head count move to next consecutive head if space bar was pressed...
:)


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 15:23 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
I compiled the tool for 32-bit Win 10. However, it only managed to parse the first segment. Can anyone else reproduce this result?

Code:
C:\Downloads\Software\Head_Map_for_DDI>To_Upload
Do you wish to enable printing start/end LBA and lengths? 1 = Enable; 0 = Disable1
Do you wish to save Lba Log? 1 = Save; 0 = Dont Save1
HEAD# 1         START LBA = 0   LENGTH= 55676600 (26.55 GB)     END LBA= 55676599


                                                 S U M M A R Y


The maximum heads found in the HDD as per map  is 4 heads starting count from number 1
Total Size Parsed as per individual lengths = 26.55 GB
Head# 0 zone is 0.00 GB which is 0.00 percent and average zone size is nan GB. ZONE AVG SKIP SIZE: nan sectors
Head# 1 zone is 26.55 GB which is 100.00 percent and average zone size is 26.55 GB. ZONE AVG SKIP SIZE: 55676600 sectors
Head# 2 zone is 0.00 GB which is 0.00 percent and average zone size is nan GB. ZONE AVG SKIP SIZE: nan sectors
Head# 3 zone is 0.00 GB which is 0.00 percent and average zone size is nan GB. ZONE AVG SKIP SIZE: nan sectors
Head# 0 has 0 zones
Head# 1 has 1 zones
Head# 2 has 0 zones
Head# 3 has 0 zones
Sum total of individual heads zone is 26.55 GB

Number of zones are 177 in the full LBA
CONDITION: LBA size in terms of over all length equals to the sum of the LBA size under each individual head
MAP PARSING SUCCESSFUL AS ABOVE CONDITION IS MET

C:\Downloads\Software\Head_Map_for_DDI>dir
Volume in drive C is Boot_Disc
Volume Serial Number is FC57-01A7

Directory of C:\Downloads\Software\Head_Map_for_DDI

28/01/2021  06:13 AM    <DIR>          .
28/01/2021  06:13 AM    <DIR>          ..
28/01/2021  06:14 AM        55,676,601 DE LBA PARSED.MAP
27/01/2021  04:59 PM    <DIR>          inputmap
28/01/2021  06:14 AM               148 lba_log.txt
28/01/2021  06:14 AM               968 summary.txt
23/01/2021  06:20 AM         1,013,158 TOUPLOAD.zip
23/01/2021  09:37 PM            16,555 To_Upload.cpp
28/01/2021  06:13 AM         1,438,297 To_Upload.exe
24/01/2021  08:28 AM               126 unt
24/01/2021  08:36 AM               199 unt.zip
               8 File(s)     58,146,052 bytes
               3 Dir(s)  717,031,485,440 bytes free

AFAICT, the first IF succeeds, but all subsequent tests fail.

Code:
////////////////////////MAIN Locators START//////////////////////////////////////////////


    while ( i <fileLen) {

int j=i;

if(buffer[j] == footer[0] && buffer[j+1] == footer[1] && buffer[j+2] == footer[2] && buffer[j+3] == footer[3]  && buffer[j+4] == footer[4] && buffer[j+5] == footer[5] && buffer[j+6] == footer[6] && buffer[j+7] == footer[7] && buffer[j+8] == footer[8] && buffer[j+9] == footer[9] )

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 15:36 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
I ended up writing my own tool.

The Palmer and Charger results look very weird (excerpt from Palmer log shown). Not only are the segments strange, but the total capacity doesn't make sense, even if I allow for a sector size of 4096 bytes.

Code:
Head     Start LBA       End LBA     Len (LBA)    Len (MB)
----  ------------  ------------  ------------  ----------
   0             0          7095          7096        3.63
   0          7096         17527         10432        5.34
   0         17528         19855          2328        1.19
   0         19856         28855          9000        4.61
   0         28856         29951          1096         .56
   1         29952         34871          4920        2.52
   0         34872         35159           288         .15
   1         35160         35423           264         .14
   0         35424         39335          3912        2.00
   1         39336         40383          1048         .54
   0         40384         55519         15136        7.75
   1         55520         56167           648         .33
   0         56168         59583          3416        1.75
   1         59584         82751         23168       11.86
   0         82752         85783          3032        1.55
   1         85784         85791             8        0.00
   0         85792         85967           176         .09
...

                    S U M M A R Y

Head  Serpents  Cap (GB)   % Cap   Avg (MB)     Avg (LBA)
----  --------  --------  ------  ---------  ------------
   0     11522     54.44   68.60       4.72          9228
   1     11174     24.92   31.40       2.23          4356


Total number of serpentine segments = 22,696
Total number of heads in Data Extractor MAP file = 2
Total LBAS (sum of all serpentine segments) = 154,998,296
Total capacity (GB) of all serpents = 79.36


Attachments:
palmertest_log.7z [124.32 KiB]
Downloaded 537 times

_________________
A backup a day keeps DR away.
Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 17:42 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
Hello! whatsup!
It is very essential for 3k to generate map files till the fullest capacity of the drive.
Now these map files are created by 1) create head map 2) create submap using head info 3) exporting the sub map created into a external .map file.

I have observed if I ask 3k to use logical sector size instead of the physical sector(4096 i believe) size for palmer/charger and possibly other wd families, it creates (2) not till the final LBA of the drive and that is the only reason either of our tools (your version and my version ) would not be effective.

There is this specific notification which says " Size of physical sector is different than the logical sector. Use physical sector? to which if i say dont use, it creates sub maps lesser than the actual lba of the drive.

So if now again, if i dont create the MAP with physical sector size, 3k itself wont really give me an option to switch off head after certain lba as it itself has created a limited HM when logical sector size is selected.

This is not the case if 4k sector size is used while creating HM. However the LBA value would be way too less (??4k --> lesser lba ??)

Luckily just one charger gave map of about 30GB lesser than the final value. I guess it has to do with the way 3k builds the HM with reference to 190 and other modules.

However, for Seagate LM series that are 4096, the option (2) creates full length maps. TS can't really answer what this is all about.
I believe its just the way 3k uses the translator, the second level translator along side a few bunch of modules that are used to map newer WD drives.


So, now I actually am trying to parse the original head.map file for two reasons
a) i dont want to create the HM first and then create the sub map using heads info
b) more programming...yay :)


Addendum: I am unsure why the exterior loop is executing just once for your compiler.. Can you try to re download the file and test?
--


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 17:48 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
Adding to above post, the map files provided in the zip provided are created by
1) create head map 2) create submap using head info 3) exporting the sub map created into a external .map file.

in 3) one could simply export the map as sq3/database file and it can be further converted to xls. Which means, the file can also be CSV and one could simply convert CSV values and parse them, instead of writing lengthy code! :P
But yeah to reach uptill the CSV, the process gets longer..


NOTE: The lengths column when auto summed don't yield final lba value as discussed above any way.

We can safely conclude, nothing is wrong with our parsers
--


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:00 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
In my code I always check for continuity between segments. That is, I confirm that the end LBA of the current segment immediately precedes the start LBA of the next segment. In all your MAP files this has always been the case. Therefore the sum of all the individual lengths does in fact correspond to the final LBA, even in the Palmer and Charger examples.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:06 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
exclude the palmer and charger examples...their final lengths are not parsed correctly by 3k when logical sizes are used and thats how those map files are made here.

Something looks buggy with your compiler...would you allow me to do a TV / AnyDesk and see for myself?
Clearly thats not the case with me..I would really want to know if anybody else experienced the same as you did..

Did you use DevC++ ?
--


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:07 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
Adding further, i have more material as there is an Exos 8TB being confirmed by a client...I expect it to have atleast 10 heads :D :)


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:15 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
Yes, I used DevC++.

As for TeamViewer, sorry, but the only people who have access to my machine are myself and the hackers who target me from time to time. :-)

I will try to add my own "debug" lines to see what is happening. It might take me a while, though. FWIW, I did notice that DevC++ gave me a warning message when I launched it in 32-bit mode, but I didn't think anything of it.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:16 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
Just for transparency sake:- https://youtu.be/Q8x8pq4GfKg :)

wait a min, I have removed the above video as it displayed a few things that Indian competitors should not look at :P


Last edited by sin on January 28th, 2021, 18:23, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:17 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
as sourcerer pointed to be in initial stages of coding, where i thought i was ready and he indicated that I should keep -warn parameters on while compiling to see all warnings. It took me about 30mins to sort the initial errors and then kept of developing error free. I guess he would join in soon.


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:26 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
Just curious. How long does it take your tool to process the chotelal.map file, with logging switched off?

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:31 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
see, the fwrite function would create a huge ass binary map file... just disable it using //
even with 'logging on' it will be matter of seconds
but yeah, if you are keeping the fwrite function on, a 1tb drive binary map would be created weighing 1.8GB ..
which takes time...say around or under 5 mins on my SSD.

heres the link, you can see for yourself. I disabled the fwrite function so a huge ass map is not created.

This would still log the lba start end length values and summary...but no DE LBA.Map file will be created
https://youtu.be/NNV-ClxP-Iw


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:41 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
I have an old Core 2 Duo, 4GB, SSD box which does the job in 13 seconds (using my tool). I'm not much of a programmer, but ISTM that you could replace those repetitive single-byte file writes with a single write, if you were to build a large string.

In FreeBasic there is the String function which allows me to create a string up to 2GB long consisting of a single repeating character:

https://www.freebasic.net/wiki/KeyPgStringFunction

I don't know if there is an equivalent function in C++.

I also use User Defined Types ("struct") for holding the segment data. That obviates the necessity to search for the footer text, and it avoids endianness conversions. I would think that would simplify and accelerate your code as well.

_________________
A backup a day keeps DR away.


Last edited by fzabkar on January 28th, 2021, 18:46, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:45 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
You are correct Franc. I was trying out something similar. But, strangely, if i didnt use the for loop and used the following
fwrite(&BinaryHeadNumber, 1, length_of_Sectors , binary_map); //no pointers used, & , size

i would get hilarious results. Which not only included names of some countries, time zones and what not to even parts of the original map file being parsed. I guess it hast to do with memory updation that I had no control over.
Hence the for loop :D

--


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 28th, 2021, 18:48 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
I am sure there is a better way and I will find it.
Good night for now, its already 4am and its about time Dr.Arora would wake up and not find me on my bed.
I would get some pastings from him :P ...I am about 29 now and as an Indian kid , we can get scolded at even 50 years of age.
He is a very noble man that way with over 50k 60k operations under his belt as the main Anesthesiologist without any on table deaths...... The guy who gifted me a soldering iron when I was about 5/ 8 years and taught me how to read schematics around 12. I still remember how he would correct me with the transistor pin outs while I tried building some flip flops triggering alternative LEDs on and off :D

Good night.


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 29th, 2021, 17:53 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
I added two debugging lines to your code:

Code:
    while ( i <fileLen) {

int j=i;

printf("%x,%x,%x,%x,%x,%x,%x,%x,%x,%x,footer\n",footer[0],footer[1],footer[2],footer[3],footer[4],footer[5],footer[6],footer[7],footer[8],footer[9]);

printf("%x,%x,%x,%x,%x,%x,%x,%x,%x,%x\n",buffer[j],buffer[j+1],buffer[j+2],buffer[j+3],buffer[j+4],buffer[j+5],buffer[j+6],buffer[j+7],buffer[j+8],buffer[j+9]);

if(buffer[j] == footer[0] && buffer[j+1] == footer[1] && buffer[j+2] == footer[2] && buffer[j+3] == footer[3]  && buffer[j+4] == footer[4] && buffer[j+5] == footer[5] && buffer[j+6] == footer[6] && buffer[j+7] == footer[7] && buffer[j+8] == footer[8] && buffer[j+9] == footer[9] )
{


I created a MAP file with two segments:

Code:
Offset(h) 00   02   04   06   08   0A   0C   0E   10   12   14   16   18   1A   1C   1E   20   22   24   26   28

00000000  FBFF FFFF 712C 0000 0000 0000 0000 0000 0000 0000 0000 0000 A805 0000 0000 0000 0000 0000 0600 4800 6500  ûÿÿÿq,..................¨.............H.e.
0000002A  6100 6400 2300 3200 A805 0000 0000 0000 A805 0000 0000 0000 D800 0000 0000 0000 0000 0000 0600 4800 6500  a.d.#.2.¨.......¨.......Ø.............H.e.
00000054  6100 6400 2300 3300                                                                                       a.d.#.3.

Your tool then produced the following output (edited for brevity):

Code:
48,0,65,0,61,0,64,0,23,0,footer
6,0,48,0,65,0,61,0,64,0
48,0,65,0,61,0,64,0,23,0,footer
0,48,0,65,0,61,0,64,0,23
48,0,65,0,61,0,64,0,23,0,footer
48,0,65,0,61,0,64,0,23,0
HEAD# 2         START LBA = 0   LENGTH= 1448 (0.00 GB)  END LBA= 1447
a8,0,65,0,61,0,64,0,23,0,footer
0,65,0,61,0,64,0,23,0,32
a8,0,65,0,61,0,64,0,23,0,footer
65,0,61,0,64,0,23,0,32,0
a8,0,65,0,61,0,64,0,23,0,footer

Notice that footer[0] has changed from 0x48 to 0xa8. Weird!?

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 29th, 2021, 18:11 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
If I amend the second While loop in your code as follows, then it produces the correct result.

Code:
    while ( i <fileLen) {

int j=i;

footer[0] = 0x48;

if(buffer[j] == footer[0] && buffer[j+1] == footer[1] && buffer[j+2] == footer[2] && buffer[j+3] == footer[3]  && buffer[j+4] == footer[4] && buffer[j+5] == footer[5] && buffer[j+6] == footer[6] && buffer[j+7] == footer[7] && buffer[j+8] == footer[8] && buffer[j+9] == footer[9] )
{

Code:
Do you wish to enable printing start/end lba and lenghts? 1= Enable; 0 = Disable1
Do you wish to save Lba Log 1= SAVE; 0 = Dont Save1
HEAD# 2         START LBA = 0   LENGTH= 1448 (0.00 GB)  END LBA= 1447
HEAD# 3         START LBA = 1280        LENGTH= 216 (0.00 GB)   END LBA= 1495


                                                 S U M M A R Y


The maximum heads found in the HDD as per map  is 4 heads starting count from number 1
Total Size Parsed as per individual lengths = 0.00 GB
Head# 0 zone is 0.00 GB which is 0.00 percent and average zone size is nan GB. ZONE AVG SKIP SIZE: nan sectors
Head# 1 zone is 0.00 GB which is 0.00 percent and average zone size is nan GB. ZONE AVG SKIP SIZE: nan sectors
Head# 2 zone is 0.00 GB which is 87.02 percent and average zone size is 0.00 GB. ZONE AVG SKIP SIZE: 1448 sectors
Head# 3 zone is 0.00 GB which is 12.98 percent and average zone size is 0.00 GB. ZONE AVG SKIP SIZE: 216 sectors
Head# 0 has 0 zones
Head# 1 has 0 zones
Head# 2 has 1 zones
Head# 3 has 1 zones
Sum total of individual heads zone is 0.00 GB

Number of zones are 2 in the full LBA
CONDITION: LBA size in terms of over all length equals to the sum of the LBA size under each individual head
MAP PARSING SUCCESSFUL AS ABOVE CONDITION IS MET

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 29th, 2021, 23:05 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15461
Location: Australia
I added these debugging lines:

Code:
printf("line 160, footer[0] = 0x%x\n",footer[0]);

for(length_Value_Feeder =0;length_Value_Feeder <= 7; length_Value_Feeder++  )
{
temp_read_length_buffer[length_Value_Feeder] = buffer[length_Offset-length_Value_Feeder];
}

printf("line 167, footer[0] = 0x%x\n",footer[0]);

That is the area where the footer[0] value changes.

Code:
footer[0]] = 0x48,  i = 38
line 160, footer[0] = 0x48
line 167, footer[0] = 0x57
HEAD# 2         START LBA = 0   LENGTH= 1367 (0.00 GB)  END LBA= 1366
footer[0]] = 0x57,  i = 39

I used this test file:

Code:
Offset(h) 00   02   04   06   08   0A   0C   0E   10   12   14   16   18   1A   1C   1E   20   22   24   26   28

00000000  FBFF FFFF 712C 0000 0000 0000 0000 0000 0000 0000 0000 0000 5705 0000 0000 0000 0000 0000 0600 4800 6500  ûÿÿÿq,..................W.............H.e.
0000002A  6100 6400 2300 3200 A805 0000 0000 0000 A805 0000 0000 0000 D800 0000 0000 0000 0000 0000 0600 4800 6500  a.d.#.2.¨.......¨.......Ø.............H.e.
00000054  6100 6400 2300 3300                                                                                       a.d.#.3.

As before, footer[0] takes on the value of buffer[18]. Clearly this is not a physical RAM problem. To me it looks like some sort of weird compiler bug.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Head Map Parser
PostPosted: January 30th, 2021, 3:24 
Offline

Joined: September 17th, 2016, 16:06
Posts: 430
Location: India
the program is divided into two halves. (well thats silly).
The first half determines max head count and stores it as a global integer variable before the main function.

The incrementing agent is 'i.' Try to first use just that above section of the program and remove the bottom section completely.. Make sure the braces are closed and return 0; is incorporated as last lines.

You should see if i increments by printing it.
--


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 101 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group