All times are UTC - 5 hours [ DST ]


Forum rules


No new topics are allowed in this forum. Please use other forums to create topics.



Post new topic Reply to topic  [ 82 posts ]  Go to page 1, 2, 3, 4, 5  Next
Author Message
 Post subject: newbie info, from and for newbies :) About firmware, SA, etc
PostPosted: December 24th, 2006, 23:47 
Offline

Joined: December 23rd, 2006, 10:50
Posts: 28
Some days ago, I wanted to start a project on modding HDD firmware code. After googling for a while, I quickly learned that technical information on this subject is VERY scarce. I found this site and forum. It contains SOME info, but still, also this board is clouded with a lot of mist. Of course, I can understand why: info is scarce and you'll need to do a LOT of work, spend a LOT of time to learn new things. However, it frustrates me to a certain extent that everybody has to go through a lot of work and trouble to 're-invent' the wheel and then letting others struggle too to reinvent the wheel AGAIN. It's almost like a child abuser, that was abused by its parents and now is inflicting the same pain on their child, that they experienced themselves :)

So, in other words, here's the summary of the things I learned. If you're a complete newbie, you can now obtain this 'newbie' knowledge in a few minutes, instead of spending a few full days of researching yourself, like I did :) So, enjoy !


* NEWBIE HDD INFO*

Modern harddisks feature an area that contains information that the CPU on the HDD logicboard uses to operate the drive. That area is called the "system area" ("SA"). This area contains for example the drive 'microcode' (a.k.a. firmware), HDD Configuration Tables, Defect sector tables, SMART information, Security info (drive passwords etc), Disk ID info (serial nr etc) and more. These categories of information are called 'modules'. So the SA contains a module for the firmware code, a module for the SMART info etc.

The SA is stored on 'negative cilinders' of the HDD and therefore is not accessible by normal read commands. However, the area can be accessed with other ATA commands. An example of a (more or less) 'standard' ATA command that can access info on the SA is the 'download microcode' ATA command, that can be used to update information in the firmware code module. However, most of the commands that can be used to access the SA are vendor specific. Since vendors (obviously) don't want users to mess around with the SA, these commands are generally not made public. However, these commands can be deduced by, for example, reverse engineering the firmware code itself.

This reverse engineering has been done and led to development of tools that can issue these (vendor specific) ATA commands and can read/write almost all sectors in the SA. One example of such tool is PC3000 ('PC3K'). A tool like this contains tables per HDD model, containing these vendor specific ATA commands and also tables with sector numbers on which the different modules are stored, also per HDD model. SA Sector numbers are counted in "UBA's". For example, one specific HDD might use UBA 4 to store the 'DISK ID' module, where another HDD model might use another sector for this module.

So in short, to create a tool that can read/write data in the SA, you need to:

A) know (and understand) the (vendor-) specific ATA commands that can be used to access this area and
B) know on which UBA sector the specific modules are stored.

If a drive has damaged data in the SA, for example in the firmware code module, it might become unusable. To repair these disks, the HDD can be switched to a so called 'safe mode', by setting specific jumpers on the drive. If the drive is operating in safe mode, it bypasses its own firmware. Instead, it wants the user to upload firmware to its ram. If the user uploads a correct 'temporary' firmware to RAM, it starts executing that firmware. If this uploaded RAM code (the 'loader') starts operating, the user can then start to issue ATA commands to the drive to modify the damaged modules.

Firmware files that you can find on a site like this, contain a lot of files. First, there is the 'loader' file (*.LDR). This file is the 'temporary' firmware code, that's being uploaded to the RAM (so, it's not being written to disk). Then, there are a lot of '*.RPM' files. These files represent the different modules, which can be written to the SA. The filenames consist of 8 numbers. The first 4 numbers specify the (hex) UBA and the second 4 numbers represent the hexadecimal module size in sectors (each sector normally contains 512 bytes, so for example, if a filename ends in 0002, then that module is 1024 bytes long). So, in short, after uploading the loader to RAM, the user can start replacing damaged modules by overwriting them with correct ones.

BTW, please note that the term 'firmware' for the packages on this site is symantically not very well chosen, since these packages contain all needed modules to repair a HDD and not just the firmware (=code) module.

Anyway, if you're looking for a specific firmware module, you can do 3 things:

1) rip the firmware modules from the SA of an identical HDD or
2) get these modules from a friend (or for example, from the files section on this site) or
3) use a firmware updater program from the vendor.

About this last option: firmware updates from vendors are pretty rare, since firmware code almost never needs to be replaced. However, Maxtor for example, had some problems with the firmware code on some Diamondmax HDD models. So, they issued a firmware update. This update consists of 2 files:

1) the executable file that issues the ATA 'download microcode' command to upload the firmware files to the HDD
2) The firmware code, consisting of the 'main' firmware code and 'overlay' code modules.

Firmware 'overlay' code are specific code functions. Why not just put all firmware code into one section ? Well, since the RAM in the drive is a limited resource, they've put some code into 'overlay files', so that this specific code can be swapped into RAM when that specific function is needed. When the fuction is not needed, it can be swapped out of ram and some other function can be swapped into it again.

The firmware update files from maxtor (I think the same goes for the other vendors) are not scrambled/encrypted/packed in anyway. In fact, you can find the exact same code in these files also in the '*.RPM' files that PC3K produces for example.

Maxtor distributes their firmware file in a so called ".DMC" file. This DMC file is a package of 4 files, a '.Bxx' file, a '.cxx' file, a '.bbr' file and a '.cbr' file. Like I mentioned, this DMC container is not packed or scrambled in anyway. You can just cut the files out of it. The first 0x150 bytes of this file is the header. This header contains the four filenames, the offsets at which bytes in the package these files can be found, the length of the files and a checksum (not 100% sure about the checksum though). The '.bxx' file is the biggest file and contains the overlay modules. You can find all code overlay modules by looking for 'MO' in the file. Right after this 2 byte string, you'll find the hexadecimal overlay module ID. The '.bbr' file contains the main firmware code. The last 2 files are very small, not sure what they contain, probably some checksums for the firmware and overlay modules.

Like said, the firmware code and overlay modules can also be found in the '*.RPM' files of course, since this represents the firmware code on disk. So, you can look through these RPM files and scan for the 'MO' string to find any specific overlay module.

So, in short, if a vendor has released a firmware uploader tool (most vendors have), BUT haven't released a firmware file for your specific drive type, you could create your firmware, if you have the dumped modules (for example, obtained from this site). You could rip the main code and overlay modules and paste them into an existing DMC package. However, since I don't know the checksum calculation and the meaning of these .cxx and .cbr files (probably checksums), you'd have to do more research, but in theory, it would be possible to create your own firmware files and flash them with such standard Vendor program to disk, so you wouldn't need to buy an expensive tool like PC3000 (at least not if your sole goal was to upload a new firmware).

Of course, you could also create your own flasher program, instead of using the one supplied by the vendor. However, since vendors use specific versions of the 'download microcode' ATA command, you'd have to do research into this.

Furthermore, you could create a program that does EVERYTHING that a tool like PC3000 does. However, like pointed out, you'll need very detailed information on the vendor specific ATA commands and the structure of the SA for that specific drive type and since this info is not made public by anyone, this means a LOT of work. "But hey, the PC3000 tool features a special hardware PCI card !". Yes, but as you'll understand by now, you can think of that card as nothing more than a copy protection. They could have perfectly created the tool without it, but I guess they would have sold quite some copies less :) So you really can't blame them for it, in fact, I think it's quite a smart move to stop piracy.

So, in short, if you want to mess around with the SA, you have 2 options: invest a lot of time and energy into learning or simply empty your pockets and buy a tool like PC3000.

I hope you enjoyed this information and if appreciated, I might write another document with a summary of my future research and findings :)

Aimtrading.


Top
 Profile  
 
 Post subject:
PostPosted: December 25th, 2006, 1:55 
Offline

Joined: July 5th, 2005, 10:11
Posts: 32
Well, I certainly enjoyed it! :wink:


Top
 Profile  
 
 Post subject:
PostPosted: December 25th, 2006, 9:54 
Offline
User avatar

Joined: December 19th, 2006, 8:49
Posts: 8163
Location: Portugal
Very nice !
Please share with us your future findings !

_________________
1Q9xrDTzTddUXeJAFRn37aqh1Yr6buDCdw - (Bitcoin Donations)
The HDD Oracle - Platform for OPEN research on Data Recovery.


Top
 Profile  
 
 Post subject:
PostPosted: December 25th, 2006, 17:08 
Offline

Joined: October 3rd, 2005, 0:40
Posts: 2579
Location: Hungary
Yes, cool !
pepe


Top
 Profile  
 
 Post subject:
PostPosted: January 9th, 2007, 10:50 
Offline

Joined: October 19th, 2006, 11:56
Posts: 222
Nice work bro. I appreciate it. Great Job. Keep it up.


Top
 Profile  
 
 Post subject:
PostPosted: January 14th, 2007, 11:07 
Offline

Joined: June 27th, 2006, 11:33
Posts: 2314
Location: In ur HDD !
Good article good step for newbie :D


Top
 Profile  
 
 Post subject:
PostPosted: May 7th, 2007, 8:18 
Offline

Joined: February 16th, 2007, 3:07
Posts: 1
hmm great way of sharing ...


Top
 Profile  
 
 Post subject:
PostPosted: May 7th, 2007, 9:07 
Offline

Joined: October 1st, 2006, 6:29
Posts: 107
Certainly worth reading, great job.


Top
 Profile  
 
 Post subject: Cheers
PostPosted: May 7th, 2007, 21:32 
Offline

Joined: May 7th, 2007, 2:43
Posts: 3
Location: Australia
Overviews are good for everyone, not only newbies.
It's important to remember the forest as we keep hitting our heads against the trees.
:-)
Thanks.


Top
 Profile  
 
 Post subject:
PostPosted: May 7th, 2007, 23:18 
Offline
User avatar

Joined: October 6th, 2005, 3:00
Posts: 658
Good Article friend...
But this is just for maxtor. How about for Seagate. I hope next article for Seagate will be available...

Thanks
GREAT !!!!!!!


Top
 Profile  
 
 Post subject: Re: newbie info, from and for newbies :) About firmware, SA,
PostPosted: May 20th, 2007, 8:23 
Offline

Joined: May 20th, 2007, 8:12
Posts: 5
HI,
I am a chinese BOY......
Thanks Very Much!
I Need SOME FW.......So,I REG this MEMBer....
SOMEone HAVE
ST 94011A
FW3.06.
HELP ME!....HELP ME!....HELP ME!....

QQ:44334524
EMAIL:tongfanghw@163.com

NOW.....thanks...........thanks .... :)


Top
 Profile  
 
 Post subject:
PostPosted: May 21st, 2007, 9:33 
Offline

Joined: May 27th, 2005, 17:17
Posts: 17
Location: Bogotá,
Thank you very much for the useful info. I think your research could use Idegrab: http://www.nazyura.newmail.ru/index.htm. Best wishes and good luck with your project. You have helped to clear the myst for me, thank you again.


Top
 Profile  
 
 Post subject: Hi Very Good article
PostPosted: May 23rd, 2007, 12:40 
Offline

Joined: June 5th, 2006, 1:09
Posts: 64
Location: New Delhi India
Hi am from india and absolute new bie .I do logical recoveries and want to learn physical recoveries.
Your article is very very helpful. Please keep it up.
I am planning to buy pc3000 chinease version. Please suggest me good vendor.
Thanks
Best Regards
chipsang


Top
 Profile  
 
 Post subject:
PostPosted: May 23rd, 2007, 12:58 
Offline

Joined: October 3rd, 2005, 0:40
Posts: 2579
Location: Hungary
Hi,

'Chinese' version is the pirated version of the original product, so there is no good vendor :)
However there is a reseller of the original manufacturer in china, I recommend to contact him.

regards,
pepe


Top
 Profile  
 
 Post subject:
PostPosted: May 23rd, 2007, 14:03 
Offline

Joined: December 23rd, 2006, 16:08
Posts: 938
Location: NJ
Does anyone know how to buy PC3000 from the real China reseller, if you don't speak Chinese?


Top
 Profile  
 
 Post subject: nice topic
PostPosted: July 9th, 2007, 5:04 
Offline

Joined: December 18th, 2006, 10:33
Posts: 6
Thankls for your good information

I bought a PC-3000 from RUSSIA . your information were so usefull for me.

Thanks alot


Top
 Profile  
 
 Post subject: Re: newbie info, from and for newbies :) About firmware, SA, etc
PostPosted: August 5th, 2007, 13:44 
Offline
User avatar

Joined: September 27th, 2005, 14:14
Posts: 125
Location: In the Middle - bit to the left
Yes, it's a good read. I think info about other vendors would be good also. The info I just read is very much "Maxtor" orientated. UBA , *.bbr files etc. But none the less it's a start for many of you :ookay:

Maybe someone could do a firmware reverse "Master Class" for us :D

_________________
A man is as old as he feels, and a woman as old as she looks


Top
 Profile  
 
 Post subject: Re: newbie info, from and for newbies :) About firmware, SA, etc
PostPosted: September 26th, 2007, 18:48 
Offline

Joined: October 21st, 2005, 0:45
Posts: 1489
Location: Mexico
My suggest for newbies

Look for ATA protocol´s and read about ATA registers, im going to put my idea about learning about data recovery on this post , because im think its need it .

On my case one good people some years ago from Brazil , begins to teach me about hard disks, data recovery, etc wich was this method wich he told me?

Read Books, read documents like , many times i ask him for quickly solutions like" can u teach me how to do a head exchange, how to repair a SA; etc" and he told me, "Data recovery its not a push button solution, u need to study, learn, think on Hard disk, etc," my first imprresiion was " he is a bad people because , he doesnt want share with me his knowledgement about Data Recovery, SA, structure, electronic, mecanic,head exchange, diangostics, etc on one word Engineering on HDD´s but he told me the only way , in wich u can learn about hard disk engineering and data recovery, its by u self. with your own cases,with practice, looking the hdd how works practice with dummy disks, etc reading, books, understanding the mechanism of hdd´s this was some late for begin to understand this but thanks GOD now, i can said " Thanks friend because on this way u force to me to understand the best way to learn abut HDD engineering many thanks and sucess ." So this is my suggestion because im start from this way, its hard, need nights without not sleep, need many patience, practice,think on Hard Disk, think on whats happening with this cases , and the most need to understand its not a easy work like many people think , im buy products " tools" , but this tools they are not solutions like " next--> next--> " Finish :lol: Smile Data recovery! from any cases!

My suggest

Begin to understand on fist low level work´s what does mean low level "problems until hard disk are detect by BIOS read about Logical Structure MBR, PT, VBS, NTFS, MFT, FAT etc , im suggest this book "Upgrading & Repairing Pc´s " From Scott Mueller, here are some useul links

U can use programs like winhex to understand, sector level´s , after u get all of this begin to read and understand the protocol command between HDD and Host, ATA protocols, registers, commands , programs like mhdd,victoria, hDDL are very good for understand some of this. this is only a little.

ATA protocol´s
http://www.t13.org/Documents/Default.aspx?keyword=ATA.


http://www.ntfs.com/ntfs-mft.htm


Best Regards

Sinceraly

Alberto Pavón


Top
 Profile  
 
 Post subject: Re: newbie info, from and for newbies :) About firmware, SA, etc
PostPosted: April 16th, 2008, 6:52 
Offline

Joined: April 10th, 2008, 12:05
Posts: 35
I have been reading the information that you have linked and it's very util. Thanks to all.

Regards.


Top
 Profile  
 
 Post subject: Re: newbie info, from and for newbies :) About firmware, SA, etc
PostPosted: April 24th, 2008, 3:36 
Offline

Joined: April 24th, 2008, 3:21
Posts: 2
Hi,
My be very, very stupid question. A have ST320410A form Apple. Can I use this thing with a pc as a master for ex.
Thanks


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 82 posts ]  Go to page 1, 2, 3, 4, 5  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group