How many spare non-LBA sectors?

[pleriche]

I wonder if anyone can tell me typically how many spare non-LBA sectors (as a propotion of the LBA size) a HDD typically has. Simple erasure tools are unable to clean these or the bad sectors that have been remapped to them but some government accredited tools (such as Blancco) can. Hence I want to try and put a theoretical absolute upper limit on the amount of data that might survive a simple erase.

As a supplementary question: can I access non-LBA sectors in any way using MHDD? I presume there must be some kind of flag that can be set via the ATA interface, or option on the READ command, to disable remapping in accordance with the G-list. And then maybe the higest readable sector might include the spare area? And then maybe I could see the raw disk size? I can't see any way to do this through MHDD.

[Arch Stanton]

I wonder if anyone can tell me typically how many spare non-LBA sectors (as a propotion of the LBA size) a HDD typically has. Simple erasure tools are unable to clean these

They're spares so no need to clean them, no user data was written to those.

Simple erasure tools are unable to clean these or the bad sectors that have been remapped to them but some government accredited tools (such as Blancco) can.

Blanco can not either touch them itself, if can report if enhanced secure erase is available and run that if it is.

[pleriche]

Thank you, but I have no intention of prolonging the nugatory discussion we had on another forum. Some of your previous comments are hard to reconcile with other normally reliable sources.

Meanwhile, if anyone can address my original question I'd be delighted to hear from them.

[Arch Stanton]

Ah, I see. And if I answer anyway are you going to report me to an admin like you did in the other forum?

[Doomer]

I wonder if anyone can tell me typically how many spare non-LBA sectors (as a propotion of the LBA size) a HDD typically has.

I just took an 8TB Seagate drive and got these results

Code:

User Partition
LBAs 000000000000-0000756080F9
PBAs 000000000000-000076893477

System Partition
LBAs 000000000000-00000013497F
PBAs 000000000000-000000146F3F

Media Cache Partition
LBAs 000074702556-0000756080F9
PBAs 0000759486D0-000076893477

Spare pool
PBAs: 00007578F548-00007586BDF5 RST Available: 8000 SCT Available: EF

Spare pool (Multi-IOEDC Region)
PBAs: 00007687B32C-0000768872C1 RST Available: 400 SCT Available: 1A

The numbers are hexadecimal, in large (4K) sectors, it means that you'd have to multiply them by 8 to convert them to "normal" LBAs
Addressable user space in LBAs is: 15628053168

Simple erasure tools are unable to clean these or the bad sectors that have been remapped to them but some government accredited tools (such as Blancco) can. Hence I want to try and put a theoretical absolute upper limit on the amount of data that might survive a simple erase.

There is no way to get LBA access to the in-addressable LBAs, no matter how much "government accreditation" a tool can get.

As a supplementary question: can I access non-LBA sectors in any way using MHDD?

no, MHDD can't access it (I'm not taking in account vendor-specific commands that theoretically can be written as scripts for MHDD)

[pleriche]

Very many thanks - that looks vey interesting. Did these figures come out of a proptietary Seagate utility, or is there a general utility that I can run on one or two of my drives?

I don't pretent to understand it all, but I assume User Partition is the user-visibale storage, System is maybe firmware etc and Cache is roughly what it says on the tin.

The spare pool would then appear to be 903,341 secors (in English), which is roughtly 0.045% of the LBA size.

At 8TB, this must be the type of drive whose whose name I've forgotten which stores data progressivlely a bit like an SSD rather than with a direct LBA to physical mapping, in which case the spare pool size may not be typical for smaller drives, nevertheless, hopefully an order of magnitude.

Thank you again - I've been trying to get this info for a while.

[Doomer]

Very many thanks - that looks vey interesting. Did these figures come out of a proptietary Seagate utility, or is there a general utility that I can run on one or two of my drives?

it's a proprietary tool

I don't pretent to understand it all, but I assume User Partition is the user-visibale storage, System is maybe firmware etc and Cache is roughly what it says on the tin.

User Partition is where user data is stored according to Seagate FW engineers (its size is 15754069960 "normal" sectors). It consists of the user-addressable LBA space (15628053168 sectors) and Media Cache partition (126016792 sectors or roughly 60GB). Media Cache almost always STORES user data inside, although it's not addressable as "normal" LBAs.

At 8TB, this must be the type of drive whose whose name I've forgotten which stores data progressivlely a bit like an SSD rather than with a direct LBA to physical mapping, in which case the spare pool size may not be typical for smaller drives, nevertheless, hopefully an order of magnitude.

This particular drive stores data as direct LBA to physical mapping.

[Arch Stanton]

How does zeroing LBA 0 to LBA max affect media cache? Is media cache potentially 60 GB of recoverable data from a zeroed drive?

[Doomer]

How does zeroing LBA 0 to LBA max affect media cache?

if we are talking about writing zeroes through the interface as a normal write LBA command then it might not affect media cache at all.

Is media cache potentially 60 GB of recoverable data from a zeroed drive?

very theoretically, yes. In reality it could be hundreds of megabytes of valid user data sitting in the media cache.
However the enhanced secure erase command would erase the media cache, as well as all the user area and the spares.

Of course if it's a SED or ISE drive then the drive could use the sanitize command to instantly make all the user data unrecoverable.

[Arch Stanton]

Code:

SED or ISE drive

This is encryption, correct? So throw away 'key' and everything is useless binary blob?

[pleriche]

How does zeroing LBA 0 to LBA max affect media cache?

if we are talking about writing zeroes through the interface as a normal write LBA command then it might not affect media cache at all.

Is media cache potentially 60 GB of recoverable data from a zeroed drive?

very theoretically, yes. In reality it could be hundreds of megabytes of valid user data sitting in the media cache.
However the enhanced secure erase command would erase the media cache, as well as all the user area and the spares.

A good question by Arch Stanton (and glad to acknowledge it). I would just add that I understand from some years back that GCHQ in the UK is of the opinion that not all hard drives properly implement the ATA secure erase function, consequently they don't sanction its use. Whether Seagate does, I haven't the slightest idea.

[Arch Stanton]

A good question by Arch Stanton (and glad to acknowledge it).

Yeah well, I am not the bad guy. All I was trying to tell you is that number of spares is of no relevance, it's number of reallocated sectors that is. 'Bad' sectors taken out of LBA pool are the ones potentially containing user data. The spare that the LBA of bad sector is mapped to is actually accessible.

Good thing out of this, we are now aware of potential security hazard that's called the media cache.

[guru]

Some drives certainly fail to erase all sectors during Security Erase.

Not a reliable solution for sanitisation, so say GCHQ & NSA

[Doomer]

Code:

SED or ISE drive

This is encryption, correct? So throw away 'key' and everything is useless binary blob?

Yes, that is correct

[Doomer]

Some drives certainly fail to erase all sectors during Security Erase.

Security Erase only erases user-addressable area. Enhanced Security Erase should erase everything, of course nobody's protected from bugs in the code.

[pleriche]

Some drives certainly fail to erase all sectors during Security Erase.

Security Erase only erases user-addressable area. Enhanced Security Erase should erase everything, of course nobody's protected from bugs in the code.

Nobody's protected from bugs in the code... but government-accredited erasure solutions are subjected to 3rd party testing which, depending on the assurance level, may include a code review. Only Seagate engineers get to review Seagate code, and seeing bugs in your own code is notoriously difficult.

Encryption: throw away 'key' and everything is useless binary blob. Provided the encryption is correctly implemented. AES is an excellent standard but a regular engineer who knows no better can use a correctly implemeted AES algorithm in a way that makes it susceptible to cryptanalysis. Or worse - some years back it was discovered that some self-encrypting memory sticks (I think Kingston) all used the same encryption key. All that happed when you entered the PIN was that the key was unlocked!

[Doomer]

Nobody's protected from bugs in the code... but government-accredited erasure solutions are subjected to 3rd party testing which, depending on the assurance level, may include a code review.

The only way to erase the in-addressable LBAs is to issue the Enhanced Secure Erase command. Which is only one command and it's very easy to implement it with no bugs. The erasure part is done by the drive itself. So the code review of a tool that just sends one command won't help to confirm that there would be no bugs in the FW of the drive.

Provided the encryption is correctly implemented.

Yes, I would only trust FIPS 140.2 certified drives in the crypto-erase part.

[Arch Stanton]

You have to go through a lot of trouble to see how for example Blancco was certified or whatever it's called, but this is for example what it was tested against: https://www.ncsc.gov.uk/files/CPA-SC_Da ... ia_2-2.pdf.

Basically says product must identify features as DCO, HPA and such. And it all comes down to data not being recoverable using using 'readily available recovery tools' and data using 'readily available specialist lab tools'. I can only hope the drives containing real secrets never leave the building only after having been 'sanitized' using Blancco as it seems to be a rather poor line of defense. If we take above Seagate as an example, Blancco (an 'accredited' product), since it can not reach the media cache itself has to rely on enhanced security erase to wipe the data. IOW, the accredit product has to rely on 'potentially buggy code'.

[Arch Stanton]

it's a proprietary tool

Is it something that requires specific hardware? Would you be willing to share with me, not to use in a professional setting but to satisfy my curiosity? BTW I am perfectly fine with the answer 'no way Jose'.