Drives and semaphores: What the heck is a semaphore?

[ikkaiteku]

Hey,

Ran across a product that was claiming to solve life with semaphores. That isn't exactly important, except that I have no idea what a semaphore is in the context of hard drives / ssds? I can't seem to find a wikipedia article or a white paper or anything describing semaphore usage in regards to hard drives / ssds.


Thanks for any and all help

[fzabkar]

Hey,

Ran across a product that was claiming to solve life with semaphores.

Is this a secret product?

[ikkaiteku]

I don't know, actually. I'm not sure if the information was given to me with the expectation of it being confidential, so I avoided the product name.

[Vulcan]

Thanks for any and all help

IMHO if you want comments, then you will need to supply more details about this "product" and its relevance to disk drives. A company might have decided to use the term semaphore to mean something new (in addition to its existing meanings) which is related to disk drives - but we'll never know, without more details from you...

[hddguy]

There are 'semaphore timeouts' when accessing degraded media via USB. In relation to Hard Disks I can only assume that access time is poor and timeouts are exceeded.

R-Studio often gives such errors when working on USB.

I rarely see this error as I avoid working over USB, and I am obviously using different 'products' than you are for data access / recovery.

[Doomer]

There are 'semaphore timeouts' when accessing degraded media via USB. In relation to Hard Disks I can only assume that access time is poor and timeouts are exceeded.

R-Studio often gives such errors when working on USB.

I rarely see this error as I avoid working over USB, and I am obviously using different 'products' than you are for data access / recovery.

It is Windows API error and it is not related to USB drives or any drives at all. It can happen with other IO access functions or even with programs which use threads but have no IO methods - "http://msdn.microsoft.com/en-us/library/windows/desktop/ms685129(v=vs.85).aspx"

[Vulcan]

@hddguy:

Thanks for trying to suggest what the OP might be meaning. However he/she didn't mention timeouts and, as Doomer explained, semaphores (and similar techniques) are a part of multi-threaded programming (that's one of the existing meanings I was referring to). Therefore I still can't guess what the OP was suggesting as something related to disks, which needs a product to "solve life with semaphores" I still believe the OP needs to explain their point, if they want the "help" they mentioned...

@Doomer:

[ikkaiteku]

Thanks for the responses guys, I really appreciate it. I followed up with the company in question and we are in fact under NDA, so I cannot clarify the name of the product. I did get permission to clarify on the technical details though. Also, I think Doomer has nailed what 'semaphore' is being used as in this context, which to me confirms that there is an important question to ask (at the end).

Basically, the company in question has a product designed to securely erase SSDs. There are known concerns with traditional wiping technology not actually erasing the contents of the physical cells on SSD media due to the interference of wear leveling, TRIM, etc. (Google reveals white papers, etc., and I'm sure you guys as the experts are aware of this ). This company is saying that they have solved that problem via the use of semaphores.

Their claim here is that by opening a semaphore to each logical sector and zeroing it, but never releasing the semaphore, the SSD thinks the sector is in use and cannot reuse it elsewhere (for wear-leveling or otherwise). By eventually opening semaphores to the total number of sectors the drive has, the wiping process will eventually physically write to every cell. If that holds true, then regardless of where a sector was physically located it will have been overwritten.


Logically, I can see how that would make sense and it sounds like it should work. What I don't know is, can we trust that opening a semaphore to lock every sector actually works? Doomer has shown us that it is Windows that is managing the semaphores, and they're open to logical sectors. For this to work, we have to accept that the SSD is going to assign physical sector for each logical one Windows says is in use, and that there is no caching or optimization work occurring anywhere that would disrupt or change that fact.

Thoughts? Thanks guys.


What do you guys think? Do you think this is technically sound?

[Doomer]

What do you guys think? Do you think this is technically sound?

It doesn't
Because SSD has more physical "sectors" than logical. It gives SSD ability to make remaps and do proper wear-leveling
Obvious conclusion that even if you open and hold(by semaphore) each logical sector it won't change hidden sectors which might have been used before and might have user data. Plus drive doesn't even know that some OS(Windows in this case) "holds" the sectors because SSD drive works using ATA or SCSI command protocols and there is no such thing as holding sectors in those protocols.
The only thing which would be affected by this semaphore holding is TRIM command but it can be achieved with much easier solution - by using older Windows version which doesn't support TRIM command

[Vulcan]

Do you think this is technically sound?

I also vote "no".

Further, I dislike this company's technical mumbo-jumbo of "opening a semaphore to each logical sector" - an OS semaphore is not needed for this, so the real meaning is still unclear (perhaps this is deliberate on their part, if they are trying to keep their product design "secret").

Their claim here is that by opening a semaphore to each logical sector and zeroing it, but never releasing the semaphore, the SSD thinks the sector is in use and cannot reuse it elsewhere (for wear-leveling or otherwise).

There is nothing that can be done via standard ATA or SCSI commands (no matter what their fancy "semapore" really is), which can force an SSD controller not to perform wear-leveling of already-written data, if it wants to. I also don't see how their process is any different than just writing to an SSD until it reports full (i.e. max_logical_LBAs written).

By eventually opening semaphores to the total number of sectors the drive has

They can't know max_physical_LBA using non-proprietary commands - only max_logical_LBA is reported via the interface standard commands (SCSI Read Capacity & ATA Identify Device). As Doomer says, max_physical_LBA > max_logical_LBA.

For this to work, we have to accept that the SSD is going to assign physical sector for each logical one Windows says is in use

But there are additional physical "sectors" (actually flash blocks, which are much larger than disk sectors), which are not reported to the host, where fragments of data could still be stored, but which are now in the "over provisioning" extra blocks used by the SSD controller. Windows can't directly tell the SSD controller to write to those.

It's not a perfect article, but this (and the research paper which it references) explains these issues, and I see nothing in the explanation which this company has given to you, that explains how they are able to guarantee they will overwrite all flash blocks, on any SSD, and thereby avoid the problems found by those researchers:

http://www.theregister.co.uk/2011/02/21 ... ing_peril/

[Michael.Reilly]

The only way to get at all of the sectors on the SSD is to wipe the nand or otherwise bypass the controller.

A Semaphore is a concept in threading, you can't hold a semaphore on a hard drive or any other piece of physical equipment, you can hold a semaphore on a piece of memory that represents access rights to a piece of physical equipment (such as a semaphore for the dining philosopher's puzzle, you're not actually holding forks in memory, just access rights to them.)

Hope this helps.

[Doomer]

The only way to get at all of the sectors on the SSD is to wipe the nand or otherwise bypass the controller.

I think eventually it will come to a standard and SSD vendors will implement full erase of NAND flash chips for Security Erase procedure

[Vulcan]

I think eventually it will come to a standard and SSD vendors will implement full erase of NAND flash chips for Security Erase procedure

Agreed. The enterprise SAS SSDs which I work with, currently have a proprietary command to do exactly this - equivalent to a factory initialisation of the NAND. (I also read about another approach the other day, but forgot some of the details already... )

[guru]

Enterprise SAS SSD's are rather lovely!!!

We routinely destroy SSD which sounds crazy I know :@)

[ikkaiteku]

Thanks guys, I really appreciate the responses. You've definitely poked large holes in the product claim, and I plan to follow up with the vendor to get more information / see what kind of research and testing they've done around this. I'll come back to this thread at some point and let you know what ended up happening.


But again, thanks, I appreciate the technical and explicit responses.

[rommano]

Haha, when I first saw the question (before reading all the replies), I thought this was a translation mistake. How to save lives with semaphores... The thing is drive and driver could easily be mistaken. "Driving and semaphores" (In my mothertongue - Brazilian Portuguese) has everything to to do with traffic, as the word semaphore means traffic light (in my language = semáforo; Google images will show you) — light used as a signal.
And it made all the sense to me that traffic lights can really save lives!

But I see it has nothing to to with what I initially thought. And I ended up learning a new concept for the word.

I found this on Wikipedia: http://en.wikipedia.org/wiki/Semaphore_(programming)