hard disk data refresh

[Doug Lassiter]

My question is about long term storage on hard disks. I am aware that MTBFs of hard disks is of order five years. I am aware that repeated power-up/power-down can make that number lower. What I'm curious about is "bit rot" -- where magnetic domains on the disk die away, and how to mitigate that. I've heard that the magnetic field strength in these bits drops by at least 1%/year. That is, if you hardly ever power up the disk at all, and just put it on the shelf, the data on it will eventually just die out.

Clearly, one solution is to rewrite hard disk data regularly. Copy data off the disk, erase the disk, and copy it back on. This is called "data refreshing". Takes some effort.

There is a somewhat ancient post about doing this refreshing more easily ...

https://larryjordan.com/articles/techni ... k-storage/

wherein just reading all the data is supposed to refresh it. I'm guessing this is probably total crap.

Where can I find a credible strategy for refreshing hard disk data? As in, if you want to keep data for the long term on a hard disk, here is what you should do to preserve it. (The answer is not "keep it on a lot of hard disks!". That's a sensible archival strategy, but that's not the question I'm asking.) If rewriting is the only way to do it, what is the recommended schedule?

You would think there would be a lot of wisdom about this, but my research has yet to turn up any. There are actually lots of forum questions about this on the web, but apparently no good answers. You would also think that HDD manufacturers would be called upon to provide such information by businesses that rely on HDs for data storage.

[Doug Lassiter]

Thank you. That's *exactly* what I'm trying to deal with. But is there a Mac application that does what SpinRite does? I gather there are ways to run SpinRite on a Mac, but it's pretty involved. This isn't a hard function to execute, so I'd rather not make it hard. Yeah, I could mount the disk on a PC, but that's not necessarily easy either, for an internal drive.

[Amarbir[CDR-Labs]]

Sorry, I don't know of any software for MAC that will do that...

Spildit ,
Since a few years all Macs are intel based ,Hence a Boot disc could be run from a USB port of the mac and the required app be run and get the job done .Alternatively he could buy 2 hdds of same capacity and possibly same lot [ Which can help in physical data recovery if need be anytime ] ,Then he can clone one to other and after few months clone the last cloned drive back to first and keep repeating this .I have 2 2TB hods from wdc that have not done even few hours and they already have bad sectors phewwww

[Doug Lassiter]

Yeah, a pair of hard drives can just be used to keep writing back and forth. Seems odd that to the extent disk refresh is a possible issue, there isn't an active app for Mac that will do it. There was a DiskRefresher that was once available for Mac. I think that did exactly this. It has been evidently discontinued. Um, are Mac users generally unaware of this phenomenon?

In fact, even backup software that does incremental backup -- that is, only writing things that have changed, seems a recipe for bit rot problems. You could dutifully incrementally back up for many many years, only to find that your backup was hosed because much of your backup was never rewritten. Seems smart to do a whole-disk backup every once in a while. Takes much, much longer, but is good insurance.

[Forhad Hossen]

Modern HDDs have an internal background media scan operation that works as long as the drive is powered up and is idle sometimes. The operation of the BMS is to do exactly the sort of refresh that you ask about. It will slowly scan the disk and if there are marginal spots it will try to rewrite them and if that can't be done it will ask to reallocate the data.

[Doug Lassiter]

Modern HDDs have an internal background media scan operation that works as long as the drive is powered up and is idle sometimes. The operation of the BMS is to do exactly the sort of refresh that you ask about. It will slowly scan the disk and if there are marginal spots it will try to rewrite them and if that can't be done it will ask to reallocate the data.

That's correct, though I believe that consumer grade HDDs don't have that capability. Also, a hard drive sitting on the shelf as an unpowered archive isn't going to be correcting itself.

[LarrySabo]

You could consider M-disk storage for stuff that's critical. It's supposed to last up to 1,000 years. Not cheap though, if you're talking many GB/TB of data.

[fzabkar]

Modern HDDs have an internal background media scan operation that works as long as the drive is powered up and is idle sometimes. The operation of the BMS is to do exactly the sort of refresh that you ask about. It will slowly scan the disk and if there are marginal spots it will try to rewrite them and if that can't be done it will ask to reallocate the data.

That's correct, though I believe that consumer grade HDDs don't have that capability.

They do. For example, Seagate's F3 drives have a BGMS_DISABLE_DATA_REFRESH parameter which is enabled by default but can be disabled for data recovery purposes.

http://www.hddoracle.com/viewtopic.php?f=59&t=1842&p=10843

[Doug Lassiter]

You could consider M-disk storage for stuff that's critical. It's supposed to last up to 1,000 years. Not cheap though, if you're talking many GB/TB of data.

That's correct, and I am using M-disks for archival data that needs to stay unchanged. A lot cheaper than it was originally, but still kind pricey. Far more expensive than HDDs.

[Doug Lassiter]

Modern HDDs have an internal background media scan operation that works as long as the drive is powered up and is idle sometimes. The operation of the BMS is to do exactly the sort of refresh that you ask about. It will slowly scan the disk and if there are marginal spots it will try to rewrite them and if that can't be done it will ask to reallocate the data.

That's correct, though I believe that consumer grade HDDs don't have that capability.

They do. For example, Seagate's F3 drives have a BGMS_DISABLE_DATA_REFRESH parameter which is enabled by default but can be disabled for data recovery purposes.

http://www.hddoracle.com/viewtopic.php?f=59&t=1842&p=10843

That's interesting. What exactly does the BGMS_DISABLE_DATA_REFRESH parameter on the Seagate F3 do? Does it periodically rewrite the disk? To the extent it rewrites "marginal" data, how exactly does it recognize marginal data? It's actually looking at the magnetic field strength in the bits?

[Doug Lassiter]

Whoa. It looks like I have a Seagate F3 in my main desktop machine - ST500DM002-1BD142. So you're saying that my system is bit-rot immune?

Where can I find out more about this built-in data-refresh function? How do I know if it is enabled? Does it keep a log of rewritten sectors?

[fzabkar]

This post explains various terms relating to Background Media Scan:
http://www.hddoracle.com/viewtopic.php? ... 2119#p2119

My other link explains how to view these parameters via the serial TTL console using the F command at level T. You will need to use a TTL adapter to access the terminal.

[Doug Lassiter]

OK, thanks.

But it's still a little unclear exactly what BMS does. I gather it looks around and preemptively recognizes disk regions that have problems. How exactly are these incipient problems preemptively recognized? With regard to bit rot, is BMS looking at magnetic flux levels and saying "Hmmm, this looks a little low. I'd better write over it again!"? Or is it saying "Hey, this checksum doesn't add up. I'd better fix it with an error correction code!" I gather it involves ECC tolerance, but I don't have a clue how that works.

That is, it isn't clear how BMS relates explicitly to bit rot.