@hdd0,
I don't want to interrupt your converstion with
northwind - I'll just add some comments, all IMHO...
Not all details of the RAID configuration are clear to me - e.g. is there a spare drive (or more than one) included in your count of 14. I couldn't see that info in the logs (on a quick review). I'm also specifically
not giving advice on what you should do next - remote RAID recovery is not always easy on RAID arrays which I
do know. On those which I don't know well (like 3Ware), the risks are too high that I'll miss something important, and I don't have the (many hours of) time needed to read & understand every part of your logs. However some general principles apply, hence these comments.
Unfortunately I've seen there are several issues with this configuration and sys admin, which led to getting into this situation.
A deep analysis could be done on the log files (I've just skimmed some of them) to try to get a better understanding, but I found indications that the drive configuraton changed between mid-April (when the sick drive seemed to be connected to port 19) and early-May (sick drive is then connected to port 3), so there appears to be even more history to this!
If you decide to take the (IMHO significant) risks of DIY, then as already suggested to you, starting with cloning the failing "drive 3" (using a non-RAID controller, and preferably a non-Windows OS) is one option. There are likely to be several hundred (e.g. 477 or more) unreadable sectors on that disk, and that's assuming it doesn't fail more catastrophically during your attempts. If it doesn't fail during cloning, then at least you've frozen the situation with that drive before it gets even worse. IMHO you need to use a cloning tool which will record where the unreadable sectors originally were - you may need that info later (see below).
To reduce the risks even more of not being able to get back to this current state, then cloning all the disks is generally a good option - yes, you'd need another 14 x 2TB disks (or equivalent filesystem space). You can start to see why RAID recovery is costly.
My personal choice of DIY cloning software, after considering & accepting the risks, is GNU ddrescue for several reasons, including its control over the cloning process & retries. However it relies on having some Linux/Unix sys admin knowledge, understanding of device naming, identification of specific devices (disks), the difference between a device node and a filesystem, concepts of "mounting" etc. etc. Without that knowledge, then ddrescue can be high risk, and we've seen some people make mistakes when using it & similar software. Other cloning software exists (other members have recommended DMDE, HDClone and more in the past). Ultimately, if you decide to do DIY, you would have to choose what suits your experience, skills, budget, expectations of support etc. etc. - we can't decide that for you, as there is no "one size fits all" solution.
I can't see in the logs when or why drives 0 & 5 (as you mentioned) were "failed" by the RAID controller for what may be a spurious reason (especially if they were "failed" at the same time). However it seems to have been like that for a while, since I can't see recent indications of that event. If I'm correct (that those 2 drives were "failed" a long time ago), then the data on those disks may be of limited value, due to being "stale" i.e. not keep updated with the rest of the RAID volume. It depends how much updating has been done of the files that you want to recover most.
If your important files
have been updated since those 2 drives were "failed", that probably means your recovery options are limited to trying to use the remaining readable drives (11 of them?), plus the flaky "drive 3" or its clone, and ignoring the two drives which were "failed" some time ago (drives 0 & 5 - again, assuming that my understanding of their history is correct). However if your important files have not been changed since those 2 drives were "failed", then that may open the possibility of using the data on one or both of those disks, in combination with the others
except drive 3, to recover those files.
It is possible that a good DR company might manage to read more data from "drive 3" than you can do - which could translate into more data recovered.
At the moment, according to its logs, your RAID controller seems to be trying to rebuild the volume by reading from flaky "drive 3" (and, presumably, writing (i.e. rebuilding) onto one or more other drives - perhaps drives 0 and/or 5). It then stops the rebuild (and from your comments, prevents the user data on the RAID volume being accessible) when a read error or timeout occurs on "drive 3". An attempt at (partial) data recovery would need to prevent that RAID controller behaviour - there can be a few ways to do that (e.g. using specialist RAID recovery software) but some techniques would need the disks to all be accessible as individual drives. I have seen people try to modify a RAID configuration to achieve that, and cause yet more problems. Therefore new/different HBAs may be needed, which is more cost & risk for you, dealing with this unfamiliar situation.
Although I work with large RAID systems in my day job (which isn't DR), your situation seems to have a significant risk of you potentially losing some/all your data - and at best, it's likely that some is already unrecoverable, due to the unreadable sectors on "drive 3" which you are now relying on. If you clone "drive 3" and it has any unreadable sectors (likely), and then you read data from the (then degraded) RAID volume by using that clone, you will get corrupted data in files using those parts of the volume where it needs the data from the previously-unreadable parts of the original "drive 3". In my experience, it can sometimes be difficult and complex to try to find which LBAs in the RAID volume (and hence to later calculate which actual files) have been corrupted in that situation. This is where you (or the relevant expert) would need to know exactly which blocks on "drive 3" were originally unreadable, to perform those calculations.
I hope you can now see my concerns about the risks of a DIY approach, given the current situation. You may want to consider contacting member DR-Kiev who has helped other people to do remote RAID recovery (for a fee). I am not promising that they will accept the job, or that they can recover all your data - but they do have experience. Of course finding a suitable experienced local DR company is another option. Good luck with whatever you decide
FAQ
Search
Login
Register
HDDGURU FILES
