HP ML350, I200 RAID card, Hot plug drives keep rebuilding

[LAITGUY]

Hi Everyone, and Happy Holidays!
I have a HP ML350 w/i200 raid card with (4) 146GB hot plug drives as a RAID 5, with #4 configured as a spare.

I see on Monday drive 3 did a full rebuild, once completed all error messages went away, thought all was good. This morning the ACU utility says drive in bay "14" is rebuilding now. Do you guys think I just have a bad hard drive that keeps rebuilding or possibly a bad RAID card. I did replace the battery on the card about 6 months ago if that matters.
Thank you in advance for any input as I want to do the right thing.

[drc]

I want to do the right thing.

Start by pulling everything and making images of all drives (or giving to someone who can)

Anytime you are messing with RAID config (especially rebuilding etc) you are a couple of button presses away from losing all of your data

[warnerr]

suggest an image of array while "good" asap if not already done. Might want to check each drives SMART and see if a drive is beginning to realocate sectors and replace it.

[lcoughey]

Almost guaranteed that one of the other drives has bad sectors. You definitely need to image all four drives. If you have a non-RAID controller that you can connect these drives to (one at a time or all 4 at once), along with a blank 1TB SATA drive and boot from my remote support CD, I can get the images and hopefully recover your files remotely.

[LAITGUY]

thanks everyone for the input. I called HP support and emailed them the diagnostics report generated from the Array card. They saw r/w errors on drive 3, the one that I caught rebuilding itself on Monday, and again this morning. I ordered a replacement drive and will be here tomorrow. At the moment, if you looked at the raid, you would think everything is fine. All drives reporting back SMART as ok, and the online spare with an ok status. I am guessing you all would agree to just go ahead and swap out drive 3 when the new one come in tomorrow.

[lcoughey]

thanks everyone for the input. I called HP support and emailed them the diagnostics report generated from the Array card. They saw r/w errors on drive 3, the one that I caught rebuilding itself on Monday, and again this morning. I ordered a replacement drive and will be here tomorrow. At the moment, if you looked at the raid, you would think everything is fine. All drives reporting back SMART as ok, and the online spare with an ok status. I am guessing you all would agree to just go ahead and swap out drive 3 when the new one come in tomorrow.

No...I don't agree. Before you rebuild you had better make sure that you have a full backup of all the data. The best thing to do is to get a full clone of each drive. Once you plug in a drive to rebuild, any damage caused by the rebuild cannot be undone...and yes, support isn't always right.

[quasimodo]

thanks everyone for the input. I called HP support and emailed them the diagnostics report generated from the Array card. They saw r/w errors on drive 3, the one that I caught rebuilding itself on Monday, and again this morning. I ordered a replacement drive and will be here tomorrow. At the moment, if you looked at the raid, you would think everything is fine. All drives reporting back SMART as ok, and the online spare with an ok status. I am guessing you all would agree to just go ahead and swap out drive 3 when the new one come in tomorrow.

No...I don't agree. Before you rebuild you had better make sure that you have a full backup of all the data. The best thing to do is to get a full clone of each drive. Once you plug in a drive to rebuild, any damage caused by the rebuild cannot be undone...and yes, support isn't always right.

Yup, have seen plenty of times where HP and Dell support have assisted in making the problem worst.

[LAITGUY]

I have a few copies of the data. Since the drives are in a RAID 5, what do you recommend to clone the drives?

[sathyan]

HDD raw copy , easy to use and free tool.
http://hddguru.com/software/HDD-Raw-Copy-Tool/

ddrescue, for users familiar with linux.

Create an image of each individual disk

[LAITGUY]

great, I needed something like that. Do you guys use a computer with multiple HD interfaces, or is there a hardware product that is cost effective that also has the interfaces needed to clone an ulta scsi to a sata or usb drive?

Learning a lot from this site and everyone's input so thank you in advance for taking the time to educate people like me that may become complacent in area's of technology that really needs more attention. There are only so many hours in the day, but it all starts with the data, and with no data there really is nothing. I am taking the precautions you all mention much more serious and thinking about what tools I need in my "bag".

[warnerr]

Since array rebuilds 'fine' I recommended imaging working array- products suct as Acronis (Server) or Paragon (server) or others that are already able to image drives should already be in place. I assumed DEVELOPING bad sectors to be the cause- thats why BACK UP IMMEDIATLY. Eventually that drive will die. I am more comfortable having a good image to go back to. Individule images are of graet use- but as you will introduce a new drive to replace failing drive I AND DRIVES ARE TEMPORARALY GOOD I suggest full image over individual images in this case. With image I am comfortable introducing new drive to replace failing drive. Check dates on drives- likely over three years- consider walking fresh ones in- todays drives are fast!

[BlackST]

Without better options I would do the same : put it in 'working order' then image then change the array and image back. Or image, change 1 drive at a time and rebuild - takes some time but it is automatical . And YES, if it is 24/7 operational the reasonable life expectancy of the drives should be 3 years or so - you can keep them for 'emergency' if they are still OK but have them checked thoroughly.

[warnerr]

Did a simular job last month to your situation. Since drives were only 80 gb ea I used four solid state 148 gb drives- the machines speed is through the roof! Client is quite pleased.

[BlackST]

Did a simular job last month to your situation. Since drives were only 80 gb ea I used four solid state 148 gb drives- the machines speed is through the roof! Client is quite pleased.

VERY BAD IDEA in my opinion. Depending on situation, you'll know why sooner than you think (SSDs on server.... 2003/2008 maybe.... )