Recovery 4 Layer of complexity sucks!(Raid<-vmfs<-vmdk<ext3)

[4levels]

The idiots story:

- Virtualization is great.
- Especially when you need to recover! You have more layers of complexity.....(RAID<-VMFS<-VMDK<-NativeFS) which is more fun?
- Ever tried to do that last minute thing at 2 o'clock in the morning?
- Replacing a harddisk in a raid5 is easy, you did this a 100times ...( hotspare and coldspare ready at 2'oclock right??)
- Actually replacing the wrong one is a great thing. You will find your raid not degraded....it will fail your raid!
- (of course you need be silly enough to trust the vendors disk check utility which says of course the "nonfailed wrong replaced disk" is of course fine..and after rebuilding fails you take again the wrong disk out of the array
check it again and it for sure says still it's fine and then you gonna quick lowlevel format (which destroy the first 100MB and last 100MB of the drive)
- YOU're RAID is DOWN now - GREAT JOB!
- Anything easier than that? -> you do regular secure tape backup do you? And you do not ignore data explosion warnings because live storage overpass backup storage and is not backup therefore?

Nothing of this happened to me - that's why I'm not writing on this forum as a last chance before spending 1000 of bucks for inexpensive data & disaster services....

Recovery story:

- job one --> buy replacment drives....3 1TB drives to do a clone of the "failed array" ---> Lesson learned: Never ever work on failed drives
- recover level 1: destripe a Raid 5 (3 Drive 1TB) into a 2TB single drive ---> Lesson learned It's much easier to work with single drives on recovery
- recover level 2: scanning vmfs for vmdk? what? there is no such thing!!! google? vmdk_undelete! which is only working for old esxi.......
- recover level 4: scanning with the well known tools (getdata, recovermyfiles, r-data......) --> welcome to insanity! this will only give you scrambled garbage and some jpgs to recover!!!
- job two: reading vmfs, vmdk specification? what? this is proprietary and closed source! great job!
- job 3: trying to reach out the "Net"-People: http://www.sanbarrow.com (MOA), and various tricks (like creating a similar vmfs and coping the header looked promising but did not help...)

Guys any other ideas?

VMDK file its just image container (without any compression) , its easy to recovery , you need just cut it in winhex (from boot to boot for example) and rename to *.vmdk .
Rarely but it happens that it is "fragmented" , in this situation you need use "virtual translator" in DE for example.

easy to recover?????

[DR-Kiev]

Don't panic. You did mistake somewhere.
Are you sure you managed array correctly? What controller or server was it?
Show us printscreen of your R-studio scan result.

[drc]

http://en.wikipedia.org/wiki/VMware_VMF ... ementation

[4levels]

>>Are you sure you managed array correctly?
Yes-went though that process with some dell consultant.

-->putting (mirrored) drives in again
-->place back raid settings without initialize
-->make the "real failed drive" offline
--> Destripe: copy the raid to a single drive on a non-raid-controller
(Of corse the first 100MB and last 100MB are somewhat lost where MFT and Superblocks are)
(as I low-level-format it ....which gives then something like:
(00000000-DATADATA---000000000-DATADATA in the first 50MB destriped disk)

>>What controller or server was it?
Standart Dell Server with Perc SAS 5e


To be honest: I see not much hope getting back the data.
Lesson learned: do not put important data in a virtual environment, I do not consider OS and Apps as important, but the created user data

Attachments

SCAN