Undelete a large QCOW2 file on EXT3 file system

[sxlderek]

Hello,

I am new here. I accidentally deleted a very important .qcow2 file via the web interface. I am looking for ways to recover it. Any advice are appreciated.

I know the path of the deleted file is 'images/103/vm-103-disk-1.qcow2' and file size is exactly 200GB.

I tried extundelete

Code:

# extundelete --restore-file 'images/103/vm-103-disk-1.qcow2' /dev/mapper/pve-data WARNING: Extended attributes are not restored.
Loading filesystem metadata ... 6497 groups loaded.
Loading journal descriptors ... 25936 descriptors loaded.
Writing output to directory RECOVERED_FILES/
Unable to restore inode 33234953 (images/103/vm-103-disk-1.qcow2): No undeleted copies found in the journal.


It failed, but now I know the inode no is 33234953

Then I use ext3grep to print the inode information.

Code:

ext3grep --inode 33234953 /dev/mapper/pve-data
Running ext3grep version 0.10.1
No --ls used; implying --print.

WARNING: I don't know what EXT3_FEATURE_COMPAT_EXT_ATTR is.
Number of groups: 6497
Minimum / maximum journal block: 106398210 / 106431525
Loading journal descriptors... sorting... done
The oldest inode block that is still in the journal, appears to be from 1452683699 = Wed Jan 13 19:14:59 2016
Number of descriptors in journal: 25935; min / max sequence numbers: 17027181 / 17038553

Hex dump of inode 33234953:
0000 | a4 81 00 00 00 00 00 00 c4 9f 97 56 55 a1 97 56 | ...........VU..V
0010 | 55 a1 97 56 55 a1 97 56 00 00 00 00 00 00 00 00 | U..VU..V........
0020 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
0030 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
0040 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
0050 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
0060 | 00 00 00 00 34 2b ed e7 00 00 00 00 00 00 00 00 | ....4+..........
0070 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
0080 | 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
0090 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
00a0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
00b0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
00c0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
00d0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
00e0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
00f0 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................

Inode is Unallocated
Group: 4057
Generation Id: 3891079988
uid / gid: 0 / 0
mode: rrw-r--r--
size: 0
num of links: 0
sectors: 0 (--> 0 indirect blocks).

Inode Times:
Accessed:       1452777412 = Thu Jan 14 21:16:52 2016
File Modified:  1452777813 = Thu Jan 14 21:23:33 2016
Inode Modified: 1452777813 = Thu Jan 14 21:23:33 2016
Deletion time:  1452777813 = Thu Jan 14 21:23:33 2016

Direct Blocks: 0



It says the oldest inode still in journal is older then the time I deleted the file. I think I still have a hope to restore the file.

Then I restore the inode

Code:

ext3grep --restore-inode 33234953 /dev/mapper/pve-data


The output file only 72GB, but the deleted file was exactly 200GB.

I mounted the 72Gb file in Linux

Code:

modprobe nbd max_part=63
qemu-nbd -c /dev/nbd0 disk1.qcow2
mount /dev/nbd0p1 /mnt/image/
ls -la /mnt/image/


But it is empty.

I also tried "photorec", "UFS Explorer" and "R-studio" (trail version), they all failed.

The file is really important to me.
Can anyone give me any advice?

Thank you

[data-medics]

First off, you should really make a clone of the drive and work from that. Anything you do on the original drive runs the risk of overwriting the data you want to recover (if you didn't already do that). It may well be possible to make you a custom R-Studio search xml file that can recover the file (as long as it isn't fragmented).

Can you make a few small sample QCOW2 files the same way you created the lost one so I can look at the file format in HEX to see if there's a clear open/closing signature identifier? Then post them so I can download them. That or just grab the first few and last few kb in hex so I can look at that.

If the file type has any unique identifiers I can teach R-Studio to find the file type.

[drHDD]

If you stopped writing to volume just after deleting and file have kind of predictable info inside - there is a chance. But I don't think you can do it by your self. You need a pro with good knowledge of ext3 and there is a lot of manual work.

[sxlderek]

Thank you for your advice data-medics and drHDD.

I understand it is important to not write any data into the drive. I already cloned the drive (with ddrescue) and mount the cloned drive as read-only to recover the file.

The qcow2 file is a standard Linux KVM/QEMU virtual machine disk format. It was generated by the Proxmox VE system (www.proxmox.com).

According to the spec, the begining signature of qcow2 file is QFI and 0xfb
Unfortunately there is no ending signature.

More about qcow2 format here: http://website-humblec.rhcloud.com/unde ... corruption

I generated a new 4GB qcow2 file using the same system, below are the first and last 64 bytes in the file.

Code:

ls -l
-rw-r--r-- 1 root root 4295884800 Jan 21 17:37 vm-103-disk-1.qcow2

hexdump -C -n 64 vm-103-disk-1.qcow2
00000000  51 46 49 fb 00 00 00 03  00 00 00 00 00 00 00 00  |QFI.............|
00000010  00 00 00 00 00 00 00 10  00 00 00 01 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 08  00 00 00 00 00 03 00 00  |................|
00000030  00 00 00 00 00 01 00 00  00 00 00 01 00 00 00 00  |................|

hexdump -C -n 64 -s $(expr 4*1024*1024-64) vm-103-disk-1.qcow2
00000004  00 00 00 03 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000014  00 00 00 10 00 00 00 01  00 00 00 00 00 00 00 00  |................|
00000024  00 00 00 08 00 00 00 00  00 03 00 00 00 00 00 00  |................|
00000034  00 01 00 00 00 00 00 01  00 00 00 00 00 00 00 00  |................|


Base on the fact that the deleted file was 200GB in size and how ext3 file system was designed, the deleted file must be fragmented with many many redirected blocks.

However, the ext3grep command (see above) said the ext3 journal is older then the time I deleted the file. I think it is still possible to reconstruct all the blocks by interperting the journal. I just need a tool or a command line which can do this.

I am happy to pay if anyone can successfully recover the file.

Any further advice?

Thank you again

[drHDD]

You could put drive image to cloud and share it with us. I will try to get it back.

[sxlderek]

The disk is 1TB in size and contains important business files of my customer, I cannot put it on the cloud and share with everyone.

If anybody has good knowledge of ext3 file systems and has good confidence to recover the file. I can ship a copy of the disk to him/her, and pay for the service charge if the file is successfully recovered.

Anyway. Thank you again for your reply drHDD.