All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: Video file inexplicably became a folder (NTFS partition)
PostPosted: June 22nd, 2019, 16:03 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 293
Location: France
Not sure if this should go here or in “Forensics”, but here it goes...

On one of my HDDs, a 3TB WD30EZRX, a MP4 video file inexplicably became a folder. It may or may not be related to errors found by CHKDSK a few days ago when I first noticed the issue :
Code:
Suppression de l'enregistrement d'attribut endommag‚ (128, "") du segment d'enregistrement de fichier 639974.
Suppression de l'enregistrement d'attribut endommagé (128, "") du segment d'enregistrement de fichier 639981.
L'entrée de liste d'attributs endommagée de type de code 128 dans le fichier 640010 a été supprimée.
Suppression de l'enregistrement d'attribut endommagé (128, "") du segment d'enregistrement de fichier 640009.
L'entrée de liste d'attributs endommagée de type de code 128 dans le fichier 640012 a été supprimée.
Suppression de l'enregistrement d'attribut endommagé (128, "") du segment d'enregistrement de fichier 827301.
Suppression de l'enregistrement d'attribut endommagé (128, "") du segment d'enregistrement de fichier 827302.

The MFT record for that file (obtained from DMDE) is number 620549, so it's not one of those which were fixed. If I open the partition in R-Studio, that file also appears as a folder, but if I open it with the Hex viewer, I can see the normal contents (valid MP4 header, correct size), however if I attempt to recover it, it is extracted as a folder (a regular folder with a size of 0 byte).
Otherwise, it behaves just like a folder : I can open a command prompt in it, copy it with Robocopy, even move a file inside...
First, how is that possible ? NTFS directory entries have a distinct structure, yet there is no warning that this one is abnormal.
Then, after the above CHKDSK analysis, a new analysis found nothing abnormal, despite this clear inconsistency.
Reading some more about MFT records (this page helped the most), I found out that the distinction between a file and a folder at the filesystem level depends upon a flag at offset 0x16 in the header of a MFT record, which is set to 01 for a file, 03 for a folder — and indeed it was set to 03 in the MFT record for that problematic file. Since that file was downloaded and is still available in case of a screwup, I attempted to modify that flag in-place with WinHex (I had to determine the exact spot on the whole drive, based on the data runs for the fragmented MFT found in DMDE, since WinHex couldn't edit the MFT when accessed through the partition as a single stream of data) : after a few hiccups it did work, I could see that file listed as a file again in Windows Explorer (just to be sure, I re-downloaded it anyway on another drive : the MD5 matched). Then I ran CHKDSK again :
Code:
CHKDSK est en train de vérifier les fichiers (étape 1 sur 3)...
  830208 enregistrements de fichier traités.
La vérification des fichiers est terminée.
  2859 enregistrements de grand fichier traités.
  0 enregistrements de fichier incorrect traités.
  100 enregistrements EA traités.
  1 enregistrements d'analyse traités.
CHKDSK est en train de vérifier les index (étape 2 sur 3)...
53 % effectués. (610795 entrées d'index sur 940792 traitées)
Correction des informations incorrectes dans le segment d'enregistrement de fichier 620549. <====
  940792 entrées d'index traitées.
La vérification des index est terminée.
  0 fichiers non indexés analysés.
  0 fichiers non indéxés récupérés.
CHKDSK est en train de vérifier les descripteurs de sécurité (étape 3 sur 3)
  830208 SD/SID de fichiers traités.
La vérification des descripteurs de sécurité est terminée.
  55292 fichiers de données traités.
CHKDSK vérifie le journal USN...
  8539088 octets USN traités.
Vérification du journal USN terminée.
Windows a vérifié le système de fichiers sans trouver de problème.

And then the file was once again listed as a folder, and the 0x16 flag in the MFT record was back to 03 ! How can it be ? Where could there be a reference to that file, listed as a folder, which would explain why CHKDSK considers that it should be listed as such ? (I did the above twice with the same outcome.) And how can I fix it so that CHKDSK does no longer complain ? Could it be something in the directory entry where that file/folder is located (which is the root directory) ?

(Just for the record, the drive's SMART status is flawless, this is strictly a logical issue.)

(Attached : ZIP archive with the MFT record for that file, 1) before any fix attempt, and 2) after a first CHKDSK analysis, followed by an in-place fix attempt, then a second CHKDSK analysis.)


Attachments:
620549.zip [1.36 KiB]
Downloaded 152 times
Top
 Profile  
 
 Post subject: Re: Video file inexplicably became a folder (NTFS partition)
PostPosted: June 22nd, 2019, 21:11 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 293
Location: France
Just to make it clear : recovering the file is not the issue (as I could re-download it), I'm trying to understand what's going on, and improve my understanding of the NTFS filesystem.

Attachment:
2019_06_0101_11 - Arte - Festival We Love Green 2018 - Lomepal, Charlotte Gainsbourg, Beck, Jorja Smith, King Krule.mp4 -- explorateur.png
2019_06_0101_11 - Arte - Festival We Love Green 2018 - Lomepal, Charlotte Gainsbourg, Beck, Jorja Smith, King Krule.mp4 -- explorateur.png [ 149.45 KiB | Viewed 3158 times ]

Attachment:
2019_06_0101_11 - Arte - Festival We Love Green 2018 - Lomepal, Charlotte Gainsbourg, Beck, Jorja Smith, King Krule.mp4 -- R-Studio hexadecimal viewer.png
2019_06_0101_11 - Arte - Festival We Love Green 2018 - Lomepal, Charlotte Gainsbourg, Beck, Jorja Smith, King Krule.mp4 -- R-Studio hexadecimal viewer.png [ 96.75 KiB | Viewed 3158 times ]


Top
 Profile  
 
 Post subject: Re: Video file inexplicably became a folder (NTFS partition)
PostPosted: June 23rd, 2019, 9:00 
Offline

Joined: January 29th, 2012, 1:43
Posts: 801
Location: United States
I am not 100% sure this answer is accurate, but it should be helpful.

NTFS has journaling, I believe it can be found in the metafile $LogFile. You would need to research that on your own if you wanted to know more. I think checkdisk looks at the log(s) to make sure everything matches, and if it finds something in a state that does not match the log, it puts it in the state it thinks it should be in. Obviously in your case something got corrupt, and it thinks the file should be a folder. The solution would (should) be to fix it in the MFT like you did, extract it to an external drive (making a local copy may also work), delete the original, and empty the recycle bin. Maybe then run checkdisk so it cleans itself up, then copy the file back, and run checkdisk again to make sure it still doesn’t try to make it a folder.

The scary thing is that considering a file got its metadata corrupted, how do you know nothing else got corrupted? Maybe the proper solution is to backup all data and reinstall Windows :)

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Video file inexplicably became a folder (NTFS partition)
PostPosted: June 23rd, 2019, 12:36 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 293
Location: France
Alright, thanks for the input, it might be on the right track indeed.

Quote:
The scary thing is that considering a file got its metadata corrupted, how do you know nothing else got corrupted? Maybe the proper solution is to backup all data and reinstall Windows :)

Scary indeed... but corruption happens, doesn't it ?
It may or may not be related with the regular BSODs I've had over the past months (ever since I installed the system on a NVMe SSD it would seem, even that is uncertain), but I couldn't find the origin and the supposed cause is different almost every time (“PAGEFAULT_IN_NONPAGED_AREA”, “PFN_LIST_CORRUPT”, “BAD_POOL_HEADER”, “SYSTEM_SERVICE_EXCEPTION”...), making the diagnostic especially tricky.
This time it was different though, there was no BSOD but filesystem errors happening while the system was still operational : on June 18th at 17:13 I saved a Web page on my main storage drive (ST2000DX001 – which is in perfect state according to HD Sentinel) and it worked, but then at 17:16 (nothing special happened in between as far as I can recall), I wanted to save another file with Firefox, I got an error saying (translated) “the file or directory is damaged and unreadable”. At first I thought it was an issue with Firefox, I made a screenshot of the error message, tried to save it to the same partition : same error message. Then I copied some text and Clipx, a tool which saves clipboard entries for immediate re-use and also archives them in a log, issued a more specific warning (as it couldn't update the log file) saying that “the file or directory E:\$Mft is damaged and unreadable. Run CHKDSK” (and there are several error warnings in the Event viewer at that same time saying the same thing). Next, a bit panicked, I tried to back up important files which weren't backed up yet, since I don't do backup updates nearly frequently enough, but at some point the system became nigh unresponsive (all programs which were using the E: partition got stuck / frozen and couldn't get stopped in the task manager or with a taskkill command), I had to reboot. (Just to be clear : this possibly related issue concerned a different drive, as mentioned above, a Seagate ST2000DX001, while the issue previously mentioned in this thread happened on a secondary drive, a Western Digital WD30EZRX, but both happened the same day, or were noticed the same day.)
Then, with no other option at that point {1}, I did run CHKDSK, which spitted out a crazy amount of text, I could only find the begining in the Event viewer and the end in the command prompt window {2} – apparently it had to reconstruct the root directory, which contains more than 16000 files, so each file had to be reassociated with the new root directory. Apparently all those files are fine.
But, much bigger issue, my e-mail inbox folder got wiped, which I only realized when I opened the e-mail program {3}. I do have a backup from June 16th, and the new incoming messages should still be on the servers, but, like that MP4 file (on the WD30EZRX drive), I attempted to fix it by studying the structure of MFT records. Since I made a backup of the whole MFT before running CHKDSK (it's quite large on this partition, more than 2GB), I found the main MFT record in its former state. The indicated size was 344854528, significantly less than expected (the backup is 468061408 so it should be at least as much) ; but it does point to secondary MFT records (so apparently, for files with multiple MFT extents, the size indicated in the main MFT record is only the size of the data runs contained within that record – I'm not sure about that).
Code:
800000002000001A0000000000000000
10662000000005000000740073000000 206610 = MFT record #2123280 (offset 2174238720)
800000002000001AB16F010000000000
EB692000000001000000000000000000 2069EB = MFT record #2124267 (offset 2175249408)
800000002000001A818D010000000000
FD752000000001000000000000000000 2075FD = MFT record #2127357 (offset 2178413568)
800000002000001A51A9010000000000
7B7B2000000001000000000000000000 207B7B = MFT record #2128763 (offset 2179853312)

So I examined the last of these secondary MFT records, and with WinHex I managed to follow the list of data runs, comparing them with the backup.
Code:
41 20 86 9C C8 03 => 32 clusters starting from cluster number 63478918 or sector 507831344 => corresponds to offset 445976576 of the 20190616 backup and indeed there are 32 matching clusters or 131072 identical bytes
21 10 90 00 => 16C at +144C => 507832496S => correct (446107648-446173183)
11 10 30 => 16C at +48C => 507832880S => begining is correct (446173184) but interrupted after 2 clusters (overwritten ? "idle space" according WinHex, yet extraction of "idle space" produces an empty file)
11 10 70 => 16C at +112C => 507833776S => correct (446238720-446304255)
11 20 50 => 32C at +80C => 507834416S => correct (446304256-446435327)
11 10 60 => 16C at +96C => 507835184S => begining is correct (446435328) but interrupted after 2 clusters ("idle space" according to WinHex)
11 20 30 => 32C at +48C => 507835568S => correct (446500864-446631935)
11 10 40 => 16C at +64C => 507836080S => correct (446631936-446697471)
11 10 60 => 16C at +96C => 507836848S => begining is correct (446697472) but interrupted after 2 clusters ("idle space" according to WinHex)
11 10 60 => 16C at +96C => 507837616S => correct (446763008-446828543)
11 10 70 => 16C at +112C => 507838512S => begining is correct (446828544) but interrupted after 2 clusters ("idle space" according to WinHex)
11 20 20 => 32C at +32C => 507838768S => correct (446894080-447025151)
11 10 3D => 16C at +61C => 507839256S => begining is correct (447025152) but interrupted after 1 cluster (this time the area is marked as "free space")
11 10 30 => 16C at +48C => 507839640S => correct (447090688-447156223)
21 10 80 00 => 16C at +128C => 507840664S => correct (447156224-447221759)
11 60 50 => 96C at +80C => 507841304S => correct (447221760-447614975)
...
31 10 DB E7 00 => 16C at +59355C => 508020288S => correct (464392192-464457727)
32 E0 02 25 45 FF => 736C at -45835C => 507637608S => doesn't match ("idle space")
21 30 39 0E => 48C at +3641C => 507666736S => correct (467472384-467668991)
11 10 40 => 16C at +64C => 507667248S => the first 6 clusters match (467668992-467695103), then begining of the differences between the backup and the current file, pasted missing segment
41 50 2E 83 92 01 => 80C at +26379054C => 718699680S => seems consistent, pasted segment
41 17 A4 07 2C 0A => 23C at +170657700C => 2083961280S => seems consistent, pasted segment
41 19 0D 38 22 0B => 25C at +186791949C => 3578296872S => seems consistent, pasted segment
11 20 21 => 32C at +33C => 3578297136S => seems consistent, pasted segment
41 20 9A A7 68 E8 => 32C at -395794534C => 411940864S => seems consistent, pasted segment
11 30 40 => 48C at +64C => 411941376S => seems consistent, pasted segment
11 40 40 => 64C at +64C => 411941888S => seems consistent, pasted segment
42 80 00 C8 C8 2A 15 => 128C at +355125448C => 3252945472S => seems consistent, pasted segment
21 14 BE 01 => 20C at +446C => 3252949040S => seems consistent, pasted segment
41 1E 21 8D 6C 02 => 30C at +40668449C => 3578296632S => seems consistent, pasted segment
41 0F 8A 5A 51 EE => 15C at -296658294C => 1205030280S => corresponds to PNG file recorded 201906190033, overwritten segment, added 61440 empty bytes
41 0F DE 31 91 01 => 15C at +26292702C => 1415371896S => seems consistent, pasted segment
42 C8 00 85 5C 85 F6 => 200C at -159032187C => 143114400S => corresponds to PNG file recorded 201906200259, added 819200 empty bytes
41 0D FF A7 7A 09 => 13C at +159033343C => 1415381144S => seems consistent, pasted segment
42 3B 01 F4 88 0A 12 => 315C at +302680308C => 3836823608S => seems consistent, pasted segment
41 17 83 40 64 EC => 23C at -328974205C => 1205029968S => the 16 first clusters (65536 bytes) seem correct, then 7 clusters overwritten by PNG file recorded 201906182140, pasted the segment then wiped the foreign area (interestingly this segment is attributed to \$Extend\$Deleted instead of being considered as "free space" or "idle space")
41 0E 17 D6 2A 00 => 15C at +2807319C => 1227488520S => inconsistent, belongs to a 2018 HTML file
22 4F 01 D0 03 => 335C at +973C => also inconsistent based on the above value, stopped the analysis at this point
42 BC 00 04 72 D4 E8 => 188C at -388730364C
21 10 CC 00 => 16C at +204C
11 20 30 => 32C at +48C
41 10 E2 8C 2B 17 => 16C at +388730082C
31 10 41 B6 6B 00 => 16C at +7059009C => should be the end of the file, so not much was missing, about 2,5MB out of ~450MB

It went mostly well (I only had some trouble to identify negative values in hexadecimal), and most of the runs were indeed identical, but a few had been overwritten by recent files written after the corruption (and before I realized the issue with the inbox file, at which point of course I stopped saving new files there), and near the very end, one particular data run (7th from the end) turned out to point to a totally inconsistent area, since (according to WinHex) it belonged to a file saved in 2018, and the next runs were also inconsistent (but since each run's location is determined by its relative offset to the one before, just one error in one run could render all the rest of the file unreadable), and I have no way of finding out if the missing clusters are somewhere else in the ~730MB of free space on that 1.81TB partition. So the file could not be repaired that way – I did all this for nothing but it was definitely instructive.
At which point I went back to what I could salvage of the CHKDSK output :
Code:
L’entrée de liste d’attributs endommagée de type de code 144 dans le fichier 5 a été supprimée.
Impossible de trouver l’attribut de type 0x90, VCN bas 0x0, balise d’instance 0x100 dans le fichier 0x5.
L’entrée de liste d’attributs endommagée de type de code 160 dans le fichier 5 a été supprimée.
Impossible de trouver l’attribut de type 0xa0, VCN bas 0x0, balise d’instance 0x102 dans le fichier 0x5.
L’entrée de liste d’attributs endommagée de type de code 176 dans le fichier 5 a été supprimée.
Impossible de trouver l’attribut de type 0xb0, VCN bas 0x0, balise d’instance 0x101 dans le fichier 0x5.
Impossible de localiser l’attribut de balise d’instance 0xe8 et de référence de segment 0x5000000000005. Le type d’attribut attendu est 0x90.
Suppression de l’enregistrement d’attribut endommagé (144, $I30) du segment d’enregistrement de fichier 5.
Impossible de localiser l’attribut de balise d’instance 0xea et de référence de segment 0x5000000000005. Le type d’attribut attendu est 0xa0.
Suppression de l’enregistrement d’attribut endommagé (160, $I30) du segment d’enregistrement de fichier 5.
Impossible de localiser l’attribut de balise d’instance 0xe9 et de référence de segment 0x5000000000005. Le type d’attribut attendu est 0xb0.
Suppression de l’enregistrement d’attribut endommagé (176, $I30) du segment d’enregistrement de fichier 5.
L’enregistrement d’attribut de type 0x80 et de balise d’instance 0x0 a un lien croisé qui commence à 0x8fa698a pour 0x17 clusters éventuels.
Certains clusters occupés par l’attribut de type 0x80 et de balise d’instance 0x0 dans le fichier 0x17447f sont déjà utilisés.
L’entrée de liste d’attributs endommagée de type de code 128 dans le fichier 1524863 a été supprimée.
Impossible de localiser l’attribut de balise d’instance 0x0 et de référence de segment 0x5000000206610. Le type d’attribut attendu est 0x80.
Suppression de l’enregistrement d’attribut endommagé (128, "") du segment d’enregistrement de fichier 2123280.
Impossible de localiser l’attribut de balise d’instance 0x0 et de référence de segment 0x10000002069eb. Le type d’attribut attendu est 0x80.
Suppression de l’enregistrement d’attribut endommagé (128, "") du segment d’enregistrement de fichier 2124267.
Impossible de localiser l’attribut de balise d’instance 0x0 et de référence de segment 0x10000002075fd. Le type d’attribut attendu est 0x80.
Suppression de l’enregistrement d’attribut endommagé (128, "") du segment d’enregistrement de fichier 2127357.
Impossible de localiser l’attribut de balise d’instance 0x0 et de référence de segment 0x1000000207b7b. Le type d’attribut attendu est 0x80.
Suppression de l’enregistrement d’attribut endommagé (128, "") du segment d’enregistrement de fichier 2128763.

So, as far as I understand it, CHKDSK determined that there were indeed segments in the MFT record of that file which were already used, and “decided” to wipe it altogether (as well as all four secondary MFT records). That's scarier than even myself (and a girl I met lately wrote me that I was “just scary” – coming from someone who reads Schopenhauer in the tramway it makes ya wonder! :shock: ).


{1} Well, yes, of course doing a full clone is always wise before attempting any kind of in-place fix, but I had no empty 2TB drive, at least I did backup the whole MFT which would have made it possible to go back to the former state in case of a major CHKDSK SNAFU.
{2} I had the feeling that it would do something like this, I should have redirected the output to a text file for further examination.
{3} Which, I'm a bit ashamed to admit, is msimn.exe, AKA Outlook Express – I couldn't find a proper replacement when I moved from Windows XP to 7 so I used a trick to keep using it.


Top
 Profile  
 
 Post subject: Re: Video file inexplicably became a folder (NTFS partition)
PostPosted: June 23rd, 2019, 19:47 
Offline

Joined: January 29th, 2012, 1:43
Posts: 801
Location: United States
Just for reference, I work in the IT field, and at the first sign that checkdisk needs to be run, the drive is scanned with the computers built in hard disk test, and if it passes that, the user data is backed up (if any) and the computer is reimaged. Chasing errors after checkdisk is at best problematic. If checkdisk finds and fixes problems, it should be assumed that something could (if not likely) still be wrong. If I were to have an issue with my own system, I would take the same approach.

_________________
http://www.sdcomputingservice.com
Home of HDDSuperClone and HDDSuperTool


Top
 Profile  
 
 Post subject: Re: Video file inexplicably became a folder (NTFS partition)
PostPosted: June 24th, 2019, 23:31 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 293
Location: France
But what would it mean in this particular case ? Both drives are storage drives only, the (Windows 7) system and softwares are installed on a NVMe SSD as I mentioned (Samsung 950 Pro 256GB), and both drives had file system issues seemingly simultaneously, while both still have a flawless SMART status. There is something wrong obviously (I tried to solve the issue with those repeated system shutdowns but the pattern is so random that I came up with nothing so far), but most likely not with those drives themselves (as physical devices).
On another forum I've been told that the fact that I cloned a former Windows 7 install on a SATA SSD to a NVMe SSD could be the root of these problems, and that I was even lucky that it was working at all ; also, that Windows 7 wasn't optimized for NVMe SSDs which weren't produced when it came out. Do you agree with those statements ? Should I either go back to the SATA SSD, or move on to Windows 10 ? (Which I don't like, but then again I didn't like Windows 7 that much compared with Windows XP which I kept using at least two years past the end-of-support date, various ergonomy aspects have degraded in subsequent MS systems...)
I got a NVMe SSD in part because it was supposed to make start-up and wake-up from hibernation blazingly fast, but start-up time doesn't seem impressive, while wake-up from hibernation is excruciatingly slow, 5 minutes on average ! :shock: Likewise, I tried to solve this issue but found nothing conclusive, and sadly, like many annoying or harrowing things in life, I got used to it...


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: Google [Bot], Google Adsense [Bot] and 50 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group