All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 43 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 18th, 2018, 21:46 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
I have a 2TB HDD, with one NTFS partition, almost full of data, which is no longer recognized by Windows (Windows 7 Pro 64bit). The drive letter appears but not the partition size, and there's a request to format it. In Windows storage manager, it appears as “RAW”. CHKDSK gives up analyzing it right away, with an error message saying that it's unable to determine the version and the state of the volume.

Yet, if I open that drive with R-Studio, the partition appears right away with its correct size (no scanning is even required), I can open it and access all the files that were there the last time I used it normally, with the whole directory tree, and the files' contents seem 100% correct as far as I can see. Likewise, if I open the whole drive with WinHex, it correctly recognizes the partition, and displays the files / folders with their correct contents. (If I open directly the “F:” partition, it says “incorrect parameter”, but then proceeds to open the filetree anyway.)

I thought that the partition table could have been corrupted somehow, ran TestDisk to try and fix it, but after opening the drive it gave me this :
Code:
    Partition       Start          End            Size in sectors
1 * DiskSecure MB   13578 105 19   13310 178 61   1920221962
Bad relative sector.
2 * Sys=74          33885 131 23   153418 150 44  1920298864
Bad relative sector.
3 * Linux Swap      14043 1 25     47914 125 15   544145418
3 * Linux Swap      14043 1 25     47914 125 15   544145418
Bad relative sector.
4 * SpeedStor       171841 203 21  171845 2 60    51637
Bad relative sector.

Yet I don't remember using that drive on a Linux system, let alone tampering with the partitioning / formatting. And then, if I run the “Quick Search”, it's analyzing the whole surface (normally it should be much quicker), without finding anything relevant. It just finished scanning and displayed this :
Attachment:
ST2000DM001 Z4Z0T7LY TestDisk 6.png
ST2000DM001 Z4Z0T7LY TestDisk 6.png [ 13.58 KiB | Viewed 29078 times ]


The drive is a ST2000DM001 – I know that these have a bad reputation here, but I don't see how this could have any bearing on what appears as a strictly logical issue, and a not-so-severe one if I can still access the files flawlessly with R-Studio / WinHex. It should also be noted that a few weeks ago that drive fell on the floor from a relatively low height while unplugged; it doesn't seem to have affected its physical integrity whatsoever : HD Sentinel displays a 100% health status, with no bad sector or other critical flaw. Yet the Ultra ATA CRC Error Count has risen from 0 to 237 since November, and I get this warning :
Quote:
Problems occurred between the communication of the disk and the host 237 times. In case of sudden system crash, reboot, blue-screen-of-death, inaccessible file(s)/folder(s), it is recommended to verify data and power cables, connections – and if possible try different cables to prevent further problems.
More information: http://www.hdsentinel.com/hard_disk_cas ... _error.php

(The drive is currently connected through a hot-swap cage, directly plugged with a SATA cable to the motherboard, but the last time I plugged it, when this issue appeared, it was connected through an external enclosure, in eSATA, so it may have been a bit flimsy.)


So what is going on here ? And how can I regain direct access to the drive's contents, short of extracting everything with R-Studio ? (Actually the contents are mostly a recovery from another drive, a 3TB one, which I still have and haven't used since, so I could still extract the files again from that one, it's just that I've done quite a lot of work to sort out the files, remove duplicates or corrupted ones, I don't want to start all over again.)


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 18th, 2018, 23:32 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Quick update : I ran a read test with HD Sentinel, stopped it after a few minutes (no error) by clicking on a block to launch the Contents Inspector module, then clicked on “Go to...”, typed the LBA of a MKV file identified by R-Studio, it displayed the same valid MKV header. But then if I click on “Detect file information for sector” it displays this :
“Error: Unable to detect partition / logical drive information. E: -4”

I analyzed the “F:” partition with MyDefrag : it does recognize the filesystem (although the partition size is unspecified), and provides seemingly valid informations for any block I hover over with the mouse pointer (correct file names, sizes, LCN values).
Defraggler on the other hand fails to recognize anything.
“Cannot perform operation. Operation will be canceled. Drive may be read only.”

Any clue ?


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 1:07 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Any... clue... ? :cry:


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 1:43 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Another update :
Earlier I made the mistake of scanning the F: partition with TestDisk, instead of the whole drive. If I scan the whole drive it does find the NTFS partition, starting at sector 64, but then if I try to list the files by typing “P” it says : “Can't open filesystem. Filesystem seems damaged.”
And yet the filesystem is not severely damaged since at least three different tools can still make perfect sense of it (R-Studio, WinHex, MyDefrag).
So what could be the culprit, and how could I fix it ?
Again, I could just extract the whole contents to another drive with R-Studio and work with that, but I'd like to understand what is going on, and attempt an in-place fix, which must be possible in a case like this.
Thanks ! :)


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 2:03 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15989
Location: Australia
Please don't write to the drive. I suspect that the enclosure may have been configured for a sector size of 4096 bytes.

Can you show us the Partitions window in DMDE? There may be a single-click fix.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 2:46 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Regarding the 4096 bytes formatting : I bought this drive (used / 2nd hand) inside a Seagate enclosure, which indeed had a special kind of formatting; it caused me some trouble at the begining, when I switched between using it inside or outside of this enclosure, but when I realized it, I formatted it outside of the enclosure and stopped using the enclosure. The other enclosures I have do not have such a trick.

I tried TestDisk again : after a “Quick Search”, it finds a partition at “0 1 1” (sector 64 if I'm not mistaken), but with a size of only 41945652 sectors / 20GB. If I run the “Deeper search”, it finds another, at “0 32 33” (that would be sector 2049 ?), with a correct size of 3907026944 sectors / 1863GB. If I list the files for that second partition, I get the expected contents. But the partition start should be at sector 64, according to R-Studio or WinHex (or 63 counting from 0), so I don't know where this other value comes from.

Attachment:
ST2000DM001 Z4Z0T7LY TestDisk 7.png
ST2000DM001 Z4Z0T7LY TestDisk 7.png [ 13.96 KiB | Viewed 29031 times ]

Attachment:
ST2000DM001 Z4Z0T7LY TestDisk 8.png
ST2000DM001 Z4Z0T7LY TestDisk 8.png [ 14.59 KiB | Viewed 29031 times ]


In WinHex, “Partition table (template)” shows this :
Attachment:
ST2000DM001 Z4Z0T7LY WinHex Partition table (template).png
ST2000DM001 Z4Z0T7LY WinHex Partition table (template).png [ 35.21 KiB | Viewed 29031 times ]

I'm not familiar with partition table analysis, but there seems to be an inconsistency : the C/H/S values should be much higher, right ? Also, they're different from those of the erroneous partition found by TestDisk : “1023 254 63” / “2610 254 63”.

And DMDE shows this, if that's the window you requested (I've used this software only once and more than a year ago, seemed powerful but tricky to use) :
Attachment:
ST2000DM001 Z4Z0T7LY DMDE Partitions.png
ST2000DM001 Z4Z0T7LY DMDE Partitions.png [ 51 KiB | Viewed 29031 times ]

Or maybe this can help too :
Attachment:
ST2000DM001 Z4Z0T7LY DMDE Partitions 2.png
ST2000DM001 Z4Z0T7LY DMDE Partitions 2.png [ 65.6 KiB | Viewed 29031 times ]


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 3:16 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15989
Location: Australia
AISI, the partition table and NTFS boot sector seem OK. I don't know where TestDisk is getting its information from. :?

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 3:53 
Offline

Joined: March 19th, 2015, 15:01
Posts: 1388
Location: isreal
clone the drive


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 8:23 
Offline

Joined: January 8th, 2008, 5:21
Posts: 925
Location: uk
Why is Dmde showing the drive as a raid volume?


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 15:06 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
@fzabkar
Quote:
AISI, the partition table and NTFS boot sector seem OK. I don't know where TestDisk is getting its information from. :?

Isn't the “Last Cyl 1023” value erroneous ? If I'm counting correctly, it corresponds to 16450560 sectors, or about 8GB...
How would you explain that some softwares (R-Studio, WinHex, MyDefrag, DMDE) deal with it fine, while others (Windows Explorer, CHKDSK, Defraggler) choke on it ?
DMDE can also display the whole file tree instantly, but reports an error for (at least) 3 MFT entries, saying “ERROR Attribute Offset” :
Attachment:
ST2000DM001 Z4Z0T7LY DMDE Root.png
ST2000DM001 Z4Z0T7LY DMDE Root.png [ 112.93 KiB | Viewed 28939 times ]

[FLASHBACK]
I've had a similar issue about a year and a half ago where a 1TB external USB HDD had its single partition no longer recognized, and CHKDSK could do nothing. In that case, R-Studio couldn't readily list the contents, although it probably could after a complete scan (I kept a few screenshots of the case but don't remember all the details) ; DMDE (that's the one time I've used it extensively so far) could list the contents after a full scan ; but in that case WinHex couldn't : it reported “unexpected data” where the MFT should have been and wasn't able to proceed further. When opening the whole drive with WinHex, I noticed that the MFT appeared to be shifted by 1 sector, relative to the value indicated by R-Studio (it was supposed to start at sector 6291456 + 2048 [partition offset] yet the actual begining was located at sector 6293505 – I still have no idea how this could happen spontaneously). So I first extracted the whole contents to another drive with DMDE, made a backup of the first 5GB with WinHex, then attempted to shift the whole MFT up by 1 sector ; then I ran CHKDSK again : it failed as well (said that some critical files in the MFT were damaged). Then I restored the backed-up first 5GB, and this time attempted to just copy the first sector of the MFT (which points to the MFT itself) onto the sector before, which was supposed to be the actual start of the MFT ; then I ran CHKDSK again : and it worked ! It was able to proceed, it did fix the filesystem, and the whole contents were accessible again. Then I thoroughly compared with WinMerge the in-place contents with the contents extracted with DMDE, everything matched.
[/FLASHBACK]

In this case (the current issue with the 2TB HDD), when examined with WinHex, the MFT seems to be at the right place (6291456 + 63), but what should be MFT records for $MFTMirr, $LogFile and $Volume (respectively MFT records 1, 2, 3 – just the ones for which DMDE reports an error) are filled with “FF” values (“ÿ” in ANSI), there are 3072 “FF” bytes, then MFT record 4 is fine ; MFTMirr (located at sector 16 relative to the partition's start) looks just the same (first record fine, then three corrupted). What does it mean, how could it happen, and how can it be fixed “cleanly” ? How come CHKDSK can't deal with just three corrupted MFT records ? Perhaps it precisely needs to access those three particular system files and can't, since they're no longer correctly referenced ? How can WinHex / R-Studio / DMDE locate those files and display their metadata accurately without the correct data from the MFT ? What is likely to happen if I just copy MFT records 1-3 from another drive ? (Some values won't match but maybe CHKDSK will be able to fix them... it's not that “clean” though, I'd prefer to know exactly what I'm doing ! :) )

Side question : if I remember correctly, having the partition start at sector 63 is the default value when a drive is formatted with Windows XP, and means that the partition is not properly “aligned”, relative to 4096 bytes clusters ; could it be a problem with a 2TB HDD, or is it relevant only for SSDs ?


@Spildit
Spildit wrote:
jermy wrote:
clone the drive


AGREEE !!! DO IT NOW !!!

And stop messing with the original one ...

We don't even know if the drive is ok ...

You should check S.M.A.R.T., clone the drive with hddsuperclone and then run a full MHDD/VITORIA scan on the surface !

Well, I already checked S.M.A.R.T. with HD Sentinel (first thing I did after the fall). If there were any problem, wouldn't HD Sentinel report it ?
What is the advantage in testing with MHDD/Victoria instead of HD Sentinel ?
Cloning seems overkill in this particular case : I already tried extracting a few files with R-Studio (which also has a SMART analysis pannel and reports the drive as “GOOD”), it worked flawlessly, I could just extract all files like this.
Regarding the fall : I have a 2.5" Samsung drive which once literally flew across the room (I had forgotten that it was on top of a bunch of papers which I removed hastily during a phone call or something like that...), and fell on hard floor from about one meter in height, while unplugged, and is still working flawlessly, probably more than two years later (I'm using it as a backup for my brother's computer and external HDD). I'm not saying that such a fall can never affect the physical integrity of a HDD, but apparently it can happen with no dire consequence. But if things go wrong, I'll have been duly warned ! :wink:

@dick
Quote:
Why is Dmde showing the drive as a raid volume?

Not sure, but all the others are as well, even though none of them are actually configured as RAID. Must be related to the motherboard's storage configuration. It's an Asus Maximus Hero VIII.


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 16:37 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Update :
Actually, R-Studio does not locate $MFTMirr, $LogFile and $Volume (they don't appear in the “Metafiles” folder, where they should be, I verified by opening another NTFS partition). WinHex does locate them, but using a volume snapshot made about a year ago, when the partition was still accessible (if I take a new volume snapshot they will probably disappear there as well) ; so at least, if required, I can get the correct offset values for those three files, their correct timestamps (which are exactly the same as those of the other system files), and their correct sizes if they haven't changed since then ($MFTMirr is always 4KB, $Volume is resident in the MFT, $LogFile doesn't appear to have changed in size as there is a valid directory right after its supposed boundary).
DMDE in the “$MetaData” directory displays three files with unknown names and attributes :
Attachment:
ST2000DM001 Z4Z0T7LY DMDE MetaData.png
ST2000DM001 Z4Z0T7LY DMDE MetaData.png [ 94.22 KiB | Viewed 28922 times ]

Now, how can I fix / recreate those three corrupted MFT records ?


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 16:51 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
@Spildit
Quote:
If you don't want to clone the drive you should extract the files that you do need right away with R-Studio as your drive most likely will die very shortly. Just retrieve all the files that you can.

Cloning should be the best option but extracting the files is an option as well.

Do NOT try to fix the drive partition, etc UNLESS YOU DO HAVE A CLONE.

If you do mess up you might loose access to the file allocation table and you will end up having to do a raw recovery.

Also be aware that the drive might die at any moment leaving you without any data at all.

Well, I appreciate your concern, and I will extract the files before attempting an in-place fix – I'm gonna do that right away, promis juré craché ! :shock: ; but why are you so adamant about an impending hardware failure, when there's no sign of this whatsoever right now ? (Well, except for the fact that this particular range of drives appears to be doomed right from the begining... but aren't we all ? :) )
And in a case where a drive can indeed die at any moment, if there's a way to access the files directly and at a regular rate, I think that it's a better strategy to extract them by order of importance, as opposed to attempting a full clone, which might provide nothing relevant in the end.


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 17:21 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15989
Location: Australia
CHS mode assigns 10 bits for the cylinder number, resulting in a limit of 1024 cylinders (0 - 1023). In CHS mode, sector numbers count from 1, not 0 (seems crazy, but that's the way it is).

This site should tell you everything you want to know about partition tables:

MBR/EBR Partition Tables:
http://thestarman.pcministry.com/asm/mbr/PartTables.htm

Misalignment of the partition to the physical sector size results in a performance penalty during writing (due to the additional rotation required for a read-modify-write cycle). There should be no other anomaly. In fact Seagate touts SmartAlign to transparently address misalignments in legacy OS-es.

Advanced Format Technology Brief:
https://www.hgst.com/sites/default/files/resources/AFtechbrief.pdf

SmartAlign:
https://www.seagate.com/docs/pdf/whitepaper/mb_smartalign_technology_faq.pdf
https://www.seagate.com/docs/pdf/whitepaper/tp615_smartalign_for_af_4k.pdf

MHDD is a DOS based program, so its timing results will not be affected by background processes which compete for the CPU's attention in a multitasking OS.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 17:41 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15989
Location: Australia
@abolibibelot, just in case you had any doubts, I have just tested an NTFS volume and confirmed that the first cluster of the $MFT is byte-for-byte identical to the first cluster of $MFTMirr.

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 19th, 2018, 18:14 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
So it's extracting right now... onto a 3TB WD30EZRX with 23 unstable sectors... all I have available right now... (Normally the bad sectors are all located within the same video large file, which I purposefully let on the drive and isolated in a special folder so that I wouldn't be tempted to access it later on, so it should be stable for now... if those 2TB worth of files are copied without a hiccup that'll prove it, if not, it will prove to be a worse junk than the dreaded Seagate !)
Extraction rate above 100MB/s, about 5 hours to go, no warning whatsoever from R-Studio or HD Sentinel.

@Spildit
Quote:
You stated this :

I also stated very specific observations about the current state of the filesystem, which are most likely not related to a hardware failure (very unlikely that bad sectors would have affected the same three MFT records on both the MFT and the MFT mirror).

Quote:
For me the drive does have bad sectors/bad blocks that did affect the file allocation tables.

Again, if that were the case, it would appear in the S.M.A.R.T. report. Even if you consider the drives unreliable, don't you at least rely on what S.M.A.R.T. says, even for Seagate drives ?

Quote:
If this were to be my drive i would :

- Backup firmware.
- Patch sysfile 93.
- Use hardware based imaging/cloning tools and build map based on file structure.
- Select the needed / important files.
- Image.
- Select the rest of the files.
- Image.
(as alternative select the rest of the drive space).

I currently have no professional tool which can selectively extract data areas according to a pre-defined map based on the file structure (I only know how to extract the MFT first with ddr_utilities and ddrescue). I don't have your experience with firmware issues (nor do I have the required tools, again), but if there was anything wrong with the firmware, no regular software could have accessed the partition and listed the contents. So, with all due respect, although it is certainly a wise course of action for the apparently common hardware / firmware failures this range of drives seems to be plagued with, I don't think that it is relevant in this particular case.


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 20th, 2018, 2:04 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
@fzabkar
Quote:
CHS mode assigns 10 bits for the cylinder number, resulting in a limit of 1024 cylinders (0 - 1023). In CHS mode, sector numbers count from 1, not 0 (seems crazy, but that's the way it is).
This site should tell you everything you want to know about partition tables:
MBR/EBR Partition Tables:
http://thestarman.pcministry.com/asm/mbr/PartTables.htm

I had already read articles on the subject, but it's hard to register without concrete examples ; this one is clear and thorough, definitely helpful. I'm begining to have a better grasp of it, quite confusing indeed...

Quote:
I have just tested an NTFS volume and confirmed that the first cluster of the $MFT is byte-for-byte identical to the first cluster of $MFTMirr.

Yes, that's its purpose, from what I understand $MFTMirr (for “mirror”) is just a backup of the first cluster of the actual $MFT (and only the first – I've read repeatedly that there was a “secondary” MFT or a “backup” MFT on a NTFS partition, meaning a complete copy, which is not true), and it's supposed to allow those crucial first four MFT records to be fixed in case they get corrupted. But in this case, three of them have somehow been corrupted in both the MFT and its (very partial) mirror simultaneously.
Again, is there a simple way to recreate them ? Or to “retro-engineer” them by analysing valid corresponding records from other partitions, and replacing some values to reflect the actual sizes and timestamps of each file on this partition ? Or is this a case where you would just extract the files, re-format, and re-transfer the files without thinking twice about the intricacies of the filesystem ?
Now all the files have been extracted with no issue, so there's no risk of losing anything, I'd like to fix the partition in-place if possible, if only for the sheer challenge of it (even though I might end up spending more time figuring it out than I would to get rid of the issue with the basic method above !).
The most problematic one is probably $Volume, as it is resident, meaning that not only the record but the actual contents of the file have been lost, and it apparently contains a unique identificator representing each partition (and also the partition name but that's not important as it can't be left empty and changed later on). I don't know if that ID can be randomly defined, if it has to conform to a specific pattern, and so on. It is probably because of the absence of that particular file that the partition can't be accessed.


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 20th, 2018, 2:34 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Here are two valid $Volume MFT records from other NTFS partitions, and the corrupted one :
Attachment:
MFT $Volume SSD Sandisk partition “TEMP”.png
MFT $Volume SSD Sandisk partition “TEMP”.png [ 136.88 KiB | Viewed 16852 times ]

Attachment:
MFT $Volume WD20EARX-WCAZAJ971138.png
MFT $Volume WD20EARX-WCAZAJ971138.png [ 140.69 KiB | Viewed 16852 times ]

Attachment:
MFT $Volume ST2000DM001-1ER164.png
MFT $Volume ST2000DM001-1ER164.png [ 120.19 KiB | Viewed 16852 times ]


Again, can I rebuild the third one by assembling static bits from the two others and replacing the changing values with the correct informations about each file ?
Should the Volume ID be exactly the same, or can it be randomly defined, so long as it doesn't conflict with another ? Does the system store those IDs somewhere, even for the drives which are not currently connected ?


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 20th, 2018, 18:25 
Offline
User avatar

Joined: September 8th, 2009, 18:21
Posts: 15989
Location: Australia
@abolibibelot, ISTM that you should take this opportunity to learn about NTFS metafiles. I could watch you and learn something as well. :-)

This seems to be a good resource:
http://dubeyko.com/development/FileSystems/NTFS/ntfsdoc.pdf

If you can find it, here is another good resource:

NTFS Forensics: A Programmers View of Raw Filesystem Data Extraction (Jason Medeiros, Grayscale Research 2008).

BTW, DMDE can display all the metadata attributes, eg ...

Code:
LBA:70021380           vol.sec:6291462 Clus:786432 sec:6 (MFT 3)
[-] File #3 ======== (3) ====        ==================
     magic ("FILE"):            FILE
     fixups offset:               30h
     fixups count:                 3
     LSNlo:                 022F0095h
     LSNHi:                 00000000h
     seq. number:                  3
     hlink number:                 1
     attrs offset:                38h
     flags:                        1h
     used size:                  1F0h      496
     record size:                400h     1024
     basefileref:                  0
     0x24:                         0h
     basefileref seq.:             0
     next attribute #:             7
     0x2A:                         0h
     file #:                       3
     fixup:                     0045h
[-] #0              $STANDARD_INFORMATION
       Attr. type:                      10h
       Attr. length:                    48h         72
       Non-resident:                     0
       Attrname len:                     0
       Attrname ofs:                    18h
       Flags:                            0h  -- --
       Attr. number:                     0
       Data Size:                       30h         48
       Data Offset:                     18h
              created:          2015-05-26 08:59:04.501
              modified:         2015-05-26 08:59:04.501
              changed:          2015-05-26 08:59:04.501
              accessed:         2015-05-26 08:59:04.501
              attrs:            -HS------------- ----------------
              Max versions:     0
              Version:          0
              Class Id:         0
[-] #1              $FILE_NAME
       Attr. type:                      30h
       Attr. length:                    68h        104
       Non-resident:                     0
       Attrname len:                     0
       Attrname ofs:                    18h
       Flags:                            0h  -- --
       Attr. number:                     1
       Data Size:                       50h         80
       Data Offset:                     18h
              directory:        5           (5    )
              created:          2015-05-26 08:59:04.501
              modified:         2015-05-26 08:59:04.501
              changed:          2015-05-26 08:59:04.501
              accessed:         2015-05-26 08:59:04.501
              allocated:        0
              size:             0
              attrs:            -HS------------- ----------------
              reparse:          0
              name len:         7
              posix:            3
              name:             $Volume
[-] #6              Other Attribute
       Attr. type:                      40h
       Attr. length:                    28h         40
       Non-resident:                     0
       Attrname len:                     0
       Attrname ofs:                     0h
       Flags:                            0h  -- --
       Attr. number:                     6
       Data Size:                       10h         16
       Data Offset:                     18h
       Hex:
       000: DE 28 17 04 10 10 A5 4B B9 35 E0 07 C6 36 F3 9E  Þ(....Â¥K¹5à.Æ6óž
[-] #2              $SECURITY_DESCRIPTOR
       Attr. type:                      50h
       Attr. length:                    80h        128
       Non-resident:                     0
       Attrname len:                     0
       Attrname ofs:                    18h
       Flags:                            0h  -- --
       Attr. number:                     2
       Data Size:                       68h        104
       Data Offset:                     18h
       Hex:
       000: 01 00 04 80 48 00 00 00 58 00 00 00 00 00 00 00  ...€H...X.......
       010: 14 00 00 00 02 00 34 00 02 00 00 00 00 00 14 00  ......4.........
       020: 9F 01 12 00 01 01 00 00 00 00 00 05 12 00 00 00  Ÿ...............
       030: 00 00 18 00 9F 01 12 00 01 02 00 00 00 00 00 05  ....Ÿ...........
       040: 20 00 00 00 20 02 00 00 01 02 00 00 00 00 00 05   ... ...........
       050: 20 00 00 00 20 02 00 00 01 02 00 00 00 00 00 05   ... ...........
       060: 20 00 00 00 20 02 00 00                           ... ...
[-] #4              $VOLUME_NAME
       Attr. type:                      60h
       Attr. length:                    18h         24
       Non-resident:                     0
       Attrname len:                     0
       Attrname ofs:                    18h
       Flags:                            0h  -- --
       Attr. number:                     4
       Data Size:                        0h          0
       Data Offset:                     18h
              Volume Name:      ?
[-] #5              $VOLUME_INFORMATION
       Attr. type:                      70h
       Attr. length:                    28h         40
       Non-resident:                     0
       Attrname len:                     0
       Attrname ofs:                    18h
       Flags:                            0h  -- --
       Attr. number:                     5
       Data Size:                        Ch         12
       Data Offset:                     18h
       Hex:
       000: 00 00 00 00 00 00 00 00 03 01 00 00 00 00 00 00  ............
[-] #3              $DATA
       Attr. type:                      80h
       Attr. length:                    18h         24
       Non-resident:                     0
       Attrname len:                     0
       Attrname ofs:                    18h
       Flags:                            0h  -- --
       Attr. number:                     3
       Data Size:                        0h          0
       Data Offset:                     18h
           FFFFFFFFh End Mark

_________________
A backup a day keeps DR away.


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 20th, 2018, 19:41 
Offline

Joined: December 8th, 2010, 11:37
Posts: 738
Location: Ottawa, Canada
fzabkar wrote:
If you can find it, here is another good resource:
NTFS Forensics: A Programmers View of Raw Filesystem Data Extraction (Jason Medeiros, Grayscale Research 2008).


Here's one source for it: http://citeseerx.ist.psu.edu/viewdoc/su ... 1.169.1973

_________________
Sabo Computer Repairs & Data Recovery


Top
 Profile  
 
 Post subject: Re: Weird logical issue (“RAW” NTFS partition)
PostPosted: June 21st, 2018, 4:06 
Offline

Joined: November 22nd, 2017, 21:47
Posts: 309
Location: France
Quote:
ISTM that you should take this opportunity to learn about NTFS metafiles. I could watch you and learn something as well. :-)

That's what I'm trying to do indeed...

Quote:

Wow, that one is thorough... Very good starting point.

Quote:
BTW, DMDE can display all the metadata attributes

It might be helpful to decipher the structure of a MFT record. What I want to do is the opposite, put the right hex values to the right fields, based on the informations I have. All I can recognize for sure so far are the timestamps (which are translated by WinHex when hovered over) ; also, for the $Volume file, the partition's name obviously, and what seems to be the partition's ID. I can't yet locate the field for the file size or the allocated clusters. With the precise template provided in the document you linked I may be able to find the others important fields. File size should be part of the $FILE_NAME attribute, right after the timestamps, but I find nothing there. I tried to convert the size of the MFT to hexadecimal, and search those values in MFT record #0, to no avail...
No, apparently the size information is in the pink area called “Data” in WinHex, but that doesn't correspond to what is called the “$DATA attribute” in the PDF, that's confusing... But it's there anyway : for a partition which has a MFT with a size of 625999872 bytes, in MFT record #0 there are two fields with 00 00 50 25, it's written in “little endian”, and 25500000 (hex) = 625999872 (dec).

Other things I know :
– $MFTMirr : has always a size of 4096 bytes, is apparently always located at sector 16
– $LogFile : has a size of 67108864 bytes (according to WinHex, and it's the same size on other partitions), is located at cluster 5910056 (according to WinHex with the old volume snapshot – it would have been tricky to find it otherwise)
– $Volume : is resident in the MFT
– The timestamps are the same for all system files so they can be copied from MFT record 0 ; there are two sets, one in StdInfo (colored in orange by WinHex), one in FileName (colored in blue by WinHex) ; looking at the MFT records of regular files, it appears that the actual file's timestamps are the first ones, in StdInfo, I don't know what the other set is about, apparently for most files the four timestamps are identical in that second set (perhaps they're updated each time the name itself is changed ?).
– Between two valid $MFTMirr MFT records, only the timestamps and two other bytes are different (those two bytes are identical, I don't know what they are yet, apparently it's called “fixup” in DMDE).
– Between two valid $LogFile MFT records, two other values are different, which apparently correspond to the first cluster's LBA (DE 7C 0B = B7CDE = 752862 or sector 6022896 / C5 45 0B = B45C5 = 738757 or sector 5910056 => same value as on the partition I'm trying to fix).
– Between two valid $Volume MFT records, the volume's name and ID are different, plus the “fixup” byte, and a few other bytes at the begining ; in DMDE I can see that the first field which varies is “LSNlo”, which must be “Logical Sector Number low”, for a 2TB partition it's 53 58 00 1D, or 1D005853, 486561875 in decimal, and for a 100GB partition it's 67 C1 7D 2F, or 2F7DC167, 796770663 in decimal... I can't see what to make of it... In the PDF there's no mention of “LSNlo”, but “LSN” in fact stands for “Logical Sequence Number” or “LogFile Sequence Number”. Even the document's author doesn't seem to know what this is exactly : “The purpose of the various LSNs is unclear. It appears that the data around offset 3C deal with the clear/dirty state of the volume, too.” The other varying field below seems to be the “used size”, apparently the portion of the MFT record which is actually in use.
– Regarding the volume's ID, I remember being in a situation where I cloned a drive to another with ddrescue, plugged both at the same time on Windows, and there was a conflict, I found out that it was due to the fact that both volumes had the exact same ID, and found a command to change it, don't remember the details, I must have kept some information about the procedure but can't find it in the huge mess of my main HDD... but the bottom line is : if it can be changed, it's probably not necessary to put the original one. I guess that it has to match some kind of pattern though... Perhaps I can just copy the ID from another partition and change one byte to avoid a conflict ?

That's all I could get so far... :? Tomorrow I'm gonna make a backup of the first few GB, then try to apply this with some “educated guesses” to fill the blanks, and see how it goes...


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 43 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 67 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group