Video Fragmentation Recovery – AI-Based Carving

[xoxo12]

Dear all,

I would like to share my recent experience with a challenging video recovery case involving a 4TB exFAT drive containing Sony 10-bit video data.

After trying several professional recovery solutions, including R-Studio, UFS Explorer, and other commercial tools, I was unable to achieve a successful recovery due to severe fragmentation of the video files.

As a result, I started developing my own AI-assisted recovery tool. I am not a professional data recovery engineer, but I built a locally hosted AI-based system that combines automated analysis with custom recovery logic.

The tool works by:

[list=]-Scanning the drive sector-by-sector and creating a saved scan database to avoid repeated rescanning.
-Detecting video file signatures and fragmented video structures.
-Locating and reconstructing moov atoms manually from fragmented MOV/MP4 data.
-Patching thousands of fragmented atoms individually.
-Testing reconstructed files in real time.
-Using metadata such as creation timestamps and file structure patterns to improve reconstruction accuracy.[/list]

With this approach, I was able to recover approximately 4TB of video data within two days, which was a major breakthrough for me.

I am sharing this project with the community because I know many people face similar problems with fragmented video recovery, especially with modern camera formats and large-capacity storage devices.

If any professional recovery company, developer, or researcher is interested in collaborating, improving the approach, or helping transform this into a more complete recovery solution with proper AI-agent integration, I would be happy to contribute my code and ideas.

My goal is to help push fragmented video recovery beyond current manual carving limitations and explore new AI-assisted approaches.

If you are interested, you can contact me at:

luckystarrocky@gmail.com

Thank you, and I hope this can help someone facing a similar recovery challenge.

Attachments

[posidon]

Congratulations for developing such innovative tool.Appreciate your technical expertise.
I am interested in this tool. Sending you mail.
Thanks

[xoxo12]

Congratulations for developing such innovative tool.Appreciate your technical expertise.
I am interested in this tool. Sending you mail.
Thanks

Sure and thanks

[Arch Stanton]

Can you explain the AI part in this please? So, why'd you call it AI rather than advanced carving for example? How is AI exactly help us here, what decisions or analysis does it make?

For example I see mdat = xxGB, moov_expected nnGB etc.. This is typically what I'd call a common sense or rule of thumb style decision / conclusion. At what point does AI kick in?

IF fragmentation is the issue, why would we regenerate the moov box, logic / rule of thumb says we may assume the moov box is present and there to be found.

Thank you.

[xoxo12]

Can you explain the AI part in this please? So, why'd you call it AI rather than advanced carving for example? How is AI exactly help us here, what decisions or analysis does it make?

For example I see mdat = xxGB, moov_expected nnGB etc.. This is typically what I'd call a common sense or rule of thumb style decision / conclusion. At what point does AI kick in?

IF fragmentation is the issue, why would we regenerate the moov box, logic / rule of thumb says we may assume the moov box is present and there to be found.

Thank you.

Hi Joep,

Thanks for taking the time to comment and ask those questions.

The recovery itself is not based on "AI magically repairing videos." The core is deterministic forensic parsing and carving: scanning raw media for MP4 atoms, validating recovered structures, correlating metadata, and pairing the correct moov with the appropriate mdat wherever possible.

I also agree with your point that if fragmentation is the issue, the original moov should be searched for and validated first before considering reconstruction. Rebuilding or regenerating metadata is only considered when the original is genuinely missing, corrupted, overwritten, or cannot be matched reliably.

The AI component is there to assist with analysis, validation, and workflow automation. It helps evaluate candidate structures, compare metadata and timestamps, choose the most appropriate recovery path, interpret recovery results, and automate work that would otherwise require manually testing a large number of possible pairings (Which will take days or months to do manually for a 4TB drive)

I've sent you a much more detailed explanation by email, including replies to the specific points you raised.

Please check your inbox (from luckystarrocky@gmail.com). I'd be interested to hear your thoughts after you've had a chance to read it.

[xoxo12]

Hi Joep,

Thanks for taking the time to comment and ask those questions.

so it doesn't cover the full workflow. The recovery itself is not based on "AI magically repairing videos." The core is deterministic forensic parsing and carving: scanning raw media for MP4 atoms, validating recovered structures, correlating metadata, and pairing the correct moov with the appropriate mdat wherever possible.

I also agree with your point that if fragmentation is the issue, the original moov should be searched for and validated first before considering reconstruction. Rebuilding or regenerating metadata is only considered when the original is genuinely missing, corrupted, overwritten, or cannot be matched reliably.

The AI component is there to assist with analysis, validation, and workflow automation. It helps evaluate candidate structures, compare metadata and timestamps, choose the most appropriate recovery path, interpret recovery results, and automate work that would otherwise require manually testing a large number of possible pairings.*(which will take days or months to do manually)

I've sent you a much more detailed explanation by email, including replies to the specific points you raised.

Please check your inbox (from luckystarrocky@gmail.com). I'd be interested to hear your thoughts after you've had a chance to read it.

[xoxo12]

Can you explain the AI part in this please? So, why'd you call it AI rather than advanced carving for example? How is AI exactly help us here, what decisions or analysis does it make?

For example I see mdat = xxGB, moov_expected nnGB etc.. This is typically what I'd call a common sense or rule of thumb style decision / conclusion. At what point does AI kick in?

IF fragmentation is the issue, why would we regenerate the moov box, logic / rule of thumb says we may assume the moov box is present and there to be found.

Thank you.

Hi Joep,

Thanks for taking the time to comment and ask those questions.

The recovery itself is not based on "AI magically repairing videos." The core is deterministic forensic parsing and carving: scanning raw media for MP4 atoms, validating recovered structures, correlating metadata, and pairing the correct moov with the appropriate mdat wherever possible.

I also agree with your point that if fragmentation is the issue, the original moov should be searched for and validated first before considering reconstruction. Rebuilding or regenerating metadata is only considered when the original is genuinely missing, corrupted, overwritten, or cannot be matched reliably.

The AI component is there to assist with analysis, validation, and workflow automation. It helps evaluate candidate structures, compare metadata and timestamps, choose the most appropriate recovery path, interpret recovery results, and automate work that would otherwise require manually testing a large number of possible pairings.*(which will take days or months to do manually)

I've sent you a much more detailed explanation by email, including replies to the specific points you raised.

Please check your inbox (from luckystarrocky@gmail.com). I'd be interested to hear your thoughts after you've had a chance to read it.

[Arch Stanton]

Thank you very much for replying and your explanation!

[Arch Stanton]

I would like to share my recent experience with a challenging video recovery case involving a 4TB exFAT drive containing Sony 10-bit video data.

But .. Was it straight from the camera, of copied there from a memory card that was used in-camera?

[xoxo12]

The data was copied from the memory cards of 2–3 different Sony cameras onto the 4 TB external drive, which had been formatted and data got heavily fragmented.

[Arch Stanton]

I don't understand. I understand data is fragmented due to how the videos are written to the card. But, once you copy those to a freshly formatted hard drive, there's no reason to expect the copied videos to be fragmented as the freshly formatted HDD is ideal the prevent that from happening. And whatever fragmentation happens on the card when recording, this automatically gets resolved once we copy the files.

[HaQue]

I think maybe he is saying the cards were copied to a hard drive, and THEN it was formatted. so maybe the original filesystem wasnt so nice, and thats where the fragmentation comes from

[Arch Stanton]

I think maybe he is saying the cards were copied to a hard drive, and THEN it was formatted. so maybe the original filesystem wasnt so nice, and thats where the fragmentation comes from

Okay, that might be it.

But fragmentation in screenshot suggests a quite simple problem, file continues each time after skipping one or two 33554432 sized blocks. It's pretty much what mdat fragmentation looks like with any of those modern camera's SD Cards.

[Arch Stanton]

Ah no, it's less simple than I initially thought.

[HaQue]

I don't know enough about video formats to comment on that. Just thought maybe the 2nd language thing might come into play

"don't listen to what I say, listen to what I mean"