A computer system was corrupting a user's data (due to what appeared to be a defective motherboard chipset). The corruption seemed random, intermittent, and each occurrence small (like a bit from a file being inverted).

A backup of the data existed; however, the backup was connected to the corrupting system and so too experienced similar corruption as the other data.

Now there are multiple data sets of what should be the same data, but due to the corruption (which itself was not identical for the data sets), the sets are different. The data sets were composed of the following media file types: JPEG, PDF, PPT (PowerPoint), MPEG, and some BMP (bitmap) documents.

Between the two data sets, there are files that should be the same, but are different (at least one being corrupt). For most of the corrupt media files, the corruption is not noticeable when opening or playing the documents in their typical associated applications (Adobe Reader, PowerPoint, Windows Media Player, etc.). The picture files that should be identical from both sets (but their binaries are slightly different) can be displayed and seem to be identical.

I want to purge only the corrupted files from the data sets. To first do this, it seems necessary to determine which are the corrupt files between the sets. How can this be done?

If the corruption is consistent, ie the same error bit is always hot or always cold, then compare the two files and choose the one where the error bit is not hot or not cold.

In Windows you can use the File Compare command in a DOS box:

fc /b file1 file2

I'm assuming that file size is not affected.

If you use winhex, you got an option specially to compare files.

File sizes were not affected by the corruption.

I was before using WinMerge to compare files.

I tried the FC.EXE utility in binary mode and noticed that differences were the same (but at different offsets) between corrupt sets. The differences, however, were not consistent across the entirety of the data sets.

It seems that corruption might be determinable algorithmically.

The corruption was on a system that had an nVidia nForce4 SLI chipset. The nVidia nForce4 SLI chipset was defective by design; the chipset would corrupt data similar to what I described in the initial post to this thread. nVidia eventually (took the company over a year) released a driver that would work around the corruption defect so that the corruption should not occur while the driver is functioning.

Perhaps the method used by the nVidia chipset driver with the anticorruption workaround can be determined by examination of the chipset driver to determine how nForce4 corruption should function (by its design and not necessarily intended design). That, however, would be more complex than I am qualified for.

Is the algorithm, if it exists, of nVidia nForce4 corruption already known outside of nVidia?