The short answer is Cyclical Redundancy Check or CRC is the best method for checking data written to a USB flash drive.

Many believe a checksum is the best method to verify data written to a flash drive (most popular is MD5 hashing method). We believe this is favored because it is easier to understand how the verification works, and also easier to implement. However, there are flaws in checksum verification and therefore not suitable for verification of data written to a flash drive.

What is the difference between Checksum and CRC verification? The checksum method uses addition in its math calculations to check whether all data was written correctly. CRC uses long division in its math calculations to check whether all data was written correctly. It is worth noting we are talking about binary long division, not the school-yard long division you so fondly remember.

Checksum methods will calculate the total bits in a packet of data and include that total checksum amount when the data is sent over communication lines. The receiver will then look at the packet, read the checksum value and then perform the same calculation to make sure everything adds up. If the calculation on the receiver’s end matches the value passed in the packet, all is good. The problem is a high probability that somewhere between the sender and receiver the bits of data are changed, corrupted or swapped yet still turn a correct checksum value after calculation on the receiving end.

The Example

In very simple terms suppose you have a bin of oranges and red apples going from Los Angeles to New York.

All the apples and oranges where counted first and written down on a manifest, then poured into the bin and shipped off. Along the way some of the red apples where replaced with green apples. When the bin arrived in New York, the total number of apples and oranges remained the same after final count; however the receiver never knew it was supposed to be only red apples.

Cyclical Redundancy Checking is about as straight forward as addition but using long division. The advantage with this approach is looking throughout the entire packet to verify if all the information arrived correctly, rather than depending on one lump sum number as verification. As with traditional long division, binary division works through the dividend number from the divisor number.

Since the CRC method is a bit more complicated to explain, let me first touch on the apple and oranges example (before you fall asleep or click off the page). Let’s say the same bin of apples and oranges are going from Los Angeles to New York, but this time the apples and oranges are placed on trays similar to what you think for an egg carton.

The trays are stacked nice and neat and the manifest says how many apples, color of apples, oranges and layers of trays for all the fruit. When the bin of apples and oranges are received in New York the receiver can easily check the number of layers in the bin and verify the total number of fruit as described in the manifest.

Cyclical Redundancy Check calculations are long division calculations for a packet of information but have just a slight bit of tweaking.

1. To start the calculation a divisor number is set or “given.”
2. Next, you add in Zeros that are one less than the divisor number. So for example if the divisor number is 5 digits long, then you would add four zeros at the end of the packet.
3. The calculation would then start for binary long division.
4. Once the binary long division is done there will be a remainder number.
5. Here is where the “cyclical” part comes in.
6. Now if the calculation swaps the remainder number with the four zeros mentioned above (when the calculation first started), then when the long division is ran again, there is no remainder (remainder is zero).

Using this method means all bits in a packet are examined during the binary long division process after the data packet is received. Probably the easiest way to understand this would be with a visual aid. Study the two charts below, then it will make sense.

When it comes to the reason for verification methods to be employed for data transmission it is typically done because of “noise” on the transmission lines, not because of a hacker or outside manipulation trying to tweak the data. Noise on a transmission line can be encountered for many reasons, such as poor design of the electrical part (say the printed circuit board of a flash drive) or not grounded properly (most flash drives use two layer PCB but the specification for USB is four layer CPB as a minimum) or poor quality materials used in the device.

By using dependable verification methods, the receiver can, more reliably, determine if all the data was sent and received correctly.

QuickHash and MD5 And CRC-32 Calculation Utilities

Two utilities, useful in Windows environment too, are the following:

• QuickHash: a GUI utility that let users to calculate checksum (MD5, SHA-1, SHA-256, SHA512, not CRC) of text, files, etc.
• MD5 an CRC32 Utilities: a little set of 2 utilities, command line and GUI, to calculate checksum and CRC of files.

On Heelpbook, you will find download links fo both of these utilities.

MD5 an CRC32 Utilities

Both utilities were created using Microsoft Visual C++ 6.0, and have no external dependencies (no .DLL dependencies, for example). They should run on any Windows 95 or later system.

Both utilities calculate the MD5 message digest only in "binary" mode (i.e. "md5sum -b" in Unix). This is the mode which is used in calculating the MD5 message digest accompanying downloads.

The graphical version of the utility presents a graphical interface:

And one may then either select files using drag'n'drop or the File menu. The utility will then calculate the MD5 message digest and CRC-32 of the file(s) selected. The results may be copied and pasted for printing, archiving, or other purposes.

The console mode version of the utility, which is named md5.exe, operates similarly, except it only calculates the MD5 (no CRC-32), and it will only process one file at a time. There is also a separate CRC-32 utility available.

• gfsw.exe: Windows graphical MD5 and CRC-32 utility.
• md5.exe: Windows console-mode MD5 utility.
• crc32.exe: Windows console-mode CRC-32 utility.

Downloads

To download the following utilities, you will have to be registered and logged on HeelpBook.net. It's totally free.
[wpfilebase tag="file" id="248"]
[wpfilebase tag="file" id="249"]
[wpfilebase tag="file" id="250"]