Skip to content

Incorrect checksum calc for odd-sized buffers on big-endian #1044

Description

On big-endian processors, the CalculateSum() func in core/utils/checksum.h will return an incorrect value when given a buffer with an odd number of bytes. This is because the last byte of the buffer is simply being promoted from 8 to 16 bits:

  // Add remaining 8-bit to the one's complement sum
  if (odd) {
    sum64 += *reinterpret_cast<const uint8_t *>(buf16);
  }

This works on LE, but on BE, the last byte should be left-shifted by 8 bits. (Recall, per the RFC, that the last odd byte should be treated as if it's the high-order byte of a 16-bit value in which the low-order byte is zero.)

For example, consider a five-byte buffer containing:

aabb ccdd ee

On BE, the expected adds should be:

     aabb
   + ccdd
   + ee00
    ~~~~~
    26598 = 659a (folded to 16 bits)

but will actually be:

     aabb
   + ccdd
   + 00ee
    ~~~~~
    17886 = 7887 (folded)

On LE, the adds should be (and will be):

     bbaa
   + ddcc
   + 00ee
    ~~~~~
    19a64 = 9a65 (folded)

Note that the LE result when byte-swapped equals the correct result from the BE calc, as expected.

This should give the correct result on both BE and LE:

   sum64 += be16_t(uint16_t(*reinterpret_cast<const uint8_t *>(buf16)) << 8).raw_value();

I can provide a test program to simulate this fix on an LE processor.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions