Skip to content

latest_of_each() returns duplicate records when history_date timestamps collide #1602

Description

@jcrodriguezu

Describe the bug
latest_of_each() fails to deduplicate historical records when two or more records for the same object share the exact same history_date. The method uses only history_date__gt to find "later" records, so when timestamps collide neither record sees the other as strictly later — both survive the ~Exists filter and the queryset returns multiple "latest" records for the same object.

Note: We're not 100% sure this qualifies as a bug — if there's a better way to handle same-timestamp records we'd love to hear it. For now we're working around it by overriding latest_of_each() in a subclass to add history_id as a tiebreaker:

    def latest_of_each(self) -> "HistoricalRequisitionQuerySet":
        qs = super().latest_of_each()
        same_timestamp_later = qs.filter(
            Q(**{self._pk_attr: OuterRef(self._pk_attr)}),
            history_date=OuterRef("history_date"),
            history_id__gt=OuterRef("history_id"),
        )
        return qs.filter(~Exists(same_timestamp_later))

To Reproduce

import time_machine
from django.utils import timezone

# Freeze the clock so save() and a subsequent M2M set() (or any second write)
# fire within the same instant, producing two history records with identical history_date.
with time_machine.travel(timezone.now(), tick=False):
    instance = MyTrackedModel.objects.create(...)
    instance.tags.set([...])  # triggers a second history write at the same timestamp

# Both history records survive — latest_of_each() returns 2 rows instead of 1
assert MyTrackedModel.history.latest_of_each().count() == 1  # FAILS, returns 2

This also reproduces without time_machine on any database backend whose clock resolution causes two rapid saves within the same microsecond (e.g. bulk operations or same-request double-writes).

Expected behavior
latest_of_each() should return exactly one historical record per tracked object — the one most recently written. When history_date values are equal, history_id (the auto-increment PK of the history table, monotonically increasing) should be used as a tiebreaker.

Screenshots
N/A

Environment

  • OS: Linux (Docker)
  • Browser: N/A
  • Django Simple History Version: 3.8.0 – 3.11.0 (3.7.0 was not affected)
  • Django Version: 5.2
  • Database Version: PostgreSQL 16

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions