Skip to content

Normalize retrieval IDs for scoring and deduplication - #4140

Open
silasbrookshaha wants to merge 1 commit into
topoteretes:devfrom
silasbrookshaha:fix-retrieval-id-normalization
Open

Normalize retrieval IDs for scoring and deduplication#4140
silasbrookshaha wants to merge 1 commit into
topoteretes:devfrom
silasbrookshaha:fix-retrieval-id-normalization

Conversation

@silasbrookshaha

Copy link
Copy Markdown
Contributor

Fixes #4130

Summary

  • Add retrieval ID helpers for canonical graph/vector ID keys and value-based triplet keys.
  • Use canonical IDs when Temporal search maps vector scores back to graph events, fixing UUID-vs-string score misses.
  • Replace identity-based triplet deduplication in QueryState and query decomposition merges with value-based triplet keys.

Audit summary

  • cognee/modules/retrieval/temporal_retriever.py: fixed score_lookup writes and event reads to use normalize_id().
  • cognee/modules/retrieval/utils/query_state.py: fixed context-extension and CoT triplet merging to use triplet_key().
  • cognee/modules/retrieval/utils/query_decomposition.py: fixed decomposed-retrieval edge merging to use triplet_key().
  • Searched cognee/modules/retrieval and cognee/modules/graph for Python id() calls after the change; no remaining identity-based dedup call sites were found.

Tests

  • /private/tmp/cognee-contract-venv/bin/python -m pytest cognee/tests/unit/modules/retrieval/temporal_retriever_test.py::test_filter_top_k_events_sorts_and_limits cognee/tests/unit/modules/retrieval/temporal_retriever_test.py::test_filter_top_k_events_matches_uuid_scores_to_string_event_ids cognee/tests/unit/modules/retrieval/temporal_retriever_test.py::test_filter_top_k_events_includes_unknown_as_infinite_but_not_in_top_k cognee/tests/unit/modules/retrieval/utils/test_query_state.py cognee/tests/unit/modules/retrieval/graph_completion_decomposition_retriever_test.py::test_merge_deduplicated_edges_deduplicates_fresh_logically_equal_edges -q
  • /private/tmp/cognee-contract-venv/bin/python -m pytest cognee/tests/unit/modules/retrieval/graph_completion_retriever_context_extension_test.py::test_get_completion_context_extension_stops_early cognee/tests/unit/modules/retrieval/graph_completion_retriever_context_extension_test.py::test_get_completion_batch_queries_context_extension_stops_early -q
  • /private/tmp/cognee-contract-venv/bin/python -m pytest cognee/tests/unit/modules/retrieval/graph_completion_decomposition_retriever_test.py -q
  • /private/tmp/cognee-contract-venv/bin/python -m ruff check cognee/modules/retrieval/utils/ids.py cognee/modules/retrieval/temporal_retriever.py cognee/modules/retrieval/utils/query_state.py cognee/modules/retrieval/utils/query_decomposition.py cognee/tests/unit/modules/retrieval/temporal_retriever_test.py cognee/tests/unit/modules/retrieval/utils/test_query_state.py cognee/tests/unit/modules/retrieval/graph_completion_decomposition_retriever_test.py --output-format concise
  • /private/tmp/cognee-contract-venv/bin/python -m py_compile changed Python files
  • git diff --check

Note: I also tried running all of cognee/tests/unit/modules/retrieval/temporal_retriever_test.py locally; one existing test reaches the LLM gateway and fails in this environment because no LLM API key is configured.

@github-actions

Copy link
Copy Markdown
Contributor

Hello @silasbrookshaha, thank you for submitting a PR! We will respond as soon as possible.

Signed-off-by: silasbrookshaha <299338296+silasbrookshaha@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant