Indexed Predicates Not Visible in Queries Within Same Transaction Before Commit

[kishenpateldotwork]

Summary

When creating nodes with indexed datetime predicates within a transaction, those predicates are not visible to subsequent queries using index-based filters within the same uncommitted transaction. The query returns empty results even though the data was successfully mutated in the same transaction context.

Environment

• Dgraph Version: v24.1.3
• Client Library: github.com/dgraph-io/dgo/v240
• Language: Go
• Running 1 Alpha node, 1 Zero node, 1 Ratel node

Schema

type <Element> {
    xid
}

type <st.Element> {
    dw.from
    dw.parented_by
    dw.state_of
    dw.to
}

<dw.from>: datetime @index(hour) .
<dw.parented_by>: uid @reverse .
<dw.state_of>: uid @reverse .
<dw.to>: datetime @index(hour) .
<xid>: string @index(exact) .

Reproduction Steps

1. Start a new transaction using dg.NewTxn()
2. Mutate (create) a node of type st.Element with predicates dw.from and dw.to set to valid datetime values
3. Within the same transaction (before commit), execute a query that filters on the indexed dw.from and dw.to predicates
4. Observe that the query returns empty results

Example Code

Please note that we observed this issue very sporadicly and have not been able to pin point exactly why it works some of the time and not other times.

package main

import (
    "context"
    "fmt"
    "time"

    "github.com/dgraph-io/dgo/v240"
    "github.com/dgraph-io/dgo/v240/protos/api"
    "google.golang.org/grpc"
)

func main() {
    conn, _ := grpc.Dial("localhost:9080", grpc.WithInsecure())
    dgraphClient := dgo.NewDgraphClient(api.NewDgraphClient(conn))

    ctx := context.Background()
    txn := dgraphClient.NewTxn()
    defer txn.Discard(ctx)

    now := time.Now()
    futureTime := now.Add(24 * time.Hour)

    // Create a new st.Element node with indexed datetime predicates
    createMutation := &api.Mutation{
        SetJson: []byte(fmt.Sprintf(`{
            "xid": "test-element-123",
            "dgraph.type": "st.Element",
            "dw.from": "%s",
            "dw.to": "%s"
        }`, now.Format(time.RFC3339), futureTime.Format(time.RFC3339))),
    }

    _, err := txn.Mutate(ctx, createMutation)
    if err != nil {
        panic(err)
    }

    // Query for the newly created node using indexed predicates
    // This should find the node we just created, but returns empty
    queryTime := now.Add(time.Minute).Format(time.RFC3339)
    query := fmt.Sprintf(`
    {
        Root(func: eq(xid, "test-element-123")) {
            g as ~dw.state_of
                @filter((type(st.Element)
                    AND lt(dw.from, "%s")
                    AND ge(dw.to, "%s")))
        }
        Details(func: uid(g)) {
            uid
            xid
            dw.from
            dw.to
        }
    }`, queryTime, queryTime)

    resp, err := txn.Query(ctx, query)
    if err != nil {
        panic(err)
    }

    // EXPECTED: Details should contain our newly created node
    // ACTUAL: Details is empty
    fmt.Println(string(resp.Json))

    // If we commit first, then query in a new transaction, it works:
    // txn.Commit(ctx)
    // newTxn := dgraphClient.NewTxn()
    // resp, _ = newTxn.Query(ctx, query)
    // fmt.Println(string(resp.Json)) // Now returns the node
}

Observed Query That Fails

This is one of our exact queries if it helps with investigations however please take note of the workarounds we already tried.

query q($time: string, $xid: string) {
    Root(func: eq(xid, $xid)) {
        g as ~dw.state_of
            @filter((type(st.Element)
                AND lt(dw.from, $time)
                AND ge(dw.to, $time)))
    }
    Structure(func: eq(xid, $xid))
        @recurse {
        Tag : uid
        Values : dw.state_of
            @filter(type(Element))
        f as Nodes : ~dw.parented_by
            @filter((type(st.Element)
                AND lt(dw.from, $time)
                AND ge(dw.to, $time)))
    }
    Details(func: uid(f, g)) {
        uid
        dgraph.type
        expand (_all_) {
            xid
            uid
        }
    }
}

Dgraph Logs

The following warnings appear in the Dgraph logs when the query is executed within the uncommitted transaction:

2025-07-23 23:04:35 I0724 04:04:35.496869       1 query.go:1783] Warning: reached default case in fillVars for var: f
2025-07-23 23:04:35 I0724 04:04:35.496919       1 query.go:1783] Warning: reached default case in fillVars for var: g

This indicates that variables f and g are not being populated, meaning the filters on the indexed predicates are not matching the newly created node.

Expected Behavior

Within a transaction, reads should be able to see uncommitted writes (read-your-own-writes consistency). The query filtering on dw.from and dw.to should return the node that was created earlier in the same transaction.

Actual Behavior

The query returns empty results for variables f and g, even though the node with matching predicate values was created in the same transaction.

Workarounds Tested (All Failed)

1. Changed filter operators: Tried le instead of lt, gt instead of ge - no change
2. Added time delay: Added time.Sleep() between mutation and query - no change
3. Different time precision: Tried with different time formats/precision - no change

Workarounds That Work (But Are Not Acceptable)

1. Commit transaction first: If we commit the mutation transaction and then query in a new transaction, the data is visible (defeats purpose of atomic transactions)
2. Remove indexes: If we remove @index(hour) from dw.from and dw.to predicates, the filtering works within the same transaction (but we need the indexes for production queries)

Hypothesis

This appears to be a bug in Dgraph's transaction isolation implementation. When indexes are present, it seems the index lookup does not consult the transaction's uncommitted write buffer, only the committed state. Non-indexed predicates (like xid with @index(exact)) and type filters (type(st.Element)) seem to work correctly.

Impact

This bug breaks atomic transactional workflows where:

1. A node is created/updated with indexed predicates
2. Subsequent operations in the same transaction need to query for that node using index-based filters
3. The entire operation needs to be atomic (all-or-nothing)

Additional Context

• The xid predicate with @index(exact) works correctly within transactions
• Type filtering (type(st.Element)) works correctly within transactions
• The issue specifically manifests with datetime @index(hour) predicates when used in comparison filters (lt, le, gt, ge)

[matthewmcneely]

Hey @kishenpateldotwork. Thanks for the great write up. I think I may have found the issue... the hour-granular datetime index is a lossy tokenizer. Seems that sometimes the logic by-passes the transaction cache and goes to the posting list for that type of tokenizer.

I'm trying to replicate and verify the fix, can you give me a sense of how often (5%, 1%, less?) you're seeing the issue? Also, have you tried with any other Dgraph versions?

[kishenpateldotwork]

Hello @matthewmcneely - great to hear you've identified a probable cause!

On our side, it reproduces very consistently for most queries that filter on the hour-granular datetime index within a transaction. It does seem system-dependent though: when a system is affected, it fails 100% of the time, but only ~25% of our systems appear to be affected.

We first noticed it in v24.1.3 (around July 2025) but it should be noted that that is also when we first added the hourly indices.

Hopefully that helps with any further investigations. Please let me know if you'd like any more information.

Thank you!

[kishenpateldotwork]

@matthewmcneely awesome to see a PR resolving this issue! Is there a specific docker tag we can use to pull down the fix and test or can you provide us with an estimated time for the next release? Thank you again!

[matthewmcneely]

@kishenpateldotwork We don't have a preview build of this. But we are planning on releasing Dgraph v25.2 this week.