Skip to content

add UUID as new entry value in the add_entries processor#6653

Open
Zhangxunmt wants to merge 1 commit intoopensearch-project:mainfrom
Zhangxunmt:feature/add_uniqueId
Open

add UUID as new entry value in the add_entries processor#6653
Zhangxunmt wants to merge 1 commit intoopensearch-project:mainfrom
Zhangxunmt:feature/add_uniqueId

Conversation

@Zhangxunmt
Copy link
Contributor

Description

Added a new entry in the add_entries processor that generates a unique ID for each record. This is needed for cases where the original source data does not contain a unique identifier. The unique ID is essential for running asynchronous batch inference jobs, as it is used to match and merge the inference results back with the source data.

UUID.randomUUID() (UUID v4) uses Java's SecureRandom, and collision probability is so low it has never been observed in practice in any production system, which is what all major distributed systems use for this exact problem.

Usage

  processor:
    - add_entries:
        entries:
          - key: recordId
            generate_uuid: true

Issues Resolved

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Xun Zhang <xunzh@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant