ai: Pre-append prompt templates and establish init barrier for Writing Assistance APIs - #62337
Draft
chromium-wpt-export-bot wants to merge 1 commit into
Draft
ai: Pre-append prompt templates and establish init barrier for Writing Assistance APIs#62337chromium-wpt-export-bot wants to merge 1 commit into
chromium-wpt-export-bot wants to merge 1 commit into
Conversation
…g Assistance APIs Pre-append static options and shared context (the system prompt template) to the on-device session during creation for AISummarizer, AIWriter, and AIRewriter via OnDeviceSession::SetInput. Per-request dynamic execution fields (article, instructions, rewrite_text, and per-request context) are marked as pending on the initial MultimodalMessage so that template substitution halts at the static prefix without evaluating empty execution strings or generating premature control tokens. This change accomplishes two objectives: - crbug.com/540980352: Blocks create() promise resolution until the underlying On-Device Model Service (ODMS) has fully initialized the engine and model weights, eliminating the cold-start penalty currently incurred during the first summarize()/write()/rewrite() call. - crbug.com/390006887: Pre-populates and caches the static prompt options and shared context in the session KV cache during session creation, avoiding redundant re-tokenization on every execution call. Automated tests: - Add prompt verification tests across AIWriter, AIRewriter, and AISummarizer unit tests to check substituted prompt strings. - Add multi-execution session reuse tests to verify shared context caching across sequential requests. - Add unit test for MultimodalMessage::Merge pending overlay clearance. Bug: 390006887, 540980352 Change-Id: I199e35e62e6f084f490965a7d8bb7eafda27393c
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-append static options and shared context (the system prompt
template) to the on-device session during creation for AISummarizer,
AIWriter, and AIRewriter via OnDeviceSession::SetInput.
Per-request dynamic execution fields (article, instructions,
rewrite_text, and per-request context) are marked as pending on the
initial MultimodalMessage so that template substitution halts at the
static prefix without evaluating empty execution strings or generating
premature control tokens.
This change accomplishes two objectives:
Automated tests:
AISummarizer unit tests to check substituted prompt strings.
caching across sequential requests.
Bug: 390006887, 540980352
Change-Id: I199e35e62e6f084f490965a7d8bb7eafda27393c
Reviewed-on: https://chromium-review.googlesource.com/8255743
WPT-Export-Revision: c7515ce5c4491f496cae3140b00af7c68002a01e