Skip to content

feat: add field argument mapping#1373

Open
dkorittki wants to merge 27 commits intomasterfrom
dominik/eng-8826-add-field-argument-mapping-support-to-engine
Open

feat: add field argument mapping#1373
dkorittki wants to merge 27 commits intomasterfrom
dominik/eng-8826-add-field-argument-mapping-support-to-engine

Conversation

@dkorittki
Copy link
Contributor

@dkorittki dkorittki commented Feb 2, 2026

Checklist

  • I have discussed my proposed changes in an issue and have received approval to proceed.
  • I have followed the coding standards of the project.
  • Tests or benchmarks have been added or updated.
  • Provide ability to disable field arg mapping
  • Wait for feat: improve memory management & request deduplication #1336 to be merged first, then basing this branch onto those changes

Context

Extends the engines variable normalizer to map field arguments as a side product of it's actual work and return the results alongside the file upload mapping.

This adds a useful feature which the router uses, called field argument mapping. It's described in detail in the corresponding router pull request.

The principle of field argument mapping is to allow Cosmo Streams hook users to be able to access values of field arguments to filter subscription events based on this. The idea is to give access the values via operation variables.
Values of any field arguments are available via variables, either because the users already use variables or because they have been extracted from literal values via variable extraction. The map that is being build holds the path to the field argument as a dot notated string and the variable name where the argument value is hold. Later on, when hook users want to access the value, the variable name can be resolved from the argument path and we can provide the variable value.

Here's an example:

query GetUser($userId: ID!) {
    user(id: $userId) {
        name
        orders(limit:12) {
            id
        }
    }
}`,

The map thats being build is:

map[argumentPath]variableName
"query.user.id", "userId"
"query.user.orders.limit", "a"

If a hook user later accesses query.user.orders.limit we resolve the value of variable a to get 12.
If the hook accesses query.user.id we can resolve userId, even if the engine performed variable remapping (userId being renamed to a or b, etc.)
This works because the variables which later are accessed are the ones from the operation context from the router and those are unaffected of variable remapping. In other words: The router config option engine.disable_variables_remapping does not have an effect on this. It works regardless.

The first implementation of this feature was done solely on the router. It worked without engine changes. A newly created walker would walk the operation AST and collect and map all field arguments. While this worked the problem was the introduction of a new walker during the request processing right in the hot path and it wasn't cachable either.

With this pull request the actual walking is done by an already existing visitor as a side product: variablesExtractionVisitor. This visitor is part of the variable normalizer and already walks field arguments. I just placed the new functionality into the visitors EnterArgument hook.

The variable normalizer does it's work by invoking the NormalizeOperation method. This method, in it's current form, returns the file upload mapping, which is cached by the router. This method has been modified to extend the return type by the field arg mapping results. Since the router already cached the result we get caching field arg mapping for free.

Field Argument Mapping can be enabled or disabled when the variable normalizer is instanciated.

func NewVariablesNormalizer(withFieldArgMapping bool) *VariablesNormalizer

This option is then passed down to the variablesExtractionVisitor, where the field argument mapping is happening.
The same visitor exists for the operation normalizer. There, however, we disable field argument mapping because it's a useful feature only on the variable normalizer.

Performance Considerations

This code hooks into an already existing walker, which walks all field arguments of an operation. The code added will allocate a string out of a slice byte via string convertion (string(myByteSlice) for most arguments. Also it will repeatedly grow a map, because the map initially is not allocated with a predetermined size. This could have performance implications on queries with vast amounts of field arguments but I think in reality we don't have a lot of them. Also once it's cached by the router, it does not matter anyway.

Summary by CodeRabbit

  • Refactor
    • Variable normalization now supports an option to enable field-argument mapping and returns a combined result with upload mappings and optional field-argument mappings; default mapping is off unless enabled.
    • Path string formatting was standardized to include inline fragment/type information in more contexts, affecting path rendering in messages and lookups.
  • Tests
    • Expanded and updated suites to cover field-argument mapping scenarios and the new structured normalization result.

The router makes sure to only pass normalized operations
to the variable normalizer. This means in practice that both,
named and inlined fragments, have already been inlined into
the query and the variable normalizer visitor never sees any
fragments. Therefore we don't need to explicitely support
them for field argument mapping. I removed it because
it would be expensive (if used), adds code complexity and
in practice is dead code.
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 2, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds optional field-argument mapping to variable normalization: the normalizer constructor now accepts a boolean to enable mapping and NormalizeOperation returns a result struct (uploads mapping + optional field-argument mapping) instead of a bare slice. Also changes Path.DotDelimitedString to accept a fragment-ref flag and updates call sites.

Changes

Cohort / File(s) Summary
Normalization core & result types
v2/pkg/astnormalization/astnormalization.go, v2/pkg/astnormalization/field_argument_mapping.go
Introduces VariablesNormalizerResult and FieldArgumentMapping. NewVariablesNormalizer now takes withFieldArgMapping bool. NormalizeOperation returns VariablesNormalizerResult. Removes direct uploads return from public API.
Variable extraction visitor
v2/pkg/astnormalization/variables_extraction.go
extractVariables gains withFieldArgMapping bool. Adds fieldArgMappingOption, extends variablesExtractionVisitor, and implements recordFieldArgumentMapping to populate FieldArgumentMapping when enabled; wires initialization and conditional recording in argument handling.
Normalization tests & helpers
v2/pkg/astnormalization/astnormalization_test.go, v2/pkg/astnormalization/variables_extraction_test.go, v2/pkg/astnormalization/variables_mapper_test.go, v2/pkg/astnormalization/variables_unused_deletion_test.go
Test harness and register functions updated to accept/propagate withFieldArgMapping. Tests updated to use VariablesNormalizerResult, assert UploadsMapping and FieldArgumentMapping, and add field-argument-mapping test cases; constructors updated to pass boolean flag.
Path API change
v2/pkg/ast/path.go
Path.DotDelimitedString() signature changed to DotDelimitedString(useFragmentRefs bool); inline fragment name emission is conditional on the flag.
Call sites affected by path change
v2/pkg/astvalidation/.../operation_rule_validate_empty_selection_set.go, v2/pkg/engine/datasource/graphql_datasource/.../graphql_datasource.go, v2/pkg/engine/datasource/graphql_datasource/representation_variable.go, v2/pkg/engine/plan/.../abstract_selection_rewriter.go, v2/pkg/engine/plan/.../datasource_filter_collect_nodes_visitor.go, v2/pkg/engine/plan/.../datasource_filter_resolvable_visitor.go, v2/pkg/engine/plan/.../key_fields_visitor.go, v2/pkg/engine/plan/.../node_selection_visitor.go, v2/pkg/engine/plan/.../path_builder_visitor.go, v2/pkg/engine/plan/.../provides_fields_visitor.go, v2/pkg/engine/plan/.../required_fields_visitor.go, v2/pkg/engine/plan/visitor.go
Replaced calls to DotDelimitedString() with DotDelimitedString(true) (or WithoutInlineFragmentNames().DotDelimitedString(true) where applicable). Adjusts emitted path strings used by lookups and error messages.

Sequence Diagram(s)

mermaid
sequenceDiagram
participant Caller
participant VariablesNormalizer
participant ExtractionVisitor
participant Path
Caller->>VariablesNormalizer: NewVariablesNormalizer(withFieldArgMapping)
Caller->>VariablesNormalizer: NormalizeOperation(operation, definition, report)
VariablesNormalizer->>ExtractionVisitor: extractVariables(withFieldArgMapping)
ExtractionVisitor->>Path: build/record field path (DotDelimitedString(useFragmentRefs?))
ExtractionVisitor-->>VariablesNormalizer: UploadsMapping + optional FieldArgumentMapping
VariablesNormalizer-->>Caller: VariablesNormalizerResult{UploadsMapping, FieldArgumentMapping}

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat: add field argument mapping' accurately describes the main feature introduced in the PR, which adds a new field-argument mapping capability to the variables normalizer.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch dominik/eng-8826-add-field-argument-mapping-support-to-engine

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Adds the ability to enable or disable field arg mapping
when the variable normalizer is created. Since this
option is only useful on the variable normalizer it is
disabled in code for the variable extraction on the
operation normalizer.
@dkorittki dkorittki marked this pull request as ready for review February 9, 2026 15:39
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
v2/pkg/astnormalization/variables_extraction.go (1)

87-95: ⚠️ Potential issue | 🟠 Major

Bug: field argument mapping not recorded when a literal value is reused via variableExists.

When two field arguments have identical literal values, variableExists finds the already-extracted variable and reuses it (lines 88-94). However, recordFieldArgumentMapping is never called on this path, so the second field argument's mapping entry is silently dropped.

Example scenario:

query {
  firstAlias: user(id: "123") { name }
  secondAlias: user(id: "123") { name }
}

Only "query.firstAlias.id" -> "a" would be recorded; "query.secondAlias.id" would be missing.

Proposed fix
 	if exists, name, _ := v.variableExists(valueBytes, inputValueDefinition); exists {
 		variable := ast.VariableValue{
 			Name: v.operation.Input.AppendInputBytes(name),
 		}
 		value := v.operation.AddVariableValue(variable)
 		v.operation.Arguments[ref].Value.Kind = ast.ValueKindVariable
 		v.operation.Arguments[ref].Value.Ref = value
+		if v.fieldArgumentMapping.enabled {
+			v.recordFieldArgumentMapping(ref, string(name))
+		}
 		return
 	}
🧹 Nitpick comments (1)
v2/pkg/astnormalization/astnormalization_test.go (1)

1196-1204: Test "empty mapping when no arguments" uses a query that would fail GraphQL validation.

The operation query GetUser { user { name } } omits the required id: ID! argument on the user field (per the schema at line 1073). While this tests the normalizer's behavior on invalid input (which is fine — it correctly returns an empty mapping), consider adding a comment noting this is intentionally invalid to aid future readers.

func (v *VariablesNormalizer) NormalizeOperation(operation, definition *ast.Document, report *operationreport.Report) []uploads.UploadPathMapping {
// NormalizeOperation processes GraphQL operation variables.
// It detects and removes unused variables, extracts variables from inline values
// and coerces variable types. It modifies the operation in place and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it coerses not types, but variable values, we could also mention what part of graphql spec it is https://spec.graphql.org/September2025/#sec-Coercing-Variable-Values

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// It detects and removes unused variables, extracts variables from inline values
// and coerces variable types. It modifies the operation in place and
// returns metadata including field argument mappings and upload paths.
// Field argument mapping is done only when v is configured to do so via NewVariablesNormalizer,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when v is configured
when VariablesNormalizer

or when collecting arguments mapping enabled

Copy link
Contributor Author

@dkorittki dkorittki Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what "when v is configured to do so via NewVariablesNormalizer" wanted to express. I'll rephrase it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

variables string
expectedMapping FieldArgumentMapping
}{
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing test case for complex input type, smth like this

type Query {
   users(paging: Paging!): [String]!
}

input Paging {
  limit: Int!
  offset: Int!
}

such query

query Search($limit: Int!, $offset: Int!) {
  users(paging: {limit: $limit, offset: $offset})
}
variables: {"limit": 10, "offset": 5}

will be normalized into

query Search($a: Paging!) {
  users(paging: $a)
}
variables: {"limit": 10, "offset": 5, "a": {"limit": 10, "offset": 5}}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// of v in v.fieldArgumentMapping, alongside its matching variable name or literal value.
// If varName is empty, it looks up the variable name from the operation or stores the literal value.
func (v *variablesExtractionVisitor) recordFieldArgumentMapping(ref int, varName string) {
// Guard to prevent nil panics.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when it is enabled we should always have value

potentially we could just put a check here

if v.fieldArgumentMapping.enabled {

Copy link
Contributor Author

@dkorittki dkorittki Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The relationship between v.fieldArgumentMapping.enabled == true and v.fieldArgumentMapping.result != nil is loose. It's enforced in EnterDocument. It could change by accident in the future and hypothetically we could end up with v.fieldArgumentMapping.enabled == true and v.fieldArgumentMapping.result == nil. Then if v.fieldArgumentMapping.enabled would not return early and we would run into a nil panic.

I did it this way to be maximum defensive. This function wants to work with v.fieldArgumentMapping.result so it must exist.

Will change the comments to make this intend more clear

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

if v.operation.Arguments[ref].Value.Kind != ast.ValueKindVariable {
// We expect the operation on this visitor to be normalized before the visitor walks it.
// This means that values of any field arguments should be extracted to variables.
// If we still land here it means the query hasn't been normalized before.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we still land here it means the query hasn't been normalized before.

a bit strange comment, because this visitor is doing this normalization

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

type FieldArgumentMapping map[string]string

// VariablesNormalizerResult contains the results of variable normalization.
type VariablesNormalizerResult struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about VariablesNormalizationResult name?

I propose to move VariablesNormalizer into its own file variables_normalizer or variables_normalization and add these types there

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// NewVariablesNormalizer creates a new variable normalizer.
// If withFieldArgMapping is true then the normalizer creates a
// mapping of field arguments as a "side product" when NormalizeOperation is called.
func NewVariablesNormalizer(withFieldArgMapping bool) *VariablesNormalizer {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about introducing either an options or config argument?

e.g.

type VariablesNormalizerOptions struct {
  withFieldArgMapping bool
}

WithFieldArgMapping()

or just

type VariablesNormalizerConfig struct {
  WithFieldArgMapping bool
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants