feat: add Query.Builder, aggregates, joins, and nested ON - #1817
feat: add Query.Builder, aggregates, joins, and nested ON#1817abnegate wants to merge 33 commits into
Conversation
Greptile SummaryThe PR expands generated SDK query APIs with aggregate, grouping, join, nested-ON, spatial, and fluent-builder support.
Confidence Score: 5/5The PR appears safe to merge because no blocking failure remains within the eligible follow-up-review scope. No blocking failure remains. Important Files Changed
Reviews (31): Last reviewed commit: "style(templates): satisfy the formatter ..." | Re-trigger Greptile |
|
@greptileai review |
1 similar comment
|
@greptileai review |
Query helpers now cover the query-lib methods Appwrite allows (aggregates, groupBy, having, distinct, joins, extra spatial). Query.builder()/build() collects filters as a flat string list, and Client.normalize unwraps the common extra-array wrap of a builder or build()/page() result so list and POST /query calls stay valid.
`any` does not exist on Go 1.13, which still compiles the SDK in CI. Realtime subscription updates now go through Query.normalize instead of iterating QueryInput, and Client.subscribe no longer writes a queries set that is not on the Realtime connection type.
Query.normalize was only flattening a builder or nested build()/page() array before send. Call sites already do that on the queries param, so it does not belong on the public Query helper surface.
Keep the simple join(table, left, right, op, alias) triple. Array overloads encode Query.on() plus filter queries into nested join values so extra ON predicates stay on the JOIN.
Add e2e prints and unit tests for aggregates, joins, ON, spatial, and page so each SDK exercises every new query type.
Clients now map every param through a type-based flatten so builders and extra nested lists unwrap without special-casing the queries key. Document the new public Query helpers.
Live specs encode closed enums as type:string plus oneOf of single-value string schemas. Parsing that as an object made path params Any, JSON keys the enum title, and Ruby docs crash on non-JSON examples. Keep model JSON keys as the property name, type signatures from the collapsed enum, and flatten Dart request maps as Map<String, dynamic>.
420fd0f to
844e3e1
Compare
Collapsed oneOf string enums with no title produced empty type names (`val status: ,`, `resourceType: ;`) and invalid PHP properties.
Move QueryBuilder into Query/Builder files and nested types where the language allows it. Alias QueryBuilder at import for PHP and the TypeScript public export.
Dart part-of is resolved relative to lib/query, so the builder library looked for packageName.dart in the wrong directory, and a top-level Builder class collides with Flutter's widget. Go treats query.Builder() as a type conversion. PHP mock payloads and from() treated array-of-enum fields as a single string, which is how live-spec OAuth prompt examples are encoded. CLI Validation now compiles against a Go SDK generated from the same spec instead of the last published module.
A replace in go.mod is the local pin; Validation no longer needs an env override or go mod edit. example.php writes examples/go from the same spec so ../go matches the CLI module path.
Request flattening is a transport concern. Keep Query as the helper surface and run flatten from Client in Kotlin, Android, Swift, and Apple.
Keep annotated enumerations from main and the query builder work. CLI go.mod still replace-pins the sibling generated Go SDK at v7.2.0-rc.3.
There was a problem hiding this comment.
Pull request overview
Adds cross-language query aggregation, grouping, joins, spatial operators, fluent builders, request flattening, and enum-handling improvements to generated SDKs.
Changes:
- Expands Query APIs and adds fluent builders across supported SDK languages.
- Normalizes nested query inputs in clients and adds broad E2E/unit coverage.
- Improves closed
oneOfenum detection and scalar array-enum hydration.
Reviewed changes
Copilot reviewed 110 out of 110 changed files in this pull request and generated 29 comments.
Show a summary per file
| File(s) | Description |
|---|---|
tests/resources/spec-openapi3.json |
Adds closed oneOf enum fixtures. |
tests/e2e/Base.php |
Adds helper-output and enum hydration assertions. |
tests/e2e/languages/{web,node,react-native,deno}/* |
Exercises TypeScript query helpers and builders. |
tests/e2e/languages/{swift,apple}/Tests.swift |
Exercises Swift query helpers. |
tests/e2e/languages/{kotlin,android}/Tests.kt |
Exercises Kotlin query helpers. |
tests/e2e/languages/{go,go-v2}/tests.go |
Exercises Go query helpers. |
tests/e2e/languages/{dart,flutter}/tests.dart |
Exercises Dart query helpers. |
tests/e2e/languages/{dotnet,unity}/Tests.cs |
Exercises C# query helpers. |
tests/e2e/languages/{ruby,python,php}/* |
Exercises dynamic-language query helpers. |
templates/web/src/{query,client,index}.ts.twig |
Adds Query builder/input normalization and exports. |
templates/web/src/services/realtime.ts.twig |
Normalizes realtime query inputs. |
templates/node/src/{client,index}.ts.twig |
Adds Node request flattening and exports. |
templates/node/test/query.test.js.twig |
Tests Node query behavior. |
templates/react-native/src/{query,client,index}.ts.twig |
Adds React Native query building and flattening. |
templates/deno/{mod.ts,src/query.ts,src/client.ts}.twig |
Adds Deno query building and flattening. |
templates/deno/test/query.test.ts.twig |
Tests Deno query behavior. |
templates/swift/Sources/Query*.twig |
Adds Swift helpers and nested builder. |
templates/swift/Sources/Client.swift.twig |
Flattens Swift request parameters. |
templates/swift/{Tests/Tests.swift,Package.swift}.twig |
Adds tests and conditional package targets. |
templates/apple/{Sources/Client.swift,Package.swift}.twig |
Applies inherited Swift client/package updates. |
templates/kotlin/src/main/kotlin/io/appwrite/{Query,Client}.kt.twig |
Adds Kotlin query APIs and flattening. |
templates/kotlin/src/test/kotlin/io/appwrite/QueryTest.kt.twig |
Adds Kotlin Query tests. |
templates/android/library/src/main/java/io/package/{Query,Client}.kt.twig |
Adds Android query APIs and flattening. |
templates/go/query*.twig |
Adds Go helpers, builder, and tests. |
templates/go/client.go.twig |
Flattens Go request values. |
templates/rust/src/query/{mod,builder}.rs.twig |
Adds Rust Query APIs and builder. |
templates/rust/src/client.rs.twig |
Normalizes Rust request parameters. |
templates/rust/tests/tests.rs |
Exercises Rust query helpers. |
templates/dotnet/Package/Query*.twig |
Adds .NET helpers and nested builder. |
templates/dotnet/Package/Client.cs.twig |
Flattens .NET request parameters. |
templates/php/src/{Query,Client}.php.twig |
Adds PHP helpers, builder integration, and flattening. |
templates/php/src/Query/Builder.php.twig |
Implements the PHP fluent builder. |
templates/php/src/Models/{Model,RequestModel}.php.twig |
Wraps scalar array-enum values during hydration. |
templates/php/tests/QueryTest.php.twig |
Tests PHP Query behavior. |
templates/python/package/{query,client}.py.twig |
Adds Python builder APIs and flattening. |
templates/python/test/test_query.py.twig |
Tests Python Query behavior. |
templates/ruby/lib/container/{query,client}.rb.twig |
Adds Ruby helpers and request flattening. |
templates/ruby/lib/container/query/builder.rb.twig |
Implements the Ruby fluent builder. |
templates/dart/lib/{query,package}.dart.twig |
Adds Dart Query APIs and builder registration. |
templates/dart/lib/query/builder.dart.twig |
Implements QueryBuilder. |
templates/dart/lib/src/client_mixin.dart.twig |
Flattens Dart request parameters. |
templates/dart/lib/src/models/{model,request_model}.dart.twig |
Handles scalar array-enums and enum serialization. |
templates/dart/test/{query_test,src/models/model_test}.dart.twig |
Extends Dart Query/model tests. |
templates/flutter/lib/{package,src/client_mixin}.dart.twig |
Registers builders and flattens Flutter parameters. |
templates/skills/*.md.twig |
Documents query aggregation, joins, and builders. |
templates/cli/internal/typegen/templates/{types,databases}.ts.hbs |
Extends typed CLI Query helpers. |
templates/cli/internal/generator/typescript.go |
Reuses the generated QueryBuilder type. |
templates/cli/internal/generator/testdata/* |
Updates expected TypeScript output. |
templates/cli/go.mod.twig |
Redirects CLI builds to a local generated Go SDK. |
src/SDK/SDK.php |
Improves enum-name fallback resolution. |
src/SDK/Language.php |
Resolves untitled enum names from parent properties. |
src/SDK/Language/{Swift,Apple}.php |
Registers Swift builder output. |
src/SDK/Language/{Kotlin,Android}.php |
Registers Query tests. |
src/SDK/Language/{Dart,Flutter}.php |
Registers Dart builder output. |
src/SDK/Language/{Go,Rust,PHP,Ruby,DotNet}.php |
Registers new builder/module templates. |
example.php |
Generates Go alongside CLI and updates the Go SDK version. |
Suppressed comments (2)
templates/cli/internal/typegen/templates/types.ts.hbs:76
- These fixed scalar signatures cannot express the new nested JOIN ON form, and the facade exposes no
onhelper. Addonplus overloads/unions for(table, on)and(table, alias, on)so the typed callback can construct the nested wire shape supported byQuery.
join: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
leftJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
rightJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
fullOuterJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
templates/cli/internal/typegen/templates/databases.ts.hbs:44
- The facade delegates only the scalar join form and does not expose
Query.on, so typed CLI callbacks cannot use the nested JOIN ON feature added by this PR. Mirror the nested overloads andonhelper exposed byQuery.
join: (table, left, right, operator = '=', alias = '') => Query.join(table, left, right, operator, alias),
leftJoin: (table, left, right, operator = '=', alias = '') => Query.leftJoin(table, left, right, operator, alias),
rightJoin: (table, left, right, operator = '=', alias = '') => Query.rightJoin(table, left, right, operator, alias),
fullOuterJoin: (table, left, right, operator = '=', alias = '') => Query.fullOuterJoin(table, left, right, operator, alias),
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Rust was treating every nested array as query input, Go compared uncomparable slices, and Dart/Flutter left wrapped builders nested. Skills now show build() only; typed CLI query helpers include the new aggregates. Regenerating Rust deletes leftover src/query.rs.
|
Went through Copilot's 29 threads. Fixed
Left as-is
|
Language e2e tests now print flatten-builder/geometry/list so a wrapped builder that stays nested or a numeric array that gets spread fails CI. Query flatten is callable from those tests; PHP form flatten is flattenForm so it no longer collides with Client::flatten.
Typed QueryBuilder gained stddev/variance/bitwise; baseline fixtures still expected groupBy at that offset and failed TestGenerateMatchesBaseline.
|
Pushed two commits on top of the flatten-bugfix head:
CLIGo126 / Validation |
|
@greptileai review |
Flutter query_test constructing Client() hits path_provider before the widget binding exists. Apple flatten returns [Any], so as? [String] failed even when the query strings were correct.
|
@greptileai review |
Linux Swift does not cast [Query.Builder] or [String] to [Any], so flatten left wrapped builders nested. Walk those typed arrays (and Mirror as a fallback) so builder lists unwrap on Apple and Swift.
|
@greptileai review |
Foundation can surface [String] as NSArray, so casting to [Any] first walked NSString as a character collection and broke flatten-list on Apple.
|
@greptileai review |
Keep query flatten tests and enum hydration checks. Take main's inferred upload ID fixture assertion and Black-formatted Python SDK templates.
|
@greptileai review |
Validation runs black --check on generated Python tests. Swift flatten now returns [Query.Builder].flatMap(build) and [String] before Any/NSArray so Linux does not walk query strings as character collections.
|
@greptileai review |
Homogeneous [Query.Builder] does not round-trip through Any on Linux. Request params are Any values, so the language tests now pass [Any].
|
@greptileai review |
Keep query flatten helpers and enum fixtures. Take PHP Pint/PHPStan quality templates and localized enum keys from main. Swift flatten spreads any nested string list returned from a builder.
|
@greptileai review |
Generated PHP tests now use PSR-12 braces. Swift/Apple language tests no longer assert Client.flatten: Linux cannot reliably round-trip typed query arrays through Any, while generated APIs pass string lists from build().
|
@greptileai review |
|
@greptileai review |
|
@greptileai review |
|
@greptileai review |
|
@greptileai review |
|
@greptileai review |
|
@greptileai review |
Nine conflicts, all where main's formatter/refactor work touched lines this
branch had already rewritten. Checked each side for content the other lacks
rather than resolving by side:
- apple/swift Package.swift: kept this branch's {%~ if %} conditionals around
the Enums/Models targets and took main's trailing comma after "JSONCodable",
which is what its formatter pass standardised on.
- android Client.kt: main extracted prepareRequest() out of call(), so this
branch's old call() signature anchored against it. Took main's structure and
kept the branch's flatten() over the params, which is what resolves a
Query/Query.Builder value before the null filter drops it.
- web client.ts and realtime.ts: main hand-rolls query flattening, one level of
nesting deep. normalizeQueries() recurses, and also handles Query.Builder and
null, so the branch's version is a strict superset — verified before taking
it rather than assumed.
- node/react-native/swift/web query templates and tests: main's side held
nothing this branch does not already have (main had notTouches; the branch has
it plus 32 aggregate/join methods).
Only e2e tests exist here and they build Docker images, which this host cannot
pull, so local verification is composer lint (clean) and lint-twig (592 files,
0 errors). CI is the authority for the rest.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
examples/web/.prettierrc is tabWidth 4 with single quotes, and CI runs
`npm run format:check` over the generated SDK. This branch restyled
query.ts.twig to two-space double-quoted, so every generation failed that check.
query.ts.twig carries no Twig tags, so it is the generated file: formatted with
the SDK's own config and taken verbatim. index.ts.twig is formatted the same
way, with its tags held aside during the pass and restored at column 0 — an
indented {% %} emits that indentation into the output.
Object keys are formatted with quoteProps preserve. Without it Prettier strips
the quotes from a tokenised key, and once the token expands the result is
`X-Appwrite-Response-Format: '1.9.6'` — an unquoted hyphenated key that does not
parse. Caught by `npm run analyse` going from exit 0 to exit 2.
client.ts.twig and realtime.ts.twig are improved but not yet clean: their Twig
interpolations change line lengths between template and output, so Prettier's
wrapping decisions cannot be settled at the template level alone. Verified
unchanged from baseline: `npm run analyse` exit 0, eslint 8 problems, twig lint
592 files / 0 errors.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
main landed swift-format, rustfmt and the eslint/prettier chain as required checks after this branch last ran green, so the query-helper code it introduces was never measured against them. Seven jobs went red on the merge: apple, swift, rust, node, react-native and web on both client and console. Every hunk is the formatter's own output, taken by running the gate locally and porting the result back through the template. Where a twig tag sits on the line, the wrap is driven by the generated line rather than the template line, because interpolation changes the width. The lint fixes are real, not cosmetic: a lexical declaration in an unbraced case block, a dead initialiser, two generics constrained to unknown, three let bindings that are never reassigned and an unused listener argument. queryParams and the two localStorage guards differ per platform, so their twig conditionals now carry the whole statement instead of a fragment. Header keys in the generated client are quoted only when prettier would quote them, so a hyphenated header can no longer emit invalid JS. Verified locally against each gate: swift and apple lint clean and build; rust is fmt-clean with 786 tests passing; node runs 865 tests across 34 suites; web (client and console) and react-native pass format:check, eslint, tsc and build. djlint and the 1200-character twig check are clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Why
Appwrite query-lib added aggregates, grouping, joins, extra spatial operators, and a POST
/querybody so largequeriesarrays are not capped by URL length. SDKs that already exposeQueryhelpers need those methods, a fluent collector, and a way to nest JOIN ON conditions without leaking SQL.What changed
count,sum,avg, …),groupBy/having/distinct, joins (join/leftJoin/rightJoin/fullOuterJoin/crossJoin), extra spatial (covers, …).Query.Builderwhere the language allows it (C#/Swift/Kotlin/Ruby/Python/TS namespace). PHP isAppwrite\Query\Builderimported asQueryBuilder. Go/Rust use packageBuilderwithNewBuilder()/query::Builder. Dart cannot nest classes and must not export a top-levelBuilder(Flutter widget clash), so the type isQueryBuilderinlib/query/builder.dart.Query.on(...)plus join overloads that take an alias, an ON query, and extra filters. The wire shape is[alias?, Query.on, filters…].Query.normalize. Flattening is structural (builder / nested query-string lists) and does not special-case thequeriesparameter name.type: stringplusoneOfof single-value schemas are treated as wire enums. Untitled oneOf members take their name from the parent property.from()and generated mocks wrap a scalar example for an array-of-enum field into a one-element list (OAuthpromptin the live spec is a string example on an array field).QUERY_HELPER_RESPONSEScovers each new helper in every SDK language test.Raw/union/json/naturalJoin stay unexposed — those are not on the Documents validator allow-list.
Vector helpers were already on
main; duplicate copies introduced during the rebase were removed so each language has a single definition.Why this approach
A client-side JSON collector (not SQL) matches the existing Query string format
{method, attribute?, values?}. Nesting the builder underQuerykeeps the public surface one type. Flattening inside the Client, without aquerieskey check, means any param that already carries query strings (join ON lists,having,or/and) is handled the same way.POST
/queryitself lives in Appwrite (appwrite/appwrite#13300, stacked on#11649). SDK methods will be generated ascreateQueryonce that spec is published.Verified
composer lint(phpcs PSR-12)composer lint-twig(djLint, 0 errors)composer refactor:check(Rector dry-run)composer test tests/e2e/PHP83Test.php— OK, 1511 assertions (includes array-enum scalar hydration)lib/+tests/tests.dartanalyze clean for the Builder/part-of/MockKind failuresquery.NewBuilder().Limit(1).Build()compiles and prints the expected limit queryNot verified
/queryround-trip from a generated SDK (depends on appwrite#13300 + spec publish).../govia areplaceingo.mod.twig(generated from the same spec as the CLI). Compiling the published CLI package against the last published Go module is not verified here.