Skip to content

Support pattern expressions as boolean expressions - #2360

Merged
jrgemignani merged 5 commits into
apache:masterfrom
gregfelice:feature_pattern_in_where
Jun 22, 2026
Merged

Support pattern expressions as boolean expressions#2360
jrgemignani merged 5 commits into
apache:masterfrom
gregfelice:feature_pattern_in_where

Conversation

@gregfelice

@gregfelice gregfelice commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds support for openCypher pattern expressions as general boolean expressions. Pattern expressions are now accepted anywhere a regular expression is valid — in WHERE, RETURN, WITH, SET, CASE, and as operands of boolean combinators — not just in WHERE.

Note on scope: An earlier version of this PR was titled "Support pattern expressions in WHERE clause via GLR parser." While implementing, Copilot correctly pointed out that adding anonymous_path to expr_atom actually enables pattern expressions in every expression context, not only WHERE. Rather than restrict the grammar with a WHERE-specific nonterminal, this PR now explicitly embraces the broader surface area — matching what openCypher defines and what Neo4j users expect — and covers the new contexts with regression tests. The PR title and description have been updated accordingly.

// WHERE (original motivation — still works)
MATCH (a:Person), (b:Person)
WHERE (a)-[:KNOWS]->(b)
RETURN a.name, b.name

// NOT pattern expression in WHERE
MATCH (a:Person)
WHERE NOT (a)-[:KNOWS]->(:Person)
RETURN a.name

// RETURN — project a boolean
MATCH (a:Person)
RETURN a.name, (a)-[:KNOWS]->(:Person) AS knows_someone

// CASE WHEN
MATCH (a:Person)
RETURN a.name,
       CASE WHEN (a)-[:KNOWS]->(:Person) THEN 'social' ELSE 'loner' END

// Combined with AND / OR
MATCH (a:Person)
RETURN a.name,
       (a)-[:KNOWS]->(:Person) AND (a)-[:WORKS_WITH]->(:Person) AS has_both

// SET — persist a computed boolean as a property
MATCH (a:Person)
SET a.is_social = (a)-[:KNOWS]->(:Person)
RETURN a.name, a.is_social

// WITH pipeline
MATCH (a:Person)
WITH a.name AS name, (a)-[:KNOWS]->(:Person) AS knows
WHERE knows
RETURN name

Implementation

Grammar (cypher_gram.y)

  • Switched the parser to GLR mode (%glr-parser) to handle the inherent ambiguity between parenthesized expressions and graph patterns. Both start with (, so a single-token lookahead cannot decide which production applies until several tokens later. GLR forks at the conflict point and discards the failing alternative.
  • Added anonymous_path to expr_atom with a %dprec 1 annotation. A bare (a) prefers the expression-variable interpretation (%dprec 2 on '(' expr ')'), so single-node pattern expressions still resolve as plain variable references.
  • Added make_exists_pattern_sublink() to wrap the pattern in an EXISTS subquery — a bare pattern (a)-[:KNOWS]->(b) in expression context is semantically equivalent to EXISTS((a)-[:KNOWS]->(b)).
  • Added a helper make_explain_stmt() and placed it immediately above its docstring comment so the grammar file stays readable.
  • Replaced (rhs)[1] with YYRHSLOC(rhs, 1) in the YYLLOC_DEFAULT macro — a required GLR-correctness fix, not stylistic. GLR's rhs stack has a different layout than LALR(1)'s; the raw index lands on the wrong slot under GLR and produces incorrect source locations.
  • Conflict budget pinned with %expect/%expect-rr. GLR produces 7 shift/reduce and 3 reduce/reduce conflicts (all arising from the ( ambiguity between path_node and parenthesized expr, plus expr_var/var_name_opt overlap on )/}/=). These are handled correctly at runtime by GLR + %dprec. The grammar declares %expect 7 and %expect-rr 3 so any deviation up or down fails the build and forces an audit of the new conflicts. The Makefile keeps -Werror and scopes the relaxation to -Wno-error=conflicts-sr -Wno-error=conflicts-rr, so other Bison warning categories (unused rules, undeclared types, etc.) still fail the build. Earlier rounds of this PR tried %expect-less variants for "Bison version portability," but Bison's %expect is exact-match — so the right answer is to pin the totals and update them deliberately if a future Bison reports different counts, not to remove the alarm.

Parser (cypher_analyze.c, cypher_clause.c)

  • Existing EXISTS sublink handling already covers the pattern case — no new transform code needed on the analyze/clause side.

Files changed

File Change
src/backend/parser/cypher_gram.y %glr-parser, %dprec annotations, anonymous_path in expr_atom, make_exists_pattern_sublink(), GLR comment block explaining why conflicts are expected
Makefile BISONFLAGS += -Werror -Wno-error=conflicts-sr -Wno-error=conflicts-rr — keep -Werror for other warning categories, scope relaxation to the two known-noise conflict categories; register pattern_expression regression test
regress/sql/pattern_expression.sql New regression suite: WHERE, NOT, nested, multi-hop, RETURN projection, mixed projections, CASE WHEN, AND/OR combinators, SET, WITH pipeline, regression guards for parenthesized arithmetic
regress/expected/pattern_expression.out Expected output for above

Test plan

  • Basic pattern in WHERE: WHERE (a)-[:KNOWS]->(b)
  • NOT pattern in WHERE: WHERE NOT (a)-[:KNOWS]->(:Person)
  • Pattern in WHERE with label: WHERE (a)-[:KNOWS]->(:Person)
  • Pattern via EXISTS subquery (original syntax still works)
  • Pattern in RETURN with AS alias → boolean column
  • Pattern in RETURN without alias (positional column)
  • Multiple pattern expressions in same RETURN
  • Pattern in CASE WHEN ... THEN ... ELSE ... END
  • Pattern combined with AND and OR in RETURN
  • Pattern in SET a.flag = (a)-[:R]->(:L)
  • Pattern in WITH projection + subsequent WHERE filter
  • Regression guards: RETURN (1 + 2), RETURN (n.name) — parenthesized expressions still work
  • All 32 regression tests pass (31 existing + pattern_expression)
  • Build is clean under -Werror with %expect 7 %expect-rr 3 pinned (Bison 3.8.2); any conflict-count drift now fails the build deliberately

Implementation notes (added during review)

  • YYRHSLOC macro change is a required GLR correctness fix, not cosmetic. GLR uses a different RHS stack layout than LALR(1); the previous (rhs)[1] indexing reads the wrong slot under GLR. The macro update is load-bearing for correct error locations under %glr-parser.
  • Pattern expressions in projection are EXISTS-sublink-per-row. A bare pattern in a RETURN/WITH/SET projection (e.g. RETURN a.name, (a)-[:KNOWS]->(:Person)) desugars to a correlated EXISTS sublink evaluated once per outer row — O(rows × subquery cost). This is the correct openCypher semantics, but distinct from a pattern in WHERE, where the planner may pull the sublink up into a semi-join. Regression tests use tiny datasets so this is invisible; on large data, projection-context pattern expressions cost more than WHERE-context ones.

Known limitation (pre-existing, tracked separately)

  • Single-node labeled pattern expressions (a:Label) do not filter by label. Because this PR lets a bare pattern appear as an expression anywhere, (a:Label) is now syntactically accepted as a boolean (e.g. WHERE (a:Person) or RETURN (a:Person)). It does not behave as an openCypher label predicate: the single-vertex pattern desugars to an EXISTS subquery with no label constraint and so is trivially true. Root cause is pre-existing in the transform layer — make_path_join_quals() (src/backend/parser/cypher_clause.c:5454) early-returns for vertex-only patterns (list_length(entities) < 3), skipping the label-filter quals that are only emitted for relationship patterns. This is not introduced by this PR (the code is byte-identical on master, and this PR touches no transform code); the grammar change merely makes the limitation reachable via a new surface. Relationship patterns — the feature this PR targets — correlate and filter correctly. Tracked as a follow-up; use a relationship pattern or label(a) = 'Label' for label checks in the meantime.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants