Feature/dpav 3018 - Discover and Product View

[nikan-negaresh-informed]

Sensitive Credential Checks

• As the author of these changes, I have checked for any sensitive credentials prior to this review being requested.
• As a reviewer of these changes, I have checked for any sensitive credentials prior to approving this merge.

Motivation and Context

Jira: DPAV-3018

Product discovery previously filtered in Java and returned a shape that could not be acted on: a
discovered product carried no id, so it could not be passed to any other endpoint, and there was no
way to retrieve a single product at all.

Two problems had to be solved together.

Discovery had to become policy-driven rather than code-driven. Entitlement logic (clearance
comparisons, purpose checks, attribute names) lived in Java, so changing who may see what meant a
release. The policy decision now returns a search contract, which the service compiles into one
parameterised SQL statement. Nothing a caller may not see is selected, paging counts are exact, and
one decision serves the whole search.

A single-product endpoint had to not become a hole in that filtering. If view applied weaker
rules than discovery, any caller holding the role could read the whole catalogue one id at a time,
including products discovery deliberately withholds. View is therefore implemented as discovery
constrained to one product: same caller gates, same row filter, same masking, from one shared
definition rather than two implementations that happen to agree.

Description

Discovery rebuilt as a contract-driven search pipeline. ProductDiscoveryServiceImpl is split
into focused components: ProductQueryPlanner (turns the decision into a contract),
ProductSearchCriteriaFactory (validates caller criteria against it), SqlPredicateCompiler and
ProductSearchQueryBuilder (compile the contract and criteria into SQL), ProductDiscoveryRepository
(executes), and DiscoveredProductAssembler (builds the response). ProductField, ProductBlock,
ProductProjection, DiscoverySchema and DiscoveryObligations support them.

New Product View endpoint. GET /api/v1/product/{productId}, role product_view, policy
product/view. Added ProductViewService and ProductViewServiceImpl; the id becomes one more
condition AND-ed onto the policy row filter. A product the row filter excludes returns 404, the
same answer as a product that does not exist, so an id cannot be used to probe what exists.

One shared policy contract. ProductPolicyContractDetails is a new base type returned by both
the discover and view rules, and lib/product_access.rego is the single definition both Rego rules
return. view_test.rego asserts the two decisions are equal for every sample organisation, so the
two endpoints cannot drift apart unnoticed.

Filter model. New FilterNode predicate tree with ComparisonOperator, Combinator,
FilterScope and FilterTarget, shared by the policy row filter and caller-supplied filters.
Attribute names are always bound parameters, never interpolated.

Dead code removed. ProductService, ProductServiceImpl and their tests are deleted, along with
the ProductRepository query methods they used. ProductControllerTest now exercises the discovery
and view services directly.

Database. Three schema migrations (product description column, the
policy_attribute_live_value view, discovery join indexes) and three local sample migrations
(product descriptions, sensitive organisation attribute flags, and the 17-product discovery
dataset).

Error handling. New InvalidSearchCriteriaException with handling in GlobalExceptionHandler,
so a malformed search is a 400 with a message naming the problem rather than a 500.

Identity. docker/keycloak/tofu/clients.tf adds the product_discovery, product_subscribe and
product_view client roles.

Documentation. POLICY_ENFORCEMENT.md, AUTHENTICATION_REQUIREMENTS.md and
DATABASE_SCHEMA.md updated; new DISCOVERY_TEST_SCENARIOS.md (guided tour over the sample data)
and docs/tests/products/ (a test requirement specification per API, which is what the testing team
checks against); policy_sample_stories.md and the OPA readme reworked.

97 files changed, 15,268 insertions, 1,826 deletions, across 5 commits.

How Has This Been Tested?

Automated. Every check below was run against a clean checkout of this branch head, so the
results describe exactly what is committed here and nothing from a local working tree:

Check	Command	Result
Java unit tests	./mvnw test	795 passed, 0 failures, 0 errors, 0 skipped
Formatting	./mvnw spotless:check	BUILD SUCCESS
Rego policy tests	opa test /p	116/116 passed
Rego strict check	opa check --strict /p	clean
Rego formatting	opa fmt --diff /p	no differences

Test coverage added by this branch includes ProductViewServiceImplTest, ProductQueryPlannerTest,
SqlPredicateCompilerTest, ProductSearchQueryBuilderTest, ProductSearchContractTest,
ProductSearchCriteriaFactoryTest, DiscoveredProductAssemblerTest,
ProductViewPolicyDecisionDetailsTest and FilterNodeSerializationTest, alongside substantially
expanded ProductDiscoveryServiceImplTest and ProductControllerTest.

Two cross-checks are asserted at build time rather than by inspection:

• discover_test.rego asserts the row filter and the per-candidate decision agree for every sample
  organisation and every sample product, catching any divergence between the SQL and the Rego.
• view_test.rego asserts view's whole decision equals discovery's, so "discoverable but not
  viewable" and the reverse are both impossible.

Manual verification follows docs/DISCOVERY_TEST_SCENARIOS.md against the local stack (Keycloak,
Postgres, OPA). Note that the policy decision point fails closed: with OPA stopped, every
policy-enforced endpoint returns 403, which is expected rather than a defect.

Not covered by the automated suite: the six Flyway migrations. The repo runs no Testcontainers
and Flyway is disabled in src/test/resources/application.yml, so the migrations are exercised only
by running the application against a local database.

Screenshots (if appropriate):

n/a, API-only change.

Checklist:

• It contains only changes required by issue (does not contain other PR)
• Includes link to an issue (if apply)
• I have added tests to cover my changes.

Reviewer notes

• Merge with squash and merge, per the template. The branch has 5 commits, three of which carry
  an identical message from a rebase, so the squashed message is worth writing by hand.
• The largest single reviewable idea is lib/product_access.rego: it is the one definition both
  product rules return, so it is where the caller gates, row filter and masking actually live.
• Behaviour changes are documented in docs/tests/products/, which the testing team checks against,
  so those files are part of the change rather than a follow-up.

[github-actions]

✅ OSS Checks Passed

All tracked OSS checks passed in this run.

📊 Total Files	🟢 Passed	🔴 Failed	🧮 Score
13	13	0	100%

Results from commit 24ec620, view the full job summary↗️ for detailed results.

♻️ This comment has been updated with latest results.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
90.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud