Search: sharded cluster operator support#816
Conversation
ce459f2 to
a917d32
Compare
a917d32 to
915b024
Compare
0a2cc17 to
dda0ae1
Compare
f449329 to
29a42da
Compare
| // Watch for MongoDBSearch resources that reference ShardedCluster MongoDB resources | ||
| // Only enqueue reconciliation requests for ShardedCluster resources, not ReplicaSet or Standalone | ||
| shardedKubeClient := mgr.GetClient() | ||
| err = c.Watch(source.Kind(mgr.GetCache(), &searchv1.MongoDBSearch{}, |
There was a problem hiding this comment.
we have now two mechanisms for watching related resources. I'm wondering if we could align it with controllers/operator/watch/resource_watcher.go one way or another
controllers/searchcontroller/mongodbsearch_reconcile_helper_test.go
Outdated
Show resolved
Hide resolved
controllers/searchcontroller/mongodbsearch_reconcile_helper_test.go
Outdated
Show resolved
Hide resolved
| } | ||
| } | ||
|
|
||
| func TestValidatePerShardTLSSecretsAllExist(t *testing.T) { |
There was a problem hiding this comment.
we should have more tests testing full reconcile, checking all the expected resource names, mongot configs, etc.
There was a problem hiding this comment.
those individual tests checking only validatePerShardTLSSecrets, validatePerShardTLSSecrets should be removed and covered in a bigger test case
| } | ||
|
|
||
| // CreateShardSearchStatefulSetFunc creates a StatefulSet for a specific shard's mongot deployment. | ||
| func CreateShardSearchStatefulSetFunc(mdbSearch *searchv1.MongoDBSearch, shardedSource ShardedSearchSourceDBResource, shardIdx int, searchImage string) statefulset.Modification { |
There was a problem hiding this comment.
this should be unified with the replicaset mongot
| return xerrors.Errorf("error creating/updating shard search service %v: %w", svcName, err) | ||
| } | ||
|
|
||
| zap.S().Debugf("Updated shard search service %v: %s", svcName, op) |
There was a problem hiding this comment.
we seem to be using zap.S().Debugf and log.Debugf , is that intentional? Should we just use one?
There was a problem hiding this comment.
we should use log passed on the arguments. In practice it's the same thing, but zap.S() is just returning a top level default logger
dda0ae1 to
706638a
Compare
d6cbc89 to
692ad98
Compare
706638a to
d36531b
Compare
692ad98 to
6fa138b
Compare
d36531b to
427e0fd
Compare
fd1daca to
22f9e5a
Compare
427e0fd to
ad08030
Compare
22f9e5a to
23f196b
Compare
|
evergreen retry |
|
evergreen refresh |
6548799 to
e528c67
Compare
75e00dc to
4c82ef8
Compare
<!-- start git-machete generated --> # Based on PR #817 ## Chain of upstream PRs as of 2026-03-03 * PR #806: `master` ← `search/base` * PR #816: `search/base` ← `search/sharded-cluster` * PR #817: `search/sharded-cluster` ← `search/multiple-mongot` * **PR #853 (THIS ONE)**: `search/multiple-mongot` ← `search/lsierant/revert-rs-cluster-index` <!-- end git-machete generated -->
4c82ef8 to
f36f5bb
Compare
<!-- start git-machete generated --> # Based on PR #817 ## Chain of upstream PRs as of 2026-03-03 * PR #806: `master` ← `search/base` * PR #816: `search/base` ← `search/sharded-cluster` * PR #817: `search/sharded-cluster` ← `search/multiple-mongot` * **PR #853 (THIS ONE)**: `search/multiple-mongot` ← `search/lsierant/revert-rs-cluster-index` <!-- end git-machete generated -->
Add sharded cluster support to the MongoDBSearch operator: - Per-shard mongot StatefulSet, Service, and ConfigMap resources - ShardedEnterpriseSearchSource for operator-managed sharded clusters - ShardedExternalSearchSource for external sharded MongoDB sources - Per-shard and shared TLS certificate handling (CertsSecretPrefix) - Sharded cluster controller applies per-shard mongod and mongos search config parameters pointing to internal shard-local mongot services - ReplicaSet controller skips search config for sharded clusters (handled by sharded controller instead) - ExternalShardedConfig/ExternalRouterConfig/ExternalShardConfig API types - CRD updated with sharded external source fields and per-shard TLS prefix
Missing e2e test for sharded search with one mongot instance
# Summary
Addressing some of the review comments from the PR.
## Proof of Work
<!-- Enter your proof that it works here.-->
## Checklist
- [ ] Have you linked a jira ticket and/or is the ticket in the title?
- [ ] Have you checked whether your jira ticket required DOCSP changes?
- [ ] Have you added changelog file?
- use `skip-changelog` label if not needed
- refer to [Changelog files and Release
Notes](https://github.com/mongodb/mongodb-kubernetes/blob/master/CONTRIBUTING.md#changelog-files-and-release-notes)
section in CONTRIBUTING.md for more details
…loyment` (#832) # Summary <!-- Enter your PR summary here. Try to emphasize on WHY this change is needed, followed by what's being done in the PR. --> ## Proof of Work <!-- Enter your proof that it works here.--> ## Checklist - [ ] Have you linked a jira ticket and/or is the ticket in the title? - [ ] Have you checked whether your jira ticket required DOCSP changes? - [ ] Have you added changelog file? - use `skip-changelog` label if not needed - refer to [Changelog files and Release Notes](https://github.com/mongodb/mongodb-kubernetes/blob/master/CONTRIBUTING.md#changelog-files-and-release-notes) section in CONTRIBUTING.md for more details
…#834) ## Summary - Unify duplicate sharded vs non-sharded functions in the search controller into single parameterized versions - Extract shared config and service helpers (baseMongotConfig, mongotServicePorts, buildSearchSetParameters) - Fix missing owner references on non-sharded StatefulSet and Service (was already set on sharded path) ## Proof of Work Compiles, existing tests pass. Behavioral change is limited to adding `SetOwnerReference` on non-sharded resources (aligning with the sharded path). ## Checklist - [x] Have you linked a jira ticket and/or is the ticket in the title? - [x] Have you checked whether your jira ticket required DOCSP changes? - [x] Have you added changelog file? --------- Co-authored-by: Łukasz Sierant <lukasz.sierant@mongodb.com>
f36f5bb to
f36ca1e
Compare
<!-- start git-machete generated --> # Based on PR #817 ## Chain of upstream PRs as of 2026-03-03 * PR #806: `master` ← `search/base` * PR #816: `search/base` ← `search/sharded-cluster` * PR #817: `search/sharded-cluster` ← `search/multiple-mongot` * **PR #853 (THIS ONE)**: `search/multiple-mongot` ← `search/lsierant/revert-rs-cluster-index` <!-- end git-machete generated -->
Based on PR #806
Chain of upstream PRs as of 2026-03-03
PR Search Sharded+LoadBalancing: base feature branch #806:
master←search/basesearch/base←search/sharded-clusterSummary
MCK already supported deploying single instance of mongot process using MongoDBSearch resource, with the MongoDB replicaset as source. That meant the customers can run search queries/use search against a ReplicaSet deployment.
In this PR, we are adding support to have sharded cluster as source to the MongoDBSearch resource. So that search can be used with sharded clusters as well.
To achieve this we have added the new field in the
ExternalMongoDBSourcetype that can be used to configure the details of the sharded cluster that is going to be used as source.The
ExternalShardedClusterConfigis mainly used to generate the mongot config, so that mongot correctly knows how to talk to the mongod processes. Rest of the changes revolve around this.Proof of Work
TBD