Conversation
Signed-off-by: Daniel Gerlag <daniel@gerlag.ca>
Signed-off-by: Daniel Gerlag <daniel@gerlag.ca>
Signed-off-by: Daniel Gerlag <daniel@gerlag.ca>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| .read_kube_config() | ||
| .map_err(|e| anyhow::anyhow!("Failed to read k3s kubeconfig: {e}"))?; | ||
| kubeconfig = kubeconfig.replace( | ||
| "https://127.0.0.1:6443", |
Check notice
Code scanning / devskim
Accessing localhost could indicate debug code, or could hinder scaling. Note test
There was a problem hiding this comment.
This is test-only code (tests/k3s_helpers.rs) that connects to a testcontainers k3s instance on localhost. It rewrites the kubeconfig server address from the container-internal address to the mapped host port. This is inherent to how testcontainers-based integration tests work and is not debug or production code.
| .map_err(|e| anyhow::anyhow!("Failed to read k3s kubeconfig: {e}"))?; | ||
| kubeconfig = kubeconfig.replace( | ||
| "https://127.0.0.1:6443", | ||
| &format!("https://localhost:{host_port}"), |
Check notice
Code scanning / devskim
Accessing localhost could indicate debug code, or could hinder scaling. Note test
There was a problem hiding this comment.
Same as above — test-only localhost reference for k3s testcontainer port mapping.
| &format!("https://localhost:{host_port}"), | ||
| ); | ||
| kubeconfig = kubeconfig.replace( | ||
| "https://localhost:6443", |
Check notice
Code scanning / devskim
Accessing localhost could indicate debug code, or could hinder scaling. Note test
There was a problem hiding this comment.
Same — test-only localhost for k3s testcontainer.
| ); | ||
| kubeconfig = kubeconfig.replace( | ||
| "https://localhost:6443", | ||
| &format!("https://localhost:{host_port}"), |
Check notice
Code scanning / devskim
Accessing localhost could indicate debug code, or could hinder scaling. Note test
There was a problem hiding this comment.
Same — test-only localhost for k3s testcontainer.
|
|
||
| mkdir -p "$HOME/.kube" | ||
| docker cp "$K3S_CONTAINER:/etc/rancher/k3s/k3s.yaml" "$HOME/.kube/config" | ||
| sed -i 's/127.0.0.1/localhost/g' "$HOME/.kube/config" |
Check notice
Code scanning / devskim
Accessing localhost could indicate debug code, or could hinder scaling. Note
There was a problem hiding this comment.
This is in setup.sh for the getting-started example, which starts a local k3s container for development. The localhost reference is rewriting the kubeconfig to point at the container's mapped port. Not production code.
|
|
||
| mkdir -p "$HOME/.kube" | ||
| docker cp "$K3S_CONTAINER:/etc/rancher/k3s/k3s.yaml" "$HOME/.kube/config" | ||
| sed -i 's/127.0.0.1/localhost/g' "$HOME/.kube/config" |
Check notice
Code scanning / devskim
Accessing localhost could indicate debug code, or could hinder scaling. Note
There was a problem hiding this comment.
Same as above — example setup script localhost reference for local k3s container.
There was a problem hiding this comment.
Pull request overview
Adds Kubernetes integration to Drasi by introducing a Kubernetes source plugin plus a matching bootstrap provider, backed by a shared drasi-kubernetes-common crate (config/client/mapping/property extraction) to avoid duplicating logic between components.
Changes:
- Introduces
drasi-kubernetes-commonwith shared config, kube client creation, and object→graph mapping/property extraction. - Adds
drasi-source-kubernetesanddrasi-bootstrap-kubernetesplugins and registers them in the workspace. - Adds an end-to-end getting-started example and supporting scripts (plus a separate POC crate under
temp/).
Reviewed changes
Copilot reviewed 32 out of 32 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| Cargo.toml | Adds new Kubernetes crates to workspace members and workspace dependencies. |
| components/kubernetes-common/Cargo.toml | New shared Kubernetes crate manifest. |
| components/kubernetes-common/src/lib.rs | Exposes shared Kubernetes modules and re-exports key types/helpers. |
| components/kubernetes-common/src/config.rs | Defines Kubernetes source config, validation, and supported kinds. |
| components/kubernetes-common/src/client.rs | Builds kube client from kubeconfig content/path or in-cluster auth; parses apiVersion. |
| components/kubernetes-common/src/mapping.rs | Maps Kubernetes objects into Drasi SourceChanges (nodes + optional OWNS relations). |
| components/kubernetes-common/src/properties.rs | Extracts kind-specific and generic metadata properties into Drasi values. |
| components/sources/kubernetes/Cargo.toml | New Kubernetes source plugin manifest. |
| components/sources/kubernetes/Makefile | Build/lint/test targets for the Kubernetes source plugin. |
| components/sources/kubernetes/README.md | Documents Kubernetes source features, configuration, and dev workflows. |
| components/sources/kubernetes/src/lib.rs | Declares plugin modules, re-exports config/mapping/properties, exports dynamic plugin. |
| components/sources/kubernetes/src/descriptor.rs | Plugin descriptor + OpenAPI schema for Kubernetes source configuration. |
| components/sources/kubernetes/src/source.rs | Implements watcher-driven Kubernetes source and state-store UID tracking. |
| components/sources/kubernetes/src/tests.rs | Unit tests for config validation and property/mapping behavior. |
| components/sources/kubernetes/tests/k3s_helpers.rs | Testcontainers helper for spinning up k3s and producing a kube client/config. |
| components/sources/kubernetes/tests/integration_tests.rs | Ignored end-to-end test exercising CRUD cycle via k3s + DrasiLib. |
| components/bootstrappers/kubernetes/Cargo.toml | New Kubernetes bootstrap provider plugin manifest. |
| components/bootstrappers/kubernetes/Makefile | Build/lint/test targets for Kubernetes bootstrap provider. |
| components/bootstrappers/kubernetes/README.md | Documents bootstrap provider behavior and usage. |
| components/bootstrappers/kubernetes/src/lib.rs | Declares bootstrap modules and exports dynamic plugin. |
| components/bootstrappers/kubernetes/src/config.rs | Defines (empty) bootstrap config type. |
| components/bootstrappers/kubernetes/src/descriptor.rs | Bootstrap plugin descriptor + OpenAPI schema for bootstrap config. |
| components/bootstrappers/kubernetes/src/provider.rs | Lists objects and emits bootstrap insert events based on source config. |
| examples/lib/kubernetes-getting-started/Cargo.toml | Example crate manifest wiring DrasiLib + k8s source/bootstrap + log reaction. |
| examples/lib/kubernetes-getting-started/main.rs | Example app showing query + log reaction over watched ConfigMaps. |
| examples/lib/kubernetes-getting-started/README.md | Getting-started documentation for running the example. |
| examples/lib/kubernetes-getting-started/setup.sh | Script to ensure a reachable cluster (starts local k3s container if needed). |
| examples/lib/kubernetes-getting-started/quickstart.sh | Runs setup then executes the example. |
| examples/lib/kubernetes-getting-started/test-updates.sh | Applies/patches/deletes a demo ConfigMap to trigger updates. |
| examples/lib/kubernetes-getting-started/diagnose.sh | Cluster + demo resource diagnostics. |
| temp/kubernetes-poc-verification/Cargo.toml | Standalone POC crate manifest for kube-rs watcher experiments. |
| temp/kubernetes-poc-verification/src/main.rs | Standalone POC program demonstrating watcher init/apply/delete behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| [package] | ||
| name = "drasi-source-kubernetes" | ||
| version = "0.1.0" | ||
| edition = "2021" | ||
| authors = ["Drasi Project"] | ||
| description = "Kubernetes source plugin for Drasi" | ||
| license = "Apache-2.0" | ||
| repository = "https://github.com/drasi-project/drasi-core" |
There was a problem hiding this comment.
This Cargo.toml is missing the standard Apache-2.0 copyright/license header comment block that other plugin crates include at the top (e.g., components/sources/http/Cargo.toml:1-13). Add the header here for consistency across plugin manifests.
There was a problem hiding this comment.
Fixed — added the standard Apache-2.0 copyright/license header.
| [package] | ||
| name = "drasi-bootstrap-kubernetes" | ||
| version = "0.1.0" | ||
| edition = "2021" | ||
| authors = ["Drasi Project"] | ||
| description = "Kubernetes bootstrap plugin for Drasi" | ||
| license = "Apache-2.0" | ||
| repository = "https://github.com/drasi-project/drasi-core" |
There was a problem hiding this comment.
This Cargo.toml is missing the standard Apache-2.0 copyright/license header comment block that other plugin crates include at the top (e.g., components/sources/http/Cargo.toml:1-13). Add the header here for consistency across plugin manifests.
There was a problem hiding this comment.
Fixed — added the standard Apache-2.0 copyright/license header.
| async fn create_bootstrap_provider( | ||
| &self, | ||
| config_json: &serde_json::Value, | ||
| source_config_json: &serde_json::Value, | ||
| ) -> anyhow::Result<Box<dyn BootstrapProvider>> { | ||
| let bootstrap_config: KubernetesBootstrapConfig = | ||
| serde_json::from_value(config_json.clone()).unwrap_or_default(); | ||
| let source_config: KubernetesSourceConfig = | ||
| serde_json::from_value(source_config_json.clone())?; | ||
| source_config.validate()?; |
There was a problem hiding this comment.
create_bootstrap_provider currently uses serde_json::from_value(...).unwrap_or_default(), which silently ignores invalid bootstrap config JSON. Since this config is intentionally empty, prefer strict deserialization (and add deny_unknown_fields on the DTO) so unexpected fields or wrong shapes fail fast instead of being dropped.
There was a problem hiding this comment.
Fixed — changed to strict deserialization with ? operator and added deny_unknown_fields on the DTO.
| init_done.insert(target_key, true); | ||
| // Checkpoint: persist seen_uids after full init list | ||
| save_seen_uids(source_id, &state_store, &seen_uids).await?; | ||
| save_initialized(source_id, &state_store, true).await?; |
There was a problem hiding this comment.
save_initialized(...) persists an initialized flag to the state store, but nothing ever reads it back (no get(..., INITIALIZED_KEY) anywhere). Either load and use this flag (e.g., for resume/skip-init decisions) or remove it to avoid writing unused state.
| save_initialized(source_id, &state_store, true).await?; |
There was a problem hiding this comment.
Fixed — removed save_initialized and the INITIALIZED_KEY constant entirely. The watcher always re-lists on startup via kube-rs, so persisted init state was never actually used.
| Event::InitApply(obj) => { | ||
| // Always track UIDs for insert-vs-update classification, | ||
| // even when not dispatching events | ||
| if let Some(uid) = extract_uid(&obj) { | ||
| seen_uids.insert(uid); | ||
| } | ||
| if matches!(config.start_from, StartFrom::Now) { | ||
| continue; | ||
| } | ||
| if let StartFrom::Timestamp(ts) = config.start_from { | ||
| if let Some(created_at) = object_created_at_millis(&obj) { | ||
| if created_at < ts { | ||
| continue; | ||
| } | ||
| } | ||
| } | ||
| process_apply_object( | ||
| source_id, | ||
| &config, | ||
| &target, | ||
| obj, | ||
| &dispatchers, | ||
| &mut seen_uids, | ||
| ).await?; | ||
| } |
There was a problem hiding this comment.
Event::InitApply pre-inserts the object's UID into seen_uids before calling process_apply_object. This makes every InitApply object look "already seen" and therefore gets emitted as an Update instead of an Insert when start_from is Beginning/Timestamp. Move the seen_uids.insert(uid) into the branches where you skip dispatching (e.g., StartFrom::Now or when filtering out by timestamp), and let process_apply_object own the insert/update decision for dispatched InitApply events.
There was a problem hiding this comment.
Fixed — UID is now only pre-inserted in the skip branches (StartFrom::Now and timestamp-filtered). Dispatched InitApply events go through process_apply_object which correctly decides insert vs update.
| use kube::api::{Api, DynamicObject}; | ||
| use kube::core::{ApiResource, GroupVersionKind}; | ||
| use kube::runtime::watcher::{self, Event}; | ||
| use kube::Client; |
There was a problem hiding this comment.
use kube::Client; is unused in this file. With workspace lints set to deny warnings, this will fail compilation; remove the import or use it explicitly.
| use kube::Client; |
There was a problem hiding this comment.
Fixed — removed the unused kube::Client import.
| build_client, is_cluster_scoped_kind, parse_api_version, AuthMode, KubernetesSourceConfig, | ||
| ResourceSpec, |
There was a problem hiding this comment.
AuthMode is imported but never used in this module. With workspace lints set to deny warnings, this will fail compilation; remove the unused import.
| build_client, is_cluster_scoped_kind, parse_api_version, AuthMode, KubernetesSourceConfig, | |
| ResourceSpec, | |
| build_client, is_cluster_scoped_kind, parse_api_version, KubernetesSourceConfig, ResourceSpec, |
There was a problem hiding this comment.
Fixed — removed unused AuthMode import.
| mkdir -p "$HOME/.kube" | ||
| docker cp "$K3S_CONTAINER:/etc/rancher/k3s/k3s.yaml" "$HOME/.kube/config" | ||
| sed -i 's/127.0.0.1/localhost/g' "$HOME/.kube/config" | ||
| export KUBECONFIG="$HOME/.kube/config" | ||
|
|
||
| echo "Waiting for node readiness..." | ||
| kubectl wait --for=condition=Ready node --all --timeout=120s | ||
| kubectl get nodes -o wide | ||
|
|
||
| echo "Setup complete." | ||
| echo "If needed, export: KUBECONFIG=$HOME/.kube/config" |
There was a problem hiding this comment.
setup.sh overwrites the user's $HOME/.kube/config when starting the local k3s container. This can destroy an existing kubeconfig and is risky for users. Prefer writing the generated kubeconfig to a dedicated file under the example directory (or a temp dir) and setting KUBECONFIG for the remainder of the script run instead of overwriting the default config path.
| mkdir -p "$HOME/.kube" | |
| docker cp "$K3S_CONTAINER:/etc/rancher/k3s/k3s.yaml" "$HOME/.kube/config" | |
| sed -i 's/127.0.0.1/localhost/g' "$HOME/.kube/config" | |
| export KUBECONFIG="$HOME/.kube/config" | |
| echo "Waiting for node readiness..." | |
| kubectl wait --for=condition=Ready node --all --timeout=120s | |
| kubectl get nodes -o wide | |
| echo "Setup complete." | |
| echo "If needed, export: KUBECONFIG=$HOME/.kube/config" | |
| K3S_KUBECONFIG="$SCRIPT_DIR/k3s-kubeconfig" | |
| docker cp "$K3S_CONTAINER:/etc/rancher/k3s/k3s.yaml" "$K3S_KUBECONFIG" | |
| sed -i 's/127.0.0.1/localhost/g' "$K3S_KUBECONFIG" | |
| export KUBECONFIG="$K3S_KUBECONFIG" | |
| echo "Waiting for node readiness..." | |
| kubectl wait --for=condition=Ready node --all --timeout=120s | |
| kubectl get nodes -o wide | |
| echo "Setup complete." | |
| echo "If needed, export: KUBECONFIG=$K3S_KUBECONFIG" |
There was a problem hiding this comment.
Fixed — kubeconfig is now written to a local file at the script directory instead of overwriting ~/.kube/config.
| /// POC: kube-rs watcher capabilities for Drasi Kubernetes source | ||
| use anyhow::Result; | ||
| use futures::StreamExt; | ||
| use k8s_openapi::api::core::v1::ConfigMap; | ||
| use kube::{ |
There was a problem hiding this comment.
The PR description focuses on adding Kubernetes source/bootstrap components, but this adds a separate temp/kubernetes-poc-verification POC workspace/crate. If this is intended to ship with the PR, it should be mentioned; otherwise consider moving it under examples/ or omitting it to keep the repository clean.
There was a problem hiding this comment.
Removed — temp/kubernetes-poc-verification has been deleted from the PR.
| if let Some(rv) = extract_resource_version(&obj) { | ||
| save_resource_version(source_id, &target_key, &rv, &state_store).await?; | ||
| } | ||
| } | ||
| } | ||
| } | ||
| Err(e) => { |
There was a problem hiding this comment.
save_resource_version(...) writes rv:{target_key} to the state store, but the value is never loaded/used to resume the watcher, and it's only updated on Delete events. Either implement resume logic (load RV per target and pass it into the watcher config / restart strategy, and update it on Apply as well) or remove this persistence to avoid dead state.
| if let Some(rv) = extract_resource_version(&obj) { | |
| save_resource_version(source_id, &target_key, &rv, &state_store).await?; | |
| } | |
| } | |
| } | |
| } | |
| Err(e) => { | |
| } | |
| } | |
| } | |
| Err(e) => { | |
| } | |
| } | |
| Err(e) => { |
There was a problem hiding this comment.
Fixed — removed save_resource_version entirely. The kube-rs watcher handles resourceVersion tracking internally, so persisting it without loading it on resume was dead state.
Signed-off-by: Daniel Gerlag <daniel@gerlag.ca>
|
|
||
| K3S_KUBECONFIG="$SCRIPT_DIR/k3s-kubeconfig" | ||
| docker cp "$K3S_CONTAINER:/etc/rancher/k3s/k3s.yaml" "$K3S_KUBECONFIG" | ||
| sed -i 's/127.0.0.1/localhost/g' "$K3S_KUBECONFIG" |
Check notice
Code scanning / devskim
Accessing localhost could indicate debug code, or could hinder scaling. Note
|
|
||
| K3S_KUBECONFIG="$SCRIPT_DIR/k3s-kubeconfig" | ||
| docker cp "$K3S_CONTAINER:/etc/rancher/k3s/k3s.yaml" "$K3S_KUBECONFIG" | ||
| sed -i 's/127.0.0.1/localhost/g' "$K3S_KUBECONFIG" |
Check notice
Code scanning / devskim
Accessing localhost could indicate debug code, or could hinder scaling. Note
danielgerlag
requested review from
agentofreality,
amansinghoriginal and
ruokun-niu
This pull request adds support for Kubernetes as a source and bootstrap provider in the project. It introduces new shared components for Kubernetes, a Kubernetes bootstrap plugin, and updates the workspace configuration to include these new components. The main focus is on enabling the system to bootstrap data from Kubernetes resources, using configuration shared with the Kubernetes source plugin.
Kubernetes plugin integration:
drasi-kubernetes-commonfor Kubernetes-related types and client utilities, facilitating code reuse between plugins. [1] [2]drasi-bootstrap-kubernetesplugin, including its implementation (src/lib.rs,src/config.rs,src/descriptor.rs,src/provider.rs), manifest (Cargo.toml), build/test automation (Makefile), and documentation (README.md). This plugin emits bootstrap events for Kubernetes resources based on the source configuration. [1] [2] [3] [4] [5] [6]Workspace and dependency updates:
components/kubernetes-common,components/bootstrappers/kubernetes, andcomponents/sources/kubernetesas workspace members inCargo.toml. [1] [2] [3]