Skip to content

Commit 0470fa1

Browse files
committed
cluster: migrate registry from host container to in-cluster deployment
Replace the shared host-side func-registry container with Kubernetes-native resources deployed inside each Kind cluster: - Deployment (registry:2 with hostPort 5000 + emptyDir volume) - ClusterIP Service (port 5000) - Contour Ingress at registry.localtest.me Key changes: - registryAddr is now "registry.localtest.me" (was "localhost:50000") - containerd mirrors point at http://localhost:5000 via hostPort (was http://func-registry:5000 via Docker network DNS) - Each cluster owns its own registry, destroyed with kind delete cluster - Delete flow simplified: no shared container teardown, just host trust revert on last-cluster removal - Removed: ensureRegistry, registryStatus, teardownRegistry, setupPodmanMacOSForwarding, and all host-container lifecycle code
1 parent da6327e commit 0470fa1

6 files changed

Lines changed: 247 additions & 189 deletions

File tree

CLAUDE.local.md

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
# Functions — Hugr Agent Instructions
2+
3+
Repository: https://github.com/knative/func
4+
5+
This file is kept in the hugr repo and symlinked into func src workspaces for
6+
agents.
7+
8+
## Testing
9+
10+
### Quick validation: `make check`
11+
12+
Run `make check` before signaling completion. It runs linting, vet, and
13+
template checks — fast enough to run on every change.
14+
15+
### Unit tests: `make test`
16+
17+
Run `make test` for unit-level validation. Sufficient for most changes —
18+
bug fixes, refactors, new functions with test coverage.
19+
20+
### Full E2E suite
21+
22+
The full E2E test suite (`make test-e2e`) runs against a live Kind cluster
23+
with a local registry, Knative Serving/Eventing, Tekton, and MetalLB. It
24+
deploys real functions, invokes them, and validates the full lifecycle.
25+
26+
**When to run locally:**
27+
- Debugging CI failures that don't reproduce with unit tests alone
28+
- Validating changes to cluster infrastructure (`pkg/cluster/`)
29+
- Changes to the deploy/build/push pipeline
30+
- Before pushing complex multi-file changes that touch the runtime path
31+
32+
**When it's unnecessary:**
33+
- Small, well-tested changes where `make test` passes
34+
- Documentation, template, or tooling changes
35+
- The full suite runs in remote CI on every PR — local runs are for
36+
replicability and confidence, not gating
37+
38+
**Prerequisites:**
39+
```
40+
export FUNC_ENABLE_CLUSTER=1
41+
export FUNC_E2E_KUBECONFIG=~/.config/func/clusters/func.local/kubeconfig.yaml
42+
```
43+
44+
Create the cluster first with `func cluster create --tekton` (requires the binary
45+
built and `FUNC_ENABLE_CLUSTER=1`). The E2E suite takes ~30 minutes.
46+
47+
## Landing flow
48+
49+
Work in your jj workspace (`$SHARD_ROOT/src/`). When ready to land:
50+
51+
1. Clean commits — squash fixups, reword for clarity
52+
2. Run `make check` (always) and `make test` (minimum)
53+
3. DM your requester for review, including test results
54+
4. After approval, push and open a PR:
55+
```
56+
jj git push -c @- # or whichever commit; creates a remote tracking branch
57+
```
58+
Then open a PR against upstream with a clear explanation of the change.
59+
5. PR must pass CI (unit tests, integration tests, linting)
60+
61+
## Conventions
62+
63+
- Follow upstream Go conventions (gofmt, govet, golint)
64+
- Tests are required for new functionality
65+
- Commit messages follow Conventional Commits format
66+
- Keep PRs focused — one logical change per PR
67+
68+
## Managing multiple changes
69+
70+
### Independent changes (simultaneous PRs)
71+
72+
Base each change on upstream's `main` so it can be pushed and PR'd
73+
independently, without waiting on other pending work. This avoids stalling
74+
when one PR is slow to merge.
75+
76+
```
77+
jj new main # start Feature A from main
78+
# ... work ...
79+
jj git push -c @- # push Feature A, open PR
80+
81+
jj new main # start Bugfix A from main (not from Feature A)
82+
# ... work ...
83+
jj git push -c @- # push Bugfix A, open separate PR
84+
85+
jj new main # start Feature C from main
86+
# ... work ...
87+
```
88+
89+
Each change gets its own bookmark (auto-created by `jj git push`) and its
90+
own PR. They can land in any order.
91+
92+
### Dependent changes (sequential PRs)
93+
94+
When a change depends on another that already has an open PR, make it a
95+
descendant of the prerequisite change. Do **not** push or open a PR until
96+
the prerequisite has been merged into `main`.
97+
98+
```
99+
jj new main # Feature A: independent, can PR now
100+
# ... work ...
101+
jj git push -c @- # push Feature A
102+
103+
jj new <feature-a> # Feature B: depends on A, descends from it
104+
# ... work ...
105+
# wait for Feature A to land before pushing Feature B
106+
```
107+
108+
A change may depend on multiple prerequisites — create it with multiple
109+
parents to reflect that. It should not be pushed until all prerequisites
110+
appear in `main`.
111+
112+
### Syncing after upstream merges
113+
114+
When a PR has been merged upstream (or when a rebase is required for it to
115+
land):
116+
117+
```
118+
jj git fetch --remote upstream
119+
jj rebase -d upstream/main # rebase working changes onto updated main
120+
```
121+
122+
Upstream uses merge commits, so your merged changes will appear as a new
123+
commit ID on `main`. Your local copies will show as "empty" (their diff is
124+
now included in the merge commit). At that point, those empty changes can
125+
be safely abandoned with `jj abandon`.
126+
127+
**Workspace hygiene:**
128+
- Only perform these operations in your own workspace
129+
- Only abandon changes that you authored — never touch other agents' work
130+
- Track change IDs and their push bookmarks on the associated issue for
131+
quick lookup
132+
133+
## Branch strategy
134+
135+
- Branches are auto-created by `jj git push` — no manual branch management
136+
- PRs target `main` unless otherwise specified

cmd/ci/workflow.go

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,14 @@ func createRuntimeTestStep(conf CIConfig, messageWriter io.Writer, steps []step)
8484
case "node", "typescript":
8585
testStep.withRun("npm ci && npm test")
8686
case "python":
87-
testStep.withRun("pip install . && python -m pytest")
87+
// Create the venv outside the project tree: PEP 668
88+
// (externally-managed) blocks pip against the system Python on
89+
// modern distros, so a venv is needed; but a venv inside the
90+
// project would seed it with absolute symlinks (the venv's
91+
// pip3 -> /usr/bin/python3.x etc.) which func deploy rejects
92+
// with "project may not contain absolute links". $RUNNER_TEMP
93+
// is set by both GitHub Actions and nektos/act.
94+
testStep.withRun(`python -m venv "$RUNNER_TEMP/venv" && "$RUNNER_TEMP/venv/bin/pip" install . && "$RUNNER_TEMP/venv/bin/python" -m pytest`)
8895
case "quarkus":
8996
testStep.withRun("./mvnw test")
9097
default:

cmd/config_ci_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -575,7 +575,7 @@ func TestNewConfigCICmd_TestStepPerRuntime(t *testing.T) {
575575
{
576576
name: "python runtime adds python -m pytest step",
577577
runtime: "python",
578-
expectedRun: "pip install . && python -m pytest",
578+
expectedRun: `python -m venv "$RUNNER_TEMP/venv" && "$RUNNER_TEMP/venv/bin/pip" install . && "$RUNNER_TEMP/venv/bin/python" -m pytest`,
579579
},
580580
{
581581
name: "quarkus runtime adds mvnw test step",

pkg/cluster/delete.go

Lines changed: 18 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -8,37 +8,29 @@ import (
88
"path/filepath"
99
)
1010

11-
// Delete removes a single func-managed dev cluster. The shared registry
12-
// container and the host's insecure-registries entry are removed only when
13-
// the *last* func-managed cluster is being torn down — other surviving
14-
// clusters keep using the shared registry.
11+
// Delete removes a func-managed dev cluster. The in-cluster registry is
12+
// destroyed automatically with the Kind cluster. Host-side trust config
13+
// (insecure-registries) is only reverted when this is the last func cluster,
14+
// since other surviving clusters share the same host entry.
1515
func Delete(ctx context.Context, cfg ClusterConfig, out io.Writer) error {
16-
// Set KUBECONFIG for child processes; restore the caller's value on return.
1716
defer setKubeconfig(cfg.Kubeconfig())()
1817

19-
status(out, "Deleting Cluster")
20-
21-
if err := run(ctx, out, "",
22-
cfg.kind(), "delete", "cluster",
23-
"--name="+cfg.Name,
24-
"--kubeconfig="+cfg.Kubeconfig()); err != nil {
25-
warnf(out, "failed to delete cluster %q: %v", cfg.Name, err)
18+
if _, err := os.Stat(cfg.Kubeconfig()); err == nil {
19+
status(out, "Deleting Cluster")
20+
if err := run(ctx, out, "",
21+
cfg.kind(), "delete", "cluster",
22+
"--name="+cfg.Name,
23+
"--kubeconfig="+cfg.Kubeconfig()); err != nil {
24+
warnf(out, "failed to delete cluster %q: %v", cfg.Name, err)
25+
}
26+
_ = os.RemoveAll(filepath.Dir(cfg.Kubeconfig()))
2627
}
2728

28-
// Remove this cluster's kubeconfig dir so the "last cluster?" check
29-
// below reflects the post-delete state.
30-
_ = os.RemoveAll(filepath.Dir(cfg.Kubeconfig()))
31-
32-
remaining := List()
33-
if len(remaining) == 0 {
34-
status(out, "Last func cluster removed; tearing down shared registry")
35-
teardownRegistry(ctx, cfg, out)
36-
if !cfg.SkipRegistryConfig {
37-
revertHostRegistry(out)
38-
}
39-
} else {
40-
fmt.Fprintf(out, "Registry left running; shared with %d other func-managed cluster(s): %v\n",
41-
len(remaining), remaining)
29+
if remaining := List(); len(remaining) > 0 {
30+
fmt.Fprintf(out, "Other func-managed cluster(s) still running: %v; leaving host registry config in place.\n",
31+
remaining)
32+
} else if !cfg.SkipRegistryConfig {
33+
revertHostRegistry(out)
4234
}
4335

4436
fmt.Fprintf(out, "%s Downloaded container images are not automatically removed.\n", red("NOTE:"))

pkg/cluster/kubernetes.go

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ const kindConfigTemplate = `kind: Cluster
1616
apiVersion: kind.x-k8s.io/v1alpha4
1717
nodes:
1818
- role: control-plane
19-
image: kindest/node:%[1]s
19+
image: kindest/node:%s
2020
extraPortMappings:
2121
- containerPort: 80
2222
hostPort: 80
@@ -29,14 +29,14 @@ nodes:
2929
listenAddress: "127.0.0.1"
3030
containerdConfigPatches:
3131
- |-
32-
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:%[2]d"]
33-
endpoint = ["http://%[3]s:%[4]d"]
34-
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.default.svc.cluster.local:%[4]d"]
35-
endpoint = ["http://%[3]s:%[4]d"]
32+
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.localtest.me"]
33+
endpoint = ["http://localhost:5000"]
34+
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.default.svc.cluster.local:5000"]
35+
endpoint = ["http://localhost:5000"]
3636
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."ghcr.io"]
37-
endpoint = ["http://%[3]s:%[4]d"]
37+
endpoint = ["http://localhost:5000"]
3838
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."quay.io"]
39-
endpoint = ["http://%[3]s:%[4]d"]
39+
endpoint = ["http://localhost:5000"]
4040
`
4141

4242
const metalLBPoolTemplate = `apiVersion: metallb.io/v1beta1
@@ -59,8 +59,7 @@ func installKubernetes(ctx context.Context, cfg ClusterConfig, out io.Writer) er
5959
start := time.Now()
6060
status(out, "Allocating")
6161

62-
kindConfig := fmt.Sprintf(kindConfigTemplate,
63-
kindNodeVersion, registryHostPort, registryContainerName, registryContainerPort)
62+
kindConfig := fmt.Sprintf(kindConfigTemplate, kindNodeVersion)
6463

6564
err := run(ctx, out, kindConfig,
6665
cfg.kind(), "create", "cluster",

0 commit comments

Comments
 (0)