This repository was archived by the owner on Jan 19, 2026. It is now read-only.
Modernize #39
Open
Modernize #39
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✓ Profile falconeri-gcs-test already exists $ minikube status -p falconeri-gcs-test -o json ✓ falconeri-gcs-test is already running Setting kubectl context to test profile... $ kubectl config use-context falconeri-gcs-test Switched to context "falconeri-gcs-test". ✓ minikube setup complete⚠️ Using dedicated minikube profile 'falconeri-gcs-test' === Cleaning up resources from previous runs === $ kubectl config current-context ✓ Using correct context: falconeri-gcs-test Killing kubectl port-forward processes... $ pgrep -f kubectl port-forward Killing kubectl port-forward process 61293 $ kill 61293 Deleting old gcs-test jobs... $ kubectl get jobs -o name No old gcs-test jobs found Deleting old falconerid deployment... $ kubectl delete deployment falconerid --wait=false Deleted falconerid deployment Deleting old replica sets... $ kubectl delete replicaset -l app=falconerid --wait=false Deleting old secrets... $ kubectl delete secret gcs --wait=false $ kubectl delete secret gcs-credentials --wait=false Waiting for resources to be deleted... ✓ Cleanup complete === Uploading test data to gs://faraday-falconeri-test/falconeri-test-f00e2e116581/input/sample.txt === $ gsutil cp /Users/seamus/code/rust/falconeri/examples/gcs-test/sample.txt gs://faraday-falconeri-test/falconeri-test-f00e2e116581/input/sample.txt Copying file:///Users/seamus/code/rust/falconeri/examples/gcs-test/sample.txt [Content-Type=text/plain]... / [1 files][ 242.0 B/ 242.0 B] Operation completed over 1 objects/242.0 B. === Building falconeri Docker image === $ minikube image build -t falconeri-local:test -f Dockerfile . -p falconeri-gcs-test #0 building with "default" instance using docker driver #1 [internal] load build definition from Dockerfile #1 transferring dockerfile: 1.85kB done #1 DONE 0.0s #2 [internal] load metadata for docker.io/library/ubuntu:25.10 #2 DONE 0.3s #3 [internal] load .dockerignore #3 transferring context: 76B done #3 DONE 0.0s #4 [ 1/13] FROM docker.io/library/ubuntu:25.10@sha256:4a9232cc47bf99defcc8860ef6222c99773330367fcecbf21ba2edb0b810a31e #4 DONE 0.0s #5 [internal] load build context #5 transferring context: 304.85kB done #5 DONE 0.0s #6 [ 3/13] ADD https://storage.googleapis.com/kubernetes-release/release/v1.13.4/bin/linux/amd64/kubectl /usr/local/bin/kubectl #6 DONE 0.1s #7 [ 7/13] COPY falconeri ./falconeri #7 CACHED #8 [ 4/13] RUN chmod +x /usr/local/bin/kubectl #8 CACHED #9 [ 3/13] ADD https://storage.googleapis.com/kubernetes-release/release/v1.13.4/bin/linux/amd64/kubectl /usr/local/bin/kubectl #9 CACHED #10 [ 8/13] COPY falconerid ./falconerid #10 CACHED #11 [ 6/13] COPY Cargo.toml Cargo.lock ./ #11 CACHED #12 [ 9/13] COPY falconeri-worker ./falconeri-worker #12 CACHED #13 [ 5/13] WORKDIR /build #13 CACHED #14 [12/13] WORKDIR /app #14 CACHED #15 [11/13] RUN if [ "debug" = "release" ]; then cargo build --release --bin falconerid --bin falconeri-worker && cp target/release/falconerid target/release/falconeri-worker /usr/local/bin/; else cargo build --bin falconerid --bin falconeri-worker && cp target/debug/falconerid target/debug/falconeri-worker /usr/local/bin/; fi #15 CACHED #16 [ 2/13] RUN apt-get update && apt-get install -y curl build-essential libssl-dev pkg-config libpq-dev bash openssh-client git gnupg ca-certificates unzip && curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && unzip awscliv2.zip && ./aws/install && rm -rf awscliv2.zip aws && rm -rf /var/lib/apt/lists/* #16 CACHED #17 [10/13] COPY falconeri_common ./falconeri_common #17 CACHED #18 [13/13] ADD falconerid/Rocket.toml . #18 CACHED #19 exporting to image #19 exporting layers done #19 writing image sha256:61dc0e66ab00d9f84159bda9e9fcb04c925e671a201af109fe574142eb375af8 done #19 naming to docker.io/library/falconeri-local:test done #19 DONE 0.0s WARNING: current commit information was not captured by the build: git was not found in the system: exec: "git": executable file not found in $PATH ✓ Built falconeri-local:test in minikube === Creating Kubernetes secret === $ kubectl config current-context ✓ Using correct context: falconeri-gcs-test $ kubectl create secret generic gcs-credentials --from-literal=xxx secret/gcs-credentials created === Deploying falconeri infrastructure === $ kubectl config current-context ✓ Using correct context: falconeri-gcs-test $ cargo run -p falconeri -- deploy Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.61s Running `target/debug/falconeri deploy` persistentvolumeclaim/falconeri-postgres unchanged deployment.apps/falconeri-postgres unchanged service/falconeri-postgres unchanged deployment.apps/falconerid created service/falconerid unchanged serviceaccount/falconerid unchanged role.rbac.authorization.k8s.io/falconerid unchanged rolebinding.rbac.authorization.k8s.io/falconerid unchanged Updating deployment to use falconeri-local:test with imagePullPolicy: Never and 1 replica... $ kubectl patch deployment/falconerid --type json -p [{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value": "falconeri-local:test"}, {"op": "replace", "path": "/spec/template/spec/containers/0/imagePullPolicy", "value": "Never"}, {"op": "replace", "path": "/spec/replicas", "value": 1}] deployment.apps/falconerid patched Waiting for rollout to start... Cleaning up old replica sets... $ kubectl get replicaset -l app=falconerid -o name $ kubectl get replicaset.apps/falconerid-56c8976f5 -o jsonpath='{.spec.replicas}' Keeping active replica set: replicaset.apps/falconerid-56c8976f5 (replicas=1) $ kubectl get replicaset.apps/falconerid-56f674f58f -o jsonpath='{.spec.replicas}' Deleting old replica set: replicaset.apps/falconerid-56f674f58f $ kubectl delete replicaset.apps/falconerid-56f674f58f replicaset.apps "falconerid-56f674f58f" deleted from default namespace Waiting for falconeri deployments to be ready... $ kubectl wait --for=condition=available deployment/falconerid --timeout=300s deployment.apps/falconerid condition met $ kubectl wait --for=condition=available deployment/falconeri-postgres --timeout=300s deployment.apps/falconeri-postgres condition met ✓ Falconeri infrastructure is ready === Building test Docker image (with Rust compilation) === $ cp /Users/seamus/code/rust/falconeri/examples/word-frequencies/word-frequencies.sh /Users/seamus/code/rust/falconeri/word-frequencies.sh $ minikube image build -f examples/gcs-test/Dockerfile -t gcs-test . -p falconeri-gcs-test #0 building with "default" instance using docker driver #1 [internal] load build definition from Dockerfile #1 transferring dockerfile: 765B done #1 DONE 0.0s #2 [internal] load metadata for docker.io/library/rust:latest #2 DONE 0.2s #3 [internal] load metadata for docker.io/library/ubuntu:25.10 #3 DONE 0.2s #4 [internal] load .dockerignore #4 transferring context: 76B done #4 DONE 0.0s #5 [builder 1/9] FROM docker.io/library/rust:latest@sha256:65734d21f103d104fe0d9e508a424f7f60abd10e489d36de8bd36ae6c80e746d #5 DONE 0.0s #6 [stage-1 1/4] FROM docker.io/library/ubuntu:25.10@sha256:4a9232cc47bf99defcc8860ef6222c99773330367fcecbf21ba2edb0b810a31e #6 DONE 0.0s #7 [internal] load build context #7 transferring context: 305.24kB done #7 DONE 0.0s #8 [builder 7/9] COPY falconeri ./falconeri #8 CACHED #9 [stage-1 3/4] COPY --from=builder /build/target/debug/falconeri-worker /usr/local/bin/ #9 CACHED #10 [builder 5/9] COPY falconeri-worker ./falconeri-worker #10 CACHED #11 [builder 6/9] COPY falconeri_common ./falconeri_common #11 CACHED #12 [builder 3/9] WORKDIR /build #12 CACHED #13 [builder 4/9] COPY Cargo.toml Cargo.lock ./ #13 CACHED #14 [builder 9/9] RUN cargo build -p falconeri-worker #14 CACHED #15 [builder 2/9] RUN apt-get update && apt-get install -y libpq-dev && rm -rf /var/lib/apt/lists/* #15 CACHED #16 [builder 8/9] COPY falconerid ./falconerid #16 CACHED #17 [stage-1 2/4] RUN apt-get update && apt-get install -y ca-certificates libpq5 && apt-get clean && rm -rf /var/lib/apt/lists/* #17 CACHED #18 [stage-1 4/4] ADD word-frequencies.sh /usr/local/bin/ #18 CACHED #19 exporting to image #19 exporting layers done #19 writing image sha256:893bdc35473c487aae59d3507a82898337a97705f253230491dabf6bb729440c done #19 naming to docker.io/library/gcs-test done #19 DONE 0.0s WARNING: current commit information was not captured by the build: git was not found in the system: exec: "git": executable file not found in $PATH === Verifying Docker images === Verifying falconeri-local:test: Checking /usr/local/bin/falconerid in falconeri-local:test... $ minikube -p falconeri-gcs-test ssh -- docker run --rm falconeri-local:test file /usr/local/bin/falconerid ❌ Could not check /usr/local/bin/falconerid Checking /usr/local/bin/falconeri-worker in falconeri-local:test... $ minikube -p falconeri-gcs-test ssh -- docker run --rm falconeri-local:test file /usr/local/bin/falconeri-worker ❌ Could not check /usr/local/bin/falconeri-worker Checking /usr/local/bin/kubectl in falconeri-local:test... $ minikube -p falconeri-gcs-test ssh -- docker run --rm falconeri-local:test file /usr/local/bin/kubectl ❌ Could not check /usr/local/bin/kubectl Verifying gcs-test: Checking /usr/local/bin/falconeri-worker in gcs-test... $ minikube -p falconeri-gcs-test ssh -- docker run --rm gcs-test file /usr/local/bin/falconeri-worker ❌ Could not check /usr/local/bin/falconeri-worker Checking /usr/local/bin/word-frequencies.sh in gcs-test... $ minikube -p falconeri-gcs-test ssh -- docker run --rm gcs-test file /usr/local/bin/word-frequencies.sh ❌ Could not check /usr/local/bin/word-frequencies.sh⚠️ Some image verifications failed, but continuing... (This may be expected if architecture differs) === Generated pipeline spec at /Users/seamus/code/rust/falconeri/examples/gcs-test/gcs-test.json === === Starting falconeri proxy === $ cargo run -p falconeri -- proxy Waiting for proxy to be ready... Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.44s Running `target/debug/falconeri proxy` Forwarding from 127.0.0.1:8089 -> 8089 Forwarding from [::1]:8089 -> 8089 Forwarding from 127.0.0.1:5432 -> 5432 Forwarding from [::1]:5432 -> 5432 Handling connection for 8089 ✓ Proxy is ready === Running falconeri job === $ kubectl config current-context ✓ Using correct context: falconeri-gcs-test $ cargo run -p falconeri -- job run /Users/seamus/code/rust/falconeri/examples/gcs-test/gcs-test.json Handling connection for 8089 gcs-test-ph1viw9umy Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.20s Running `target/debug/falconeri job run /Users/seamus/code/rust/falconeri/examples/gcs-test/gcs-test.json` === Waiting for job gcs-test-ph1viw9umy to complete === $ kubectl config current-context ✓ Using correct context: falconeri-gcs-test $ cargo run -p falconeri -- job describe gcs-test-ph1viw9umy Handling connection for 5432 Job still running... (0s elapsed) $ cargo run -p falconeri -- job describe gcs-test-ph1viw9umy Handling connection for 5432 Job gcs-test-ph1viw9umy completed successfully! === Verifying output === $ gsutil ls gs://faraday-falconeri-test/falconeri-test-f00e2e116581/output/sample.txt Output file exists at gs://faraday-falconeri-test/falconeri-test-f00e2e116581/output/sample.txt $ gsutil cp gs://faraday-falconeri-test/falconeri-test-f00e2e116581/output/sample.txt /var/folders/y5/qt_wf5l52jd12gzd8dxfqpt40000gp/T/tmp7xqe3bh3 Copying gs://faraday-falconeri-test/falconeri-test-f00e2e116581/output/sample.txt... Downloading to temp gzip filename /var/folders/y5/qt_wf5l52jd12gzd8dxfqpt40000gp/T/tmp7xqe3bh3_.gztmp / [1 files][ 187.0 B/ 187.0 B] Operation completed over 1 objects/187.0 B. Output contents: 5 the 3 a 2 to 2 that 2 not 2 is 2 be 1 worm 1 words 1 with 1 thousand 1 than 1 step 1 speak 1 single 1 quick 1 question 1 over 1 or 1 of 1 miles 1 louder 1 lazy 1 jumps 1 journey 1 gold 1 glitters 1 fox 1 early 1 dog 1 catches 1 brown 1 bird 1 begins 1 all 1 actions Success! Output contains 36 lines of word frequencies === Cleaning up === Stopping proxy... Killing kubectl port-forward processes... $ pgrep -f kubectl port-forward Killing kubectl port-forward process 62054 $ kill 62054 Killing kubectl port-forward process 62057 $ kill 62057 Deleting GCS test files... $ gsutil -m rm -r gs://faraday-falconeri-test/falconeri-test-f00e2e116581/ If you experience problems with multiprocessing on MacOS, they might be related to https://bugs.python.org/issue33725. You can disable multiprocessing by editing your .boto config or by adding the following flag to your command: `-o "GSUtil:parallel_process_count=1"`. Note that multithreading is still available even if you disable multiprocessing. Removing gs://faraday-falconeri-test/falconeri-test-f00e2e116581/output/sample.txt#1767559295122682... Removing gs://faraday-falconeri-test/falconeri-test-f00e2e116581/input/sample.txt#1767559277510927... / [2/2 objects] 100% Done Operation completed over 2 objects. If you experience problems with multiprocessing on MacOS, they might be related to https://bugs.python.org/issue33725. You can disable multiprocessing by editing your .boto config or by adding the following flag to your command: `-o "GSUtil:parallel_process_count=1"`. Note that multithreading is still available even if you disable multiprocessing. Deleting k8s secrets... $ kubectl config current-context ✓ Using correct context: falconeri-gcs-test $ kubectl delete secret gcs Error from server (NotFound): secrets "gcs" not found $ kubectl delete secret gcs-credentials secret "gcs-credentials" deleted from default namespace ================================================== ✓ GCS integration test PASSED
This reverts commit c5d353a.
This reverts commit 8792213.
This reverts commit 49a6218.
This reverts commit 3fbc36a.
This reverts commit b893118.
This reverts commit ed4da0d.
This reverts commit f5f6e88.
This reverts commit 8016a24.
This reverts commit 216a525.
This reverts commit f468570.
This reverts commit e7bf5dc.
This reverts commit 1fa2256.
This reverts commit b93f9fb.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.