Ai tier ga - #183
Conversation
fix: config changes for testing
…o run mirror registry
…egistry # Conflicts: # docs/deployment/k0s-quick-reference.md
fix: airgap - allowing model_staging as true, hostname fix & script t…
Adding sock proxy documentation
fix: Increase saia failure threshold
…k reference Clarify that L40S and H100 GPU worker specs are alternatives, not additive; add the repo clone step to Config Setup to match K0S_README's Quick Start; and drop the redundant macOS brew install snippet. Co-Authored-By: Claude <noreply@anthropic.com>
docs: clarify GPU hardware options and update clone steps in k0s quic…
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
k0s-cluster-config.yaml's operator image default moved to v2.4, but the test mirror registry helper still mirrored v2.3, so an air-gapped install requested an unmirrored image and hit ImagePullBackOff. Co-Authored-By: Claude <noreply@anthropic.com>
chore: bump splunk-ai-operator image to v2.4 in k0s cluster config
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: edd3e4fc25
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| local _accel | ||
| _accel=$(printf '%s' "${DEFAULT_ACCELERATOR}" | tr '[:upper:]' '[:lower:]') | ||
|
|
||
| if all_models_staged "${staging_dir}" "${_accel}"; then |
There was a problem hiding this comment.
Require verification tools before rejecting pre-staged models
When an air-gapped install uses storage.modelStaging.enabled=false, this treats every nonzero result from all_models_staged as proof that artifacts are missing. That helper also returns 1 when verification is merely unavailable—for example, the default MinIO path when mc is not installed, which preflight currently reports only as a warning, or AWS storage without the AWS CLI. Consequently, a fully populated object store cannot be used from an otherwise documented installer host; either make these clients mandatory or distinguish an unavailable check from genuinely missing markers.
Useful? React with 👍 / 👎.
| if [[ "${use_existing_cluster}" == "true" ]]; then | ||
| refresh_airgap_image_bundles_on_existing_nodes |
There was a problem hiding this comment.
Populate node arrays for the KUBECONFIG reuse path
For an air-gapped rerun detected through the KUBECONFIG branch, use_existing_cluster becomes true without ever populating CONTROLLER_IPS or WORKER_IPS; those arrays are only parsed in the SSH-detection and fresh-install branches. The new refresh therefore processes no nodes, and the following all-node verifier deterministically exits with “could not determine any cluster node IPs” whenever the staged bundle is present. Parse the configured IP strings before this refresh or derive the nodes for this reuse path.
Useful? React with 👍 / 👎.
| 4. Select the same VPC used when creating the security group. Every cluster node must have outbound internet connectivity, through an assigned public IPv4 | ||
| address. | ||
| 5. Enable a public IPv4 address on the MinIO/installer machine. It is the only | ||
| public IP required by this runbook. |
There was a problem hiding this comment.
Specify outbound connectivity consistently for every node
The launch instructions first require every cluster node to obtain outbound connectivity through an assigned public IPv4 address, but the very next item says to assign one only to the installer and calls it the only public IP required. In a VPC/subnet without a NAT gateway, following the latter instruction leaves the controller and workers unable to download the packages, images, k0s, and NVIDIA assets that this runbook explicitly requires. State either that every node needs a public address or document the private-subnet NAT route that makes installer-only public addressing valid.
Useful? React with 👍 / 👎.
| MC_HOST_check="http://${u}:${p}@127.0.0.1:9000" \ | ||
| mc stat check/ai-platform-bucket |
There was a problem hiding this comment.
Encode credentials before constructing the MinIO URL
When the chosen MinIO username or password contains URI delimiters such as @, :, /, ?, or #, interpolating it directly into MC_HOST_check changes how mc parses the authority or path, so this verification reports an authentication or endpoint failure even though the supplied credentials are valid. This is especially plausible for generated or policy-compliant passwords; use mc alias set with separate credential arguments or percent-encode the userinfo before building the URL.
Useful? React with 👍 / 👎.
| if [[ "${AIRGAP_MODE:-false}" == "true" ]]; then | ||
| echo " Air-gap mode: models must be staged from THIS machine (the cluster nodes have no internet)." >&2 | ||
| fi | ||
| echo " Do you want to download and stage model artifacts from HuggingFace now?" >&2 |
There was a problem hiding this comment.
Update the air-gap CLI help for automatic model staging
This newly offers model download and staging during the delegated air-gap install, but airgap_install.sh --help still states under “BEFORE YOU RUN” that model weights are not handled by the script and instructs users to stage them manually first. A user following the entry point's own help can therefore perform a separate 120 GB-plus download/upload workflow that the subsequent install now offers to perform itself; update the embedded help to describe the enabled/disabled configuration choice and pre-staged verification behavior.
Useful? React with 👍 / 👎.
Description
Related Issues
Type of Change
Changes Made
Testing Performed
make test)make lint)Test Environment
Test Steps
Documentation
Checklist
Breaking Changes
Impact:
Migration Path:
Screenshots/Recordings
Additional Notes
Reviewer Notes
Please pay special attention to:
Commit Message Convention: This PR follows Conventional Commits