Skip to content
This repository was archived by the owner on May 21, 2026. It is now read-only.

Replace fixed CI waits with Kubernetes rollout-based deployment checks - #1485

Open
MerwinJoshwa wants to merge 2 commits into
fossasia:developmentfrom
MerwinJoshwa:b3
Open

Replace fixed CI waits with Kubernetes rollout-based deployment checks#1485
MerwinJoshwa wants to merge 2 commits into
fossasia:developmentfrom
MerwinJoshwa:b3

Conversation

@MerwinJoshwa

@MerwinJoshwa MerwinJoshwa commented Feb 19, 2026

Copy link
Copy Markdown

Fixes #1470

Changes:

  • Removed reliance on long static CI waits during deployment.
  • Improved deployment logic to rely on Kubernetes readiness instead of fixed time delays.
  • Reduced unnecessary CI blocking that caused builds to run for ~30 minutes.

This change prevents Travis builds from being blocked while waiting for resources that are already handled by Kubernetes.

Summary by Sourcery

Enhancements:

  • Add a Kubernetes rollout status wait loop with timeout and diagnostics to the Travis deployment script to avoid long static delays.

@sourcery-ai

sourcery-ai Bot commented Feb 19, 2026

Copy link
Copy Markdown

Reviewer's Guide

CI Kubernetes deployment script is updated to wait for actual rollout completion using kubectl rollout status with a bounded timeout and diagnostic output instead of relying on fixed time delays, reducing unnecessary CI blocking.

Flow diagram for rollout-based deployment script logic

flowchart TD
    A[Start deploy_sh] --> B[Push Docker image fossasia/susi_server:TRAVIS_COMMIT]
    B --> C[Run kubectl set image on deployment susi-server in namespace web]
    C --> D[Initialize DEPLOYMENT, NAMESPACE, TIMEOUT, INTERVAL, SECONDS_WAITED=0]
    D --> E{kubectl rollout status
    deployment/$DEPLOYMENT -n $NAMESPACE
    success?}

    E -- Yes --> F[Echo Rollout successful]
    F --> G[Exit 0]

    E -- No --> H[Sleep INTERVAL
    SECONDS_WAITED += INTERVAL]
    H --> I{SECONDS_WAITED >= TIMEOUT?}

    I -- No --> E

    I -- Yes --> J[Echo Rollout timeout exceeded]
    J --> K[kubectl get pods -n $NAMESPACE]
    K --> L[kubectl describe deployment $DEPLOYMENT -n $NAMESPACE]
    L --> M[Exit 1]
Loading

File-Level Changes

Change Details Files
Replace static post-deploy waits with a rollout-status loop driven by Kubernetes readiness.
  • Update the deployment image via kubectl set image before waiting for rollout completion.
  • Introduce configuration variables for deployment name, namespace, timeout, and polling interval in the deploy script.
  • Add a loop calling kubectl rollout status with periodic sleeps until success or timeout.
  • On timeout, print pod listing and deployment description for debugging and exit with a non-zero status.
  • Emit clear log messages indicating rollout waiting and success.
kubernetes/travis/deploy.sh

Assessment against linked issues

Issue Objective Addressed Explanation
#1470 Remove long fixed time waits (e.g., 30-minute sleeps) from the CI deployment script that cause builds to run for more than 30 minutes.
#1470 Introduce a more efficient readiness/deployment check (instead of static waits) that relies on Kubernetes or similar mechanisms, so CI builds are not needlessly blocked and stay within Travis time limits.

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • The kubectl set image line appears malformed with stray backslashes and missing spacing (e.g., \ --namespace=web \susi-server=...); consider simplifying it to a single line or using properly spaced line continuations so the command runs as intended.
  • You enable set -e only after kubectl set image, so a failed image update would not stop the script; consider moving set -e near the top of the script so all deployment steps fail fast.
  • Since kubectl rollout status supports a --timeout flag, you could potentially replace the custom sleep/loop logic with a single kubectl rollout status deployment/$DEPLOYMENT -n $NAMESPACE --timeout=${TIMEOUT}s to simplify the rollout-wait logic while preserving a bounded wait.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `kubectl set image` line appears malformed with stray backslashes and missing spacing (e.g., `\ --namespace=web \susi-server=...`); consider simplifying it to a single line or using properly spaced line continuations so the command runs as intended.
- You enable `set -e` only after `kubectl set image`, so a failed image update would not stop the script; consider moving `set -e` near the top of the script so all deployment steps fail fast.
- Since `kubectl rollout status` supports a `--timeout` flag, you could potentially replace the custom sleep/loop logic with a single `kubectl rollout status deployment/$DEPLOYMENT -n $NAMESPACE --timeout=${TIMEOUT}s` to simplify the rollout-wait logic while preserving a bounded wait.

## Individual Comments

### Comment 1
<location> `kubernetes/travis/deploy.sh:41` </location>
<code_context>

 echo ">>> Updating deployment"
-kubectl set image deployment/susi-server --namespace=web susi-server=fossasia/susi_server:$TRAVIS_COMMIT
+kubectl set image deployment/susi-server \ --namespace=web \susi-server=fossasia/susi_server:$TRAVIS_COMMIT
+
+echo ">>> Waiting for Kubernetes rollout to complete"
</code_context>

<issue_to_address>
**issue (bug_risk):** Fix line continuation and spacing in the `kubectl set image` command.

The added backslashes are incorrectly spaced: `deployment/susi-server \ --namespace=web \susi-server=...` will be parsed as a single line with a literal `\susi-server` token. Update either to a single line or to a proper multiline form, e.g.

```sh
kubectl set image deployment/susi-server \
  --namespace=web \
  susi-server=fossasia/susi_server:$TRAVIS_COMMIT
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread kubernetes/travis/deploy.sh Outdated
@MerwinJoshwa

Copy link
Copy Markdown
Author

Addresses the root cause of prolonged Travis CI runtimes by replacing fixed wait-based deployment behavior with deterministic execution flow.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Builds run for more than 30 minutes

1 participant