Add nginx load balancing support for multi-instance Streamlit deployments (#336)

* Add nginx load balancer for scaling Streamlit in a single container

When STREAMLIT_SERVER_COUNT > 1, the entrypoint dynamically generates an
nginx config and launches multiple Streamlit instances on internal ports
(8510+), with nginx on port 8501 using ip_hash sticky sessions for
WebSocket compatibility. Default (STREAMLIT_SERVER_COUNT=1) preserves
existing behavior with no nginx overhead.

https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9

* Fix nginx config: create /etc/nginx directory before writing config

https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9

* Fix nginx: use absolute path /usr/sbin/nginx

The mamba environment activation shadows system binaries on the PATH.

https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9

* Switch nginx from ip_hash to least_conn for load balancing

ip_hash pins all users behind the same NAT/VPN/reverse-proxy to a
single backend, defeating the load balancer. least_conn distributes
new connections to the instance with fewest active connections, and
once a WebSocket is established it stays on that backend for the
session lifetime, so sticky sessions are not needed.

https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9

* Fix file uploads: disable nginx client_max_body_size limit

nginx defaults to 1MB max body size, which blocks Streamlit file
uploads with a 400 error. Set to 0 (unlimited) to let Streamlit
enforce its own 200MB limit from config.toml.

https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9

* Fix file uploads: switch to hash-based sticky sessions

least_conn routes each HTTP request independently, so the file upload
POST (/_stcore/upload_file) can land on a different backend than the
WebSocket session, causing a 400 error.

Use hash $remote_addr$http_x_forwarded_for consistent instead:
- Provides session affinity so uploads hit the correct backend
- Behind a reverse proxy: XFF header differentiates real client IPs
- Direct connections: falls back to remote_addr (like ip_hash)
- "consistent" minimizes redistribution when backends are added/removed

https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9

* Implement cookie-based sticky sessions for nginx load balancer

Replace ip_hash/hash-on-IP with cookie-based session affinity using
nginx's built-in map and $request_id:

- map $cookie_stroute $route_key: if browser has a "stroute" cookie,
  reuse its value; otherwise fall back to $request_id (a unique random
  hex string nginx generates per-request)
- hash $route_key consistent: route based on the cookie/random value
- add_header Set-Cookie on every response to persist the routing key

This ensures each browser gets its own sticky backend regardless of
source IP, fixing both:
- File uploads (POST must hit the same backend as the WebSocket session)
- Load distribution when all users share the same IP (NAT/VPN/proxy)

No new packages required - uses only built-in nginx directives.

https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9

---------

Co-authored-by: Claude <[email protected]>

Author: t0mdavid-m

Dockerfile

@@ -119,8 +119,8 @@ RUN rm -rf openms-build
 # Prepare and run streamlit app.
 FROM compile-openms AS run-app
 
-# Install Redis server for job queue
-RUN apt-get update && apt-get install -y --no-install-recommends redis-server \
+# Install Redis server for job queue and nginx for load balancing
+RUN apt-get update && apt-get install -y --no-install-recommends redis-server nginx \
     && rm -rf /var/lib/apt/lists/*
 
 # Create Redis data directory
@@ -154,6 +154,10 @@ RUN echo "0 3 * * * /root/miniforge3/envs/streamlit-env/bin/python /app/clean-up
 ENV RQ_WORKER_COUNT=1
 ENV REDIS_URL=redis://localhost:6379/0
 
+# Number of Streamlit server instances for load balancing (default: 1 = no load balancer)
+# Set to >1 to enable nginx load balancer with multiple Streamlit instances
+ENV STREAMLIT_SERVER_COUNT=1
+
 # create entrypoint script to start cron, Redis, RQ workers, and Streamlit
 RUN echo -e '#!/bin/bash\n\
 set -e\n\
@@ -180,9 +184,39 @@ for i in $(seq 1 $WORKER_COUNT); do\n\
     rq worker openms-workflows --url $REDIS_URL --name worker-$i &\n\
 done\n\
 \n\
-# Start Streamlit (foreground - main process)\n\
-echo "Starting Streamlit app..."\n\
-exec streamlit run app.py\n\
+# Load balancer setup\n\
+SERVER_COUNT=${STREAMLIT_SERVER_COUNT:-1}\n\
+\n\
+if [ "$SERVER_COUNT" -gt 1 ]; then\n\
+    echo "Starting $SERVER_COUNT Streamlit instances with nginx load balancer..."\n\
+\n\
+    # Generate nginx upstream block\n\
+    UPSTREAM_SERVERS=""\n\
+    BASE_PORT=8510\n\
+    for i in $(seq 0 $((SERVER_COUNT - 1))); do\n\
+        PORT=$((BASE_PORT + i))\n\
+        UPSTREAM_SERVERS="${UPSTREAM_SERVERS}        server 127.0.0.1:${PORT};\\n"\n\
+    done\n\
+\n\
+    # Write nginx config\n\
+    mkdir -p /etc/nginx\n\
+    echo -e "worker_processes auto;\\npid /run/nginx.pid;\\n\\nevents {\\n    worker_connections 1024;\\n}\\n\\nhttp {\\n    client_max_body_size 0;\\n\\n    map \\$cookie_stroute \\$route_key {\\n        \\x22\\x22      \\$request_id;\\n        default \\$cookie_stroute;\\n    }\\n\\n    upstream streamlit_backend {\\n        hash \\$route_key consistent;\\n${UPSTREAM_SERVERS}    }\\n\\n    map \\$http_upgrade \\$connection_upgrade {\\n        default upgrade;\\n        \\x27\\x27 close;\\n    }\\n\\n    server {\\n        listen 8501;\\n\\n        location / {\\n            proxy_pass http://streamlit_backend;\\n            proxy_http_version 1.1;\\n            proxy_set_header Upgrade \\$http_upgrade;\\n            proxy_set_header Connection \\$connection_upgrade;\\n            proxy_set_header Host \\$host;\\n            proxy_set_header X-Real-IP \\$remote_addr;\\n            proxy_set_header X-Forwarded-For \\$proxy_add_x_forwarded_for;\\n            proxy_set_header X-Forwarded-Proto \\$scheme;\\n            proxy_read_timeout 86400;\\n            proxy_send_timeout 86400;\\n            proxy_buffering off;\\n            add_header Set-Cookie \\x22stroute=\\$route_key; Path=/; HttpOnly; SameSite=Lax\\x22 always;\\n        }\\n    }\\n}" > /etc/nginx/nginx.conf\n\
+\n\
+    # Start Streamlit instances on internal ports (localhost only)\n\
+    for i in $(seq 0 $((SERVER_COUNT - 1))); do\n\
+        PORT=$((BASE_PORT + i))\n\
+        echo "Starting Streamlit instance on port $PORT..."\n\
+        streamlit run app.py --server.port $PORT --server.address 127.0.0.1 &\n\
+    done\n\
+\n\
+    sleep 2\n\
+    echo "Starting nginx load balancer on port 8501..."\n\
+    exec /usr/sbin/nginx -g "daemon off;"\n\
+else\n\
+    # Single instance mode (default) - run Streamlit directly on port 8501\n\
+    echo "Starting Streamlit app..."\n\
+    exec streamlit run app.py\n\
+fi\n\
 ' > /app/entrypoint.sh
 # make the script executable
 RUN chmod +x /app/entrypoint.sh

Dockerfile_simple

@@ -25,7 +25,7 @@ USER root
 
 RUN apt-get -y update
 # note: streamlit in docker needs libgtk2.0-dev (see https://yugdamor.medium.com/importerror-libgthread-2-0-so-0-cannot-open-shared-object-file-no-such-file-or-directory-895b94a7827b)
-RUN apt-get install -y --no-install-recommends --no-install-suggests wget ca-certificates libgtk2.0-dev curl jq cron
+RUN apt-get install -y --no-install-recommends --no-install-suggests wget ca-certificates libgtk2.0-dev curl jq cron nginx
 RUN update-ca-certificates
 
 # Install Github CLI
@@ -84,11 +84,52 @@ COPY clean-up-workspaces.py /app/clean-up-workspaces.py
 # add cron job to the crontab
 RUN echo "0 3 * * * /root/miniforge3/envs/streamlit-env/bin/python /app/clean-up-workspaces.py >> /app/clean-up-workspaces.log 2>&1" | crontab -
 
+# Number of Streamlit server instances for load balancing (default: 1 = no load balancer)
+# Set to >1 to enable nginx load balancer with multiple Streamlit instances
+ENV STREAMLIT_SERVER_COUNT=1
+
 # create entrypoint script to start cron service and launch streamlit app
-RUN echo "#!/bin/bash" > /app/entrypoint.sh
-RUN echo "source /root/miniforge3/bin/activate streamlit-env" >> /app/entrypoint.sh && \
-    echo "service cron start" >> /app/entrypoint.sh && \
-    echo "streamlit run app.py" >> /app/entrypoint.sh
+RUN echo -e '#!/bin/bash\n\
+set -e\n\
+source /root/miniforge3/bin/activate streamlit-env\n\
+\n\
+# Start cron for workspace cleanup\n\
+service cron start\n\
+\n\
+# Load balancer setup\n\
+SERVER_COUNT=${STREAMLIT_SERVER_COUNT:-1}\n\
+\n\
+if [ "$SERVER_COUNT" -gt 1 ]; then\n\
+    echo "Starting $SERVER_COUNT Streamlit instances with nginx load balancer..."\n\
+\n\
+    # Generate nginx upstream block\n\
+    UPSTREAM_SERVERS=""\n\
+    BASE_PORT=8510\n\
+    for i in $(seq 0 $((SERVER_COUNT - 1))); do\n\
+        PORT=$((BASE_PORT + i))\n\
+        UPSTREAM_SERVERS="${UPSTREAM_SERVERS}        server 127.0.0.1:${PORT};\\n"\n\
+    done\n\
+\n\
+    # Write nginx config\n\
+    mkdir -p /etc/nginx\n\
+    echo -e "worker_processes auto;\\npid /run/nginx.pid;\\n\\nevents {\\n    worker_connections 1024;\\n}\\n\\nhttp {\\n    client_max_body_size 0;\\n\\n    map \\$cookie_stroute \\$route_key {\\n        \\x22\\x22      \\$request_id;\\n        default \\$cookie_stroute;\\n    }\\n\\n    upstream streamlit_backend {\\n        hash \\$route_key consistent;\\n${UPSTREAM_SERVERS}    }\\n\\n    map \\$http_upgrade \\$connection_upgrade {\\n        default upgrade;\\n        \\x27\\x27 close;\\n    }\\n\\n    server {\\n        listen 8501;\\n\\n        location / {\\n            proxy_pass http://streamlit_backend;\\n            proxy_http_version 1.1;\\n            proxy_set_header Upgrade \\$http_upgrade;\\n            proxy_set_header Connection \\$connection_upgrade;\\n            proxy_set_header Host \\$host;\\n            proxy_set_header X-Real-IP \\$remote_addr;\\n            proxy_set_header X-Forwarded-For \\$proxy_add_x_forwarded_for;\\n            proxy_set_header X-Forwarded-Proto \\$scheme;\\n            proxy_read_timeout 86400;\\n            proxy_send_timeout 86400;\\n            proxy_buffering off;\\n            add_header Set-Cookie \\x22stroute=\\$route_key; Path=/; HttpOnly; SameSite=Lax\\x22 always;\\n        }\\n    }\\n}" > /etc/nginx/nginx.conf\n\
+\n\
+    # Start Streamlit instances on internal ports (localhost only)\n\
+    for i in $(seq 0 $((SERVER_COUNT - 1))); do\n\
+        PORT=$((BASE_PORT + i))\n\
+        echo "Starting Streamlit instance on port $PORT..."\n\
+        streamlit run app.py --server.port $PORT --server.address 127.0.0.1 &\n\
+    done\n\
+\n\
+    sleep 2\n\
+    echo "Starting nginx load balancer on port 8501..."\n\
+    exec /usr/sbin/nginx -g "daemon off;"\n\
+else\n\
+    # Single instance mode (default) - run Streamlit directly on port 8501\n\
+    echo "Starting Streamlit app..."\n\
+    exec streamlit run app.py\n\
+fi\n\
+' > /app/entrypoint.sh
 # make the script executable
 RUN chmod +x /app/entrypoint.sh
 

docker-compose.yml

@@ -12,6 +12,9 @@ services:
       - 8501:8501
     volumes:
       - workspaces-streamlit-template:/workspaces-streamlit-template
-    command: streamlit run openms-streamlit-template/app.py
+    environment:
+      # Number of Streamlit server instances (default: 1 = no load balancer).
+      # Set to >1 to enable nginx load balancing across multiple Streamlit instances.
+      - STREAMLIT_SERVER_COUNT=1
 volumes:
   workspaces-streamlit-template: