fix(edxapp): source Granian concurrency per install from stack config

docs/plans/granian-configuration-overhaul.md

@@ -2,8 +2,9 @@
 
 **Status:** stage 0 merged 2026-07-23 (#5083); stage 1 merged 2026-07-27 (#5135), validated
 in production 2026-08-07; stage 2 merged 2026-08-10 (#5344), validated in production
-2026-08-17; stage 3 `mitxonline` **blocked**, `edxapp` CMS rolled back and **blocked**
-pending retuning (see stage 3), LMS handled separately per install; stage 4 pending
+2026-08-17; stage 3 `mitxonline` **blocked**, `edxapp` LMS and CMS concurrency now
+sourced per install from stack config (see stage 3) -- `mitx`/`mitx-staging` tuned,
+`mitxonline`/`xpro` on holding pins; stage 4 pending
 **Project:** `wp-granian-configuration-overhaul-expose-blocking-t-3debc2`
 **Component:** `src/ol_infrastructure/components/services/k8s.py` — `GranianConfig`
 **Evidence:** witan lessons `les-granianconfig-never-exposes-blocking-threads-bac-874462`,
@@ -505,12 +506,31 @@ Component change lands once; per-app behavior changes as each app's stack is dep
   handling deliberately rather than discovering mid-incident
   (`tk-xpro-edxapp-is-6-months-stale-still-on-the-hand--43a5a4`).
 
-  **Current decision:** `edxapp` CMS must remain on its restored holding pins. The
-  production outcome above invalidates the earlier claim that CMS was a safe first
-  rollout despite having observable APISIX latency. Another attempt requires per-install
-  concurrency sizing that accounts for burst shape and backpressure, plus a scaling or
-  alerting signal that detects connection saturation; CPU and request rate alone did not.
-  LMS remains a separate per-install decision and does not make CMS unblocked.
+  **Current decision:** `edxapp` concurrency is per install, for CMS as well as LMS.
+  `k8s_resources.py` is shared by four installs, so a value written there is a value
+  written for all four at once -- which is how a default sized for a 0.25 rps authoring
+  instance reached `mitxonline` Studio. Both webapps now read
+  `edxapp:k8s_granian.{lms,cms}` from stack config, alongside `k8s_replicas` and
+  `k8s_resources`. The key is required rather than defaulted: an install's concurrency
+  should be readable in its own stack file, and a new install should have to state a
+  value rather than inherit whatever was last written into shared code.
+
+  | install | LMS | CMS |
+  | --- | --- | --- |
+  | `mitx`, `mitx-staging` | 1 × 8, backpressure 16 | 1 × 8, backpressure 16 |
+  | `mitxonline`, `xpro` | 2 × 32, backpressure 64 | 2 × 32, backpressure 64 |
+
+  This preserves every live value as of 2026-08-28 and reverses only the collateral half
+  of the rollback: #5607 restored the holding pins in shared code, which would also have
+  dragged `mitx` and `mitx-staging` CMS back to 2 workers on their next deploy. Both have
+  run 1 worker × 8 threads since 2026-08-24T17:18Z with a peak blocking queue of 5 (`mitx`)
+  and 0 (`mitx-staging`) and zero container restarts over 3 days; `mitx` CMS does touch its
+  backpressure ceiling of 16, for 52 minutes out of 3 days, which is the signal to watch if
+  its authoring traffic grows.
+
+  Raising `mitxonline` off the holding pins still requires concurrency sizing that accounts
+  for burst shape and backpressure, plus a scaling or alerting signal that detects
+  connection saturation; CPU and request rate alone did not.
 - **Stage 4 — async apps.** `mit_learn`, `learn_ai`: `workers=2→1` for `mit_learn` and an
   explicit `backpressure` for both. No `blocking_threads` involvement. Lowest expected
   impact, sequenced last because it shares no evidence with the WSGI stages.

src/ol_infrastructure/applications/edxapp/Pulumi.mitx-staging.CI.yaml

@@ -84,16 +84,21 @@ config:
   edxapp:backend_lms_domain: courses-backend-staging.ci.mitx.mit.edu
   edxapp:backend_studio_domain: studio-backend-staging.ci.mitx.mit.edu
   edxapp:backend_preview_domain: preview-staging.ci.mitx.mit.edu
-  # Stage 3 LMS defaults, following mitx's canary. Trivially loaded (~0.15 rps
-  # over 1-2 pods per pat-size-granian-blocking-threads-from-busy-thread-c-3a8b48),
-  # so no measurement wait is needed here. See
-  # docs/plans/granian-configuration-overhaul.md.
+  # Granian concurrency for the LMS and CMS webapps, per install: this stack's
+  # measured p99 demand is well under a ceiling of 8 busy blocking threads, where
+  # mitxonline needs 17.7 (LMS) and saturates 16 (CMS).
+  # See docs/plans/granian-configuration-overhaul.md.
   edxapp:k8s_granian:
     lms:
       workers: 1
       runtime_threads: 1
       blocking_threads: 8
       backpressure: 16
+    cms:
+      workers: 1
+      runtime_threads: 1
+      blocking_threads: 8
+      backpressure: 16
   edxapp:k8s_replicas:
     webapp:
       cms:

src/ol_infrastructure/applications/edxapp/Pulumi.mitx-staging.Production.yaml

@@ -84,16 +84,21 @@ config:
   edxapp:backend_lms_domain: courses-backend-staging.mitx.mit.edu
   edxapp:backend_studio_domain: studio-backend-staging.mitx.mit.edu
   edxapp:backend_preview_domain: preview-backend-staging.mitx.mit.edu
-  # Stage 3 LMS defaults, following mitx's canary. Trivially loaded (~0.15 rps
-  # over 1-2 pods per pat-size-granian-blocking-threads-from-busy-thread-c-3a8b48),
-  # so no measurement wait is needed here. See
-  # docs/plans/granian-configuration-overhaul.md.
+  # Granian concurrency for the LMS and CMS webapps, per install: this stack's
+  # measured p99 demand is well under a ceiling of 8 busy blocking threads, where
+  # mitxonline needs 17.7 (LMS) and saturates 16 (CMS).
+  # See docs/plans/granian-configuration-overhaul.md.
   edxapp:k8s_granian:
     lms:
       workers: 1
       runtime_threads: 1
       blocking_threads: 8
       backpressure: 16
+    cms:
+      workers: 1
+      runtime_threads: 1
+      blocking_threads: 8
+      backpressure: 16
   edxapp:k8s_replicas:
     webapp:
       cms:

src/ol_infrastructure/applications/edxapp/Pulumi.mitx-staging.QA.yaml

@@ -85,16 +85,21 @@ config:
   edxapp:backend_lms_domain: courses-backend-staging.qa.mitx.mit.edu
   edxapp:backend_studio_domain: studio-backend-staging.qa.mitx.mit.edu
   edxapp:backend_preview_domain: preview-staging.qa.mitx.mit.edu
-  # Stage 3 LMS defaults, following mitx's canary. Trivially loaded (~0.15 rps
-  # over 1-2 pods per pat-size-granian-blocking-threads-from-busy-thread-c-3a8b48),
-  # so no measurement wait is needed here. See
-  # docs/plans/granian-configuration-overhaul.md.
+  # Granian concurrency for the LMS and CMS webapps, per install: this stack's
+  # measured p99 demand is well under a ceiling of 8 busy blocking threads, where
+  # mitxonline needs 17.7 (LMS) and saturates 16 (CMS).
+  # See docs/plans/granian-configuration-overhaul.md.
   edxapp:k8s_granian:
     lms:
       workers: 1
       runtime_threads: 1
       blocking_threads: 8
       backpressure: 16
+    cms:
+      workers: 1
+      runtime_threads: 1
+      blocking_threads: 8
+      backpressure: 16
   edxapp:k8s_replicas:
     webapp:
       cms:

src/ol_infrastructure/applications/edxapp/Pulumi.mitx.CI.yaml

@@ -95,16 +95,21 @@ config:
   edxapp:backend_lms_domain: courses-backend.ci.mitx.mit.edu
   edxapp:backend_studio_domain: studio-backend.ci.mitx.mit.edu
   edxapp:backend_preview_domain: preview.ci.mitx.mit.edu
-  # Stage 3 canary for the LMS webapp. Omitting this key keeps the pre-overhaul
-  # holding pins (2 workers x 32 blocking threads); mitx opts in first because its
-  # measured p99 concurrency is 0.27 busy threads against this ceiling of 8, where
-  # mitxonline LMS needs 17.7. See docs/plans/granian-configuration-overhaul.md.
+  # Granian concurrency for the LMS and CMS webapps, per install: this stack's
+  # measured p99 demand is well under a ceiling of 8 busy blocking threads, where
+  # mitxonline needs 17.7 (LMS) and saturates 16 (CMS).
+  # See docs/plans/granian-configuration-overhaul.md.
   edxapp:k8s_granian:
     lms:
       workers: 1
       runtime_threads: 1
       blocking_threads: 8
       backpressure: 16
+    cms:
+      workers: 1
+      runtime_threads: 1
+      blocking_threads: 8
+      backpressure: 16
   edxapp:k8s_replicas:
     webapp:
       cms:

src/ol_infrastructure/applications/edxapp/Pulumi.mitx.Production.yaml

@@ -99,16 +99,21 @@ config:
   edxapp:backend_lms_domain: courses-backend.mitx.mit.edu
   edxapp:backend_studio_domain: studio-backend.mitx.mit.edu
   edxapp:backend_preview_domain: preview-backend.mitx.mit.edu
-  # Stage 3 canary for the LMS webapp. Omitting this key keeps the pre-overhaul
-  # holding pins (2 workers x 32 blocking threads); mitx opts in first because its
-  # measured p99 concurrency is 0.27 busy threads against this ceiling of 8, where
-  # mitxonline LMS needs 17.7. See docs/plans/granian-configuration-overhaul.md.
+  # Granian concurrency for the LMS and CMS webapps, per install: this stack's
+  # measured p99 demand is well under a ceiling of 8 busy blocking threads, where
+  # mitxonline needs 17.7 (LMS) and saturates 16 (CMS).
+  # See docs/plans/granian-configuration-overhaul.md.
   edxapp:k8s_granian:
     lms:
       workers: 1
       runtime_threads: 1
       blocking_threads: 8
       backpressure: 16
+    cms:
+      workers: 1
+      runtime_threads: 1
+      blocking_threads: 8
+      backpressure: 16
   edxapp:k8s_replicas:
     webapp:
       cms:

src/ol_infrastructure/applications/edxapp/Pulumi.mitx.QA.yaml

@@ -96,16 +96,21 @@ config:
   edxapp:backend_lms_domain: courses-backend.qa.mitx.mit.edu
   edxapp:backend_studio_domain: studio-backend.qa.mitx.mit.edu
   edxapp:backend_preview_domain: preview.qa.mitx.mit.edu
-  # Stage 3 canary for the LMS webapp. Omitting this key keeps the pre-overhaul
-  # holding pins (2 workers x 32 blocking threads); mitx opts in first because its
-  # measured p99 concurrency is 0.27 busy threads against this ceiling of 8, where
-  # mitxonline LMS needs 17.7. See docs/plans/granian-configuration-overhaul.md.
+  # Granian concurrency for the LMS and CMS webapps, per install: this stack's
+  # measured p99 demand is well under a ceiling of 8 busy blocking threads, where
+  # mitxonline needs 17.7 (LMS) and saturates 16 (CMS).
+  # See docs/plans/granian-configuration-overhaul.md.
   edxapp:k8s_granian:
     lms:
       workers: 1
       runtime_threads: 1
       blocking_threads: 8
       backpressure: 16
+    cms:
+      workers: 1
+      runtime_threads: 1
+      blocking_threads: 8
+      backpressure: 16
   edxapp:k8s_replicas:
     webapp:
       cms:

src/ol_infrastructure/applications/edxapp/Pulumi.mitxonline.CI.yaml

@@ -104,6 +104,23 @@ config:
   edxapp:autoscaling_lms_cpu_threshold: "70"
   edxapp:autoscaling_cms_requests_threshold: "20"
   edxapp:autoscaling_cms_cpu_threshold: "70"
+  # Granian concurrency for the LMS and CMS webapps, per install: pre-overhaul
+  # values. This install's traffic does not fit a ceiling of 8 busy blocking
+  # threads -- LMS p99 demand is 17.7, and CMS at backpressure 16 saturated in
+  # production on 2026-08-26. See docs/plans/granian-configuration-overhaul.md.
+  edxapp:k8s_granian:
+    lms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
+    cms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
   edxapp:k8s_replicas:
     webapp:
       lms:

src/ol_infrastructure/applications/edxapp/Pulumi.mitxonline.Production.yaml

@@ -103,6 +103,23 @@ config:
   edxapp:autoscaling_lms_cpu_threshold: "70"
   edxapp:autoscaling_cms_requests_threshold: "20"
   edxapp:autoscaling_cms_cpu_threshold: "70"
+  # Granian concurrency for the LMS and CMS webapps, per install: pre-overhaul
+  # values. This install's traffic does not fit a ceiling of 8 busy blocking
+  # threads -- LMS p99 demand is 17.7, and CMS at backpressure 16 saturated in
+  # production on 2026-08-26. See docs/plans/granian-configuration-overhaul.md.
+  edxapp:k8s_granian:
+    lms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
+    cms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
   edxapp:k8s_replicas:
     webapp:
       lms:

src/ol_infrastructure/applications/edxapp/Pulumi.mitxonline.QA.yaml

@@ -103,6 +103,23 @@ config:
   edxapp:autoscaling_lms_cpu_threshold: "70"
   edxapp:autoscaling_cms_requests_threshold: "20"
   edxapp:autoscaling_cms_cpu_threshold: "70"
+  # Granian concurrency for the LMS and CMS webapps, per install: pre-overhaul
+  # values. This install's traffic does not fit a ceiling of 8 busy blocking
+  # threads -- LMS p99 demand is 17.7, and CMS at backpressure 16 saturated in
+  # production on 2026-08-26. See docs/plans/granian-configuration-overhaul.md.
+  edxapp:k8s_granian:
+    lms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
+    cms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
   edxapp:k8s_replicas:
     webapp:
       lms:

src/ol_infrastructure/applications/edxapp/Pulumi.xpro.CI.yaml

@@ -66,6 +66,26 @@ config:
   edxapp:mail_domain: edxapp-mail-ci.xpro.mit.edu
   edxapp:sender_email_address: support@edxapp-mail-ci.xpro.mit.edu
   edxapp:enable_xqueue: "false"
+  # Granian concurrency for the LMS and CMS webapps, per install: pre-overhaul
+  # values, retained because xpro has no measurement to size them from. Its
+  # edxapp stack still runs the hand-rolled pre-OLApplicationK8s deployments
+  # (xpro-production-edxapp-{cms,lms}-webapp), so it never reaches this code
+  # path in production; the xpro-openedx namespace emits no granian_* series at
+  # all. Size these from xpro's own traffic once the stack deploys.
+  # See docs/plans/granian-configuration-overhaul.md.
+  edxapp:k8s_granian:
+    lms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
+    cms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
   edxapp:k8s_replicas:
     webapp:
       cms:

src/ol_infrastructure/applications/edxapp/Pulumi.xpro.Production.yaml

@@ -68,6 +68,26 @@ config:
   edxapp:backend_lms_domain: courses-backend.xpro.mit.edu
   edxapp:backend_studio_domain: studio-backend.xpro.mit.edu
   edxapp:backend_preview_domain: preview-backend.xpro.mit.edu
+  # Granian concurrency for the LMS and CMS webapps, per install: pre-overhaul
+  # values, retained because xpro has no measurement to size them from. Its
+  # edxapp stack still runs the hand-rolled pre-OLApplicationK8s deployments
+  # (xpro-production-edxapp-{cms,lms}-webapp), so it never reaches this code
+  # path in production; the xpro-openedx namespace emits no granian_* series at
+  # all. Size these from xpro's own traffic once the stack deploys.
+  # See docs/plans/granian-configuration-overhaul.md.
+  edxapp:k8s_granian:
+    lms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
+    cms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
   edxapp:k8s_replicas:
     webapp:
       cms:

src/ol_infrastructure/applications/edxapp/Pulumi.xpro.QA.yaml

@@ -69,6 +69,26 @@ config:
   edxapp:backend_lms_domain: courses-backend.rc.xpro.mit.edu
   edxapp:backend_studio_domain: studio-backend.rc.xpro.mit.edu
   edxapp:backend_preview_domain: preview-backend.rc.xpro.mit.edu
+  # Granian concurrency for the LMS and CMS webapps, per install: pre-overhaul
+  # values, retained because xpro has no measurement to size them from. Its
+  # edxapp stack still runs the hand-rolled pre-OLApplicationK8s deployments
+  # (xpro-production-edxapp-{cms,lms}-webapp), so it never reaches this code
+  # path in production; the xpro-openedx namespace emits no granian_* series at
+  # all. Size these from xpro's own traffic once the stack deploys.
+  # See docs/plans/granian-configuration-overhaul.md.
+  edxapp:k8s_granian:
+    lms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
+    cms:
+      workers: 2
+      runtime_mode: mt
+      runtime_threads: 2
+      blocking_threads: 32
+      backpressure: 64
   edxapp:k8s_replicas:
     webapp:
       cms: