evaluateHealth measures memory severity against heapTotal, which is the heap V8 has already committed, not the ceiling it is allowed to grow to. V8 sizes heapTotal to demand, so a busy healthy process sits near 100% of it more or less permanently.
src/health/thresholds.ts:62-65
const memPercent =
snapshot.memory.heapTotal > 0
? (snapshot.memory.heapUsed / snapshot.memory.heapTotal) * 100
: 0;
Since heapTotal <= heap_size_limit always holds, this can only ever over-report severity, never under-report it.
What I saw
On a live deployment:
|
bytes |
heapUsed |
390,771,624 |
heapTotal |
404,930,560 |
rss |
584,327,168 |
v8.getHeapStatistics().heap_size_limit |
6,492,782,592 (6192 MB) |
That is 6.0% of the heap the process may actually use. The alert read memory_critical_97%_rss557mb and status was critical.
GET /agentmemory/health then returned HTTP 503, because src/triggers/api.ts:274 maps critical to 503:
const statusCode = status === "critical" ? 503 : 200;
/agentmemory/livez returned 200 at the same moment. So an uptime monitor pointed at /health reads the service as down while it is serving normally. At that point the container was using 3.0 GB of an 8 GB limit, the circuit breaker was closed, CPU was 3.4%, and event loop lag was 0.1 ms.
Worth noting for anyone deploying from this repo: deploy/railway/railway.json sets healthcheckPath to /agentmemory/livez, so Railway's own check keeps passing and the service never restart-loops. The shipped template dodges this by accident. It is the richer /agentmemory/health endpoint, the one you would point an external monitor at, that reports the false 503.
What I expected
healthy, and HTTP 200. Six percent of the heap limit is not memory pressure.
How this relates to #158
#158 reported the same formula misfiring at heapUsed ~45 MB, heapTotal ~46 MB, and rss ~120 MB. The fix that landed added memoryRssFloorBytes (512 MB, src/health/thresholds.ts:20) so the alert stays quiet on small processes. That covers the case as reported, but the ratio itself was never corrected, so the bug comes back for any process whose RSS clears the floor.
There is also no way to tune around it from the outside. src/health/monitor.ts:87 calls evaluateHealth(snapshot) with no config argument, so DEFAULTS always wins and neither the percentages nor the RSS floor are reachable from the environment.
Repro
- Run agentmemory under a workload that keeps RSS above 512 MB.
curl -si "$AGENTMEMORY_URL/agentmemory/health" | head -1
- You get 503, with
memory_critical_NN% in health.alerts, while the process is using a small fraction of v8.getHeapStatistics().heap_size_limit.
The viewer shows the same thing. src/viewer/index.html:1582 recomputes the identical heapUsed / heapTotal ratio client-side, so the HEAP gauge renders red on a healthy process.
Environment
- agentmemory 0.9.28 (latest published: 0.9.29)
- Node v22.23.2
- Platform: Railway, self-hosted from this repo's own
deploy/railway template
- Linux container, 8 GB memory limit, single replica
- Reproduced against the deployed service, not a local dev server
I have a fix with tests and can open a PR.
evaluateHealthmeasures memory severity againstheapTotal, which is the heap V8 has already committed, not the ceiling it is allowed to grow to. V8 sizesheapTotalto demand, so a busy healthy process sits near 100% of it more or less permanently.src/health/thresholds.ts:62-65Since
heapTotal <= heap_size_limitalways holds, this can only ever over-report severity, never under-report it.What I saw
On a live deployment:
heapUsedheapTotalrssv8.getHeapStatistics().heap_size_limitThat is 6.0% of the heap the process may actually use. The alert read
memory_critical_97%_rss557mband status wascritical.GET /agentmemory/healththen returned HTTP 503, becausesrc/triggers/api.ts:274mapscriticalto 503:/agentmemory/livezreturned 200 at the same moment. So an uptime monitor pointed at/healthreads the service as down while it is serving normally. At that point the container was using 3.0 GB of an 8 GB limit, the circuit breaker was closed, CPU was 3.4%, and event loop lag was 0.1 ms.Worth noting for anyone deploying from this repo:
deploy/railway/railway.jsonsetshealthcheckPathto/agentmemory/livez, so Railway's own check keeps passing and the service never restart-loops. The shipped template dodges this by accident. It is the richer/agentmemory/healthendpoint, the one you would point an external monitor at, that reports the false 503.What I expected
healthy, and HTTP 200. Six percent of the heap limit is not memory pressure.How this relates to #158
#158 reported the same formula misfiring at
heapUsed~45 MB,heapTotal~46 MB, and rss ~120 MB. The fix that landed addedmemoryRssFloorBytes(512 MB,src/health/thresholds.ts:20) so the alert stays quiet on small processes. That covers the case as reported, but the ratio itself was never corrected, so the bug comes back for any process whose RSS clears the floor.There is also no way to tune around it from the outside.
src/health/monitor.ts:87callsevaluateHealth(snapshot)with no config argument, soDEFAULTSalways wins and neither the percentages nor the RSS floor are reachable from the environment.Repro
curl -si "$AGENTMEMORY_URL/agentmemory/health" | head -1memory_critical_NN%inhealth.alerts, while the process is using a small fraction ofv8.getHeapStatistics().heap_size_limit.The viewer shows the same thing.
src/viewer/index.html:1582recomputes the identicalheapUsed / heapTotalratio client-side, so the HEAP gauge renders red on a healthy process.Environment
deploy/railwaytemplateI have a fix with tests and can open a PR.