Skip to content

Commit 6499c9d

Browse files
ioquatixioquatix
committed
Show retained fibers.
1 parent 96b7241 commit 6499c9d

2 files changed

Lines changed: 174 additions & 3 deletions

File tree

‎examples/hello/config.ru‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,13 @@
11
#!/usr/bin/env falcon --verbose serve -c
22
# frozen_string_literal: true
33

4+
require "objspace"
5+
46
run do |env|
5-
# To test the fiber profiler, you can uncomment the following line:
6-
# Fiber.blocking{sleep 0.1}
7-
[200, {}, ["Hello World"]]
7+
if env["PATH_INFO"] == "/gc"
8+
GC.start
9+
end
10+
11+
fiber_count = ObjectSpace.each_object(Fiber).count
12+
[200, {}, ["Fiber count: #{fiber_count}"]]
813
end

‎examples/hello/notes.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
Running the server:
2+
3+
```
4+
samuel@Sakura ~/D/s/f/e/hello (main)> bundle exec falcon serve --bind http://[::]:9292 --count 1
5+
0.0s info: Falcon::Command::Serve [oid=0x5d0] [ec=0x5d8] [pid=62134] [2026-03-19 17:54:58 +1300]
6+
| Falcon v0.55.2 taking flight! Using Async::Container::Forked {count: 1, restart: true, health_check_timeout: 30.0}.
7+
| - Running on ruby 3.4.8 (2025-12-17 revision 995b59f666) +PRISM [arm64-darwin25]
8+
| - Binding to: #<Falcon::Endpoint http://[::]:9292/ {}>
9+
| - To terminate: Ctrl-C or kill 62134
10+
| - To reload configuration: kill -HUP 62134
11+
0.0s info: Async::Container::Notify::Console [oid=0x5e0] [ec=0x5d8] [pid=62134] [2026-03-19 17:54:58 +1300]
12+
| {status: "Initializing controller..."}
13+
0.0s info: Falcon::Service::Server [oid=0x5f0] [ec=0x5d8] [pid=62134] [2026-03-19 17:54:58 +1300]
14+
| Starting http://[::]:9292 on #<Falcon::Endpoint http://[::]:9292/ {}>
15+
0.0s info: Async::Service::Controller [oid=0x5f8] [ec=0x5d8] [pid=62134] [2026-03-19 17:54:58 +1300]
16+
| Controller starting...
17+
0.0s info: Async::Service::Controller [oid=0x5f8] [ec=0x5d8] [pid=62134] [2026-03-19 17:54:58 +1300]
18+
| Starting container...
19+
0.01s info: Async::Service::Controller [oid=0x5f8] [ec=0x5d8] [pid=62134] [2026-03-19 17:54:58 +1300]
20+
| Waiting for startup...
21+
0.01s info: Async::Service::Controller [oid=0x5f8] [ec=0x5d8] [pid=62134] [2026-03-19 17:54:58 +1300]
22+
| Finished startup.
23+
0.01s info: Async::Container::Notify::Console [oid=0x5e0] [ec=0x5d8] [pid=62134] [2026-03-19 17:54:58 +1300]
24+
| {ready: true, size: 1, status: "Running with 1 children."}
25+
0.01s info: Async::Service::Controller [oid=0x5f8] [ec=0x5d8] [pid=62134] [2026-03-19 17:54:58 +1300]
26+
| Controller started.
27+
```
28+
29+
Sending requests (in another terminal):
30+
31+
```
32+
samuel@Sakura ~> curl http://localhost:9292
33+
Fiber count: 9
34+
samuel@Sakura ~> curl http://localhost:9292
35+
Fiber count: 10
36+
samuel@Sakura ~> curl http://localhost:9292
37+
Fiber count: 11
38+
samuel@Sakura ~> curl http://localhost:9292
39+
Fiber count: 12
40+
samuel@Sakura ~> curl http://localhost:9292
41+
Fiber count: 13
42+
samuel@Sakura ~> curl http://localhost:9292/gc
43+
Fiber count: 8
44+
samuel@Sakura ~> curl http://localhost:9292
45+
Fiber count: 9
46+
samuel@Sakura ~> curl http://localhost:9292
47+
Fiber count: 10
48+
```
49+
50+
## Why Fibers Accumulate (CRuby GC Analysis)
51+
52+
### Fibers are Write-Barrier Unprotected
53+
54+
In `cont.c`, `rb_fiber_data_type` is defined without `RUBY_TYPED_WB_PROTECTED`:
55+
56+
```c
57+
static const rb_data_type_t rb_fiber_data_type = {
58+
"fiber",
59+
{fiber_mark, fiber_free, fiber_memsize, fiber_compact, fiber_handle_weak_references},
60+
0, 0, RUBY_TYPED_FREE_IMMEDIATELY // ← no RUBY_TYPED_WB_PROTECTED
61+
};
62+
```
63+
64+
This makes every `Fiber` object **write-barrier unprotected** ("shady" in RGenGC terminology).
65+
66+
**Why they must be WB-unprotected:** `fiber_mark` → `cont_mark` → `rb_execution_context_mark` → `rb_gc_mark_machine_context`, which does a **conservative scan** of the fiber's native C machine stack. Any word that looks like a valid heap pointer is treated as an object reference. Since C-level pointers appear and disappear from the stack without any Ruby-level write operation, write barriers simply cannot track these changes.
67+
68+
### Fibers Never Age
69+
70+
In `gc/default/default.c`, `gc_aging` is called whenever an object is first marked. For WB-unprotected objects it does nothing:
71+
72+
```c
73+
static void gc_aging(rb_objspace_t *objspace, VALUE obj) {
74+
...
75+
if (!RVALUE_PAGE_WB_UNPROTECTED(page, obj)) {
76+
// increment age, promote to old at RVALUE_OLD_AGE (== 3)
77+
RVALUE_AGE_INC(objspace, obj);
78+
}
79+
// WB-unprotected objects: skip entirely — age stays at 0 forever
80+
}
81+
```
82+
83+
The GC consistency checker enforces this invariant:
84+
85+
```c
86+
if (age > 0 && wb_unprotected_bit) {
87+
rb_bug("not WB protected, but age is %d > 0", age);
88+
}
89+
```
90+
91+
So fibers are **permanently young** (age 0). They never accumulate in `uncollectible_bits` (the old-generation bitmap) through the normal aging path.
92+
93+
### Minor GC vs. Major GC
94+
95+
At the start of each **minor GC** (`gc_marks_start`, `full_mark = false`):
96+
97+
```c
98+
// Pre-mark all old (uncollectible) objects as alive
99+
memcpy(&page->mark_bits[0], &page->uncollectible_bits[0], HEAP_PAGE_BITMAP_SIZE);
100+
101+
// Scan remembered set: old objects that recently wrote a reference to a young object
102+
rgengc_rememberset_mark(objspace, heap);
103+
104+
// Then mark from actual GC roots
105+
mark_roots(objspace, NULL);
106+
```
107+
108+
Because fibers have `uncollectible_bits == 0`, they are **not pre-marked**. They survive a minor GC only if something marks them during the traversal — either via the remembered set or via reachability from GC roots (e.g., the current thread's stack or `th->scheduler`).
109+
110+
At the start of each **major GC** (`gc_marks_start`, `full_mark = true`):
111+
112+
```c
113+
// Clear EVERYTHING — mark, uncollectible, marking, remembered bits
114+
rgengc_mark_and_rememberset_clear(objspace, heap);
115+
// Reset object counts
116+
objspace->rgengc.old_objects = 0;
117+
objspace->rgengc.uncollectible_wb_unprotected_objects = 0;
118+
objspace->marked_slots = 0;
119+
// Then do a full traversal from roots
120+
mark_roots(objspace, NULL);
121+
```
122+
123+
Every object must prove liveness from scratch.
124+
125+
### Why Completed Request Fibers Survive Minor GC
126+
127+
In a minor GC, old objects (like the long-lived `Async::Reactor`) start with their mark bit already set (pre-marked via `uncollectible_bits`). `gc_mark_set` returns immediately if the mark bit is already set:
128+
129+
```c
130+
static inline int gc_mark_set(rb_objspace_t *objspace, VALUE obj) {
131+
if (RVALUE_MARKED(objspace, obj)) return 0; // already marked — don't re-scan
132+
...
133+
}
134+
```
135+
136+
Pre-marked old objects are **not added to the grey queue** and **not re-scanned** in a minor GC. Their children are only traced if the old object is in the **remembered set** (`remembered_bits`), which is populated when a write barrier fires because the object stored a reference to a young object.
137+
138+
The remembered bit is **cleared** after each scan:
139+
140+
```c
141+
bits[j] = remembered_bits[j] | (uncollectible_bits[j] & wb_unprotected_bits[j]);
142+
remembered_bits[j] = 0; // cleared after use
143+
```
144+
145+
So once a write barrier fires (e.g., when a task fiber is first stored in the reactor's data structure) and the next minor GC consumes it, the reactor is no longer in the remembered set. After that, the fiber survives subsequent minor GCs only if the reactor modifies its internal structures again (triggering a new write barrier) or if the fiber is reachable via some other path.
146+
147+
In practice, the Async reactor and scheduler **keep completed task fibers referenced** (e.g., in `@children` sets on `Async::Node`, via `Async::Task#fiber`). Because the reactor is an old object that is frequently mutated (new tasks arrive, timers fire, I/O events arrive), write barriers keep firing, continuously refreshing the remembered set for the reactor's containers. This keeps all referenced task fibers alive through every minor GC — they behave as if they are in the old generation, even though they never age past 0.
148+
149+
### Why `GC.start` (Major GC) Frees Them
150+
151+
During a major GC, `rgengc_mark_and_rememberset_clear` resets all bits to zero. The reactor starts with mark bit = 0. When `mark_roots` marks it, it becomes grey and is added to the mark stack. Only then are its **current** children traversed.
152+
153+
If Async has already cleaned up a completed task (removed it from `@children`, dropped the reference), that task's fiber is not reachable and gets swept. If the task is still held (e.g., awaiting `.wait`), the fiber survives.
154+
155+
### Summary
156+
157+
| Aspect | Behaviour |
158+
|---|---|
159+
| WB-unprotected? | Yes — conservative machine stack scan requires it |
160+
| Ages to old? | Never — `gc_aging` skips WB-unprotected objects; age stays at 0 |
161+
| `uncollectible_bits` set? | Not by normal aging; only via `gc_remember_unprotected` (not triggered for fresh fibers) |
162+
| Survives minor GC? | Yes, if held by an old object that continuously re-enters the remembered set |
163+
| Freed by minor GC? | Only if truly unreachable after the last write barrier for the holding container is consumed |
164+
| Freed by `GC.start`? | Yes — full re-scan from roots discovers only currently-referenced objects |
165+
166+
The fiber count keeps climbing because the reactor is mutated frequently enough that its remembered bits are continuously refreshed, keeping every referenced task fiber alive through every minor GC. Only a major GC, which re-discovers the live set from scratch, can free fibers whose references have already been dropped.

0 commit comments

Comments
 (0)