Hoist from_array_interface wrapper class to module scope to avoid per-call reference cycles

[amitaypgy]

Summary

from_array_interface defines its Array wrapper class inside the function body, so a brand-new class object is created on every call. Class objects are self-referential and thus form reference cycles that plain reference counting cannot reclaim. Because from_array_interface runs on every prediction (and on every DMatrix/QuantileDMatrix construction), this leaks cyclic garbage on every request. Services that run with the cyclic GC disabled grow in memory without bound.

This PR hoists the wrapper to module scope (_ArrayInterfaceProxy) so it is created once at import. Behavior is byte-for-byte identical; only the class location changed.

Root cause

from_array_interface (currently in python-package/xgboost/_data_utils.py) looks like this:

def from_array_interface(interface, zero_copy=False):
    class Array:          # <-- re-created on EVERY call
        _interface = None
        @property
        def __array_interface__(self): ...
        @__array_interface__.setter
        def __array_interface__(self, interface): ...
        # ... __cuda_array_interface__, shape, size ...
    arr = Array()
    ...

A class statement in a function body creates a new class object each time the function runs. Every class object is inherently self-referential:

• Array.__dict__ holds the property descriptors, and each descriptor's fget/fset are functions whose __globals__/closure and __qualname__ tie back to the class namespace,
• Array.__mro__ / __bases__ reference the class,
• the class references its own __dict__.

These form reference cycles. CPython frees ordinary objects immediately via reference counting, but a cycle keeps every member's refcount ≥ 1, so a cycle can only be reclaimed by the cyclic garbage collector (gc).

from_array_interface is on the hottest path in the library — it is called by Booster.predict() (via _prediction_output) and by the DMatrix builders — so each call leaves a fresh, unreclaimable-by-refcount class cycle behind.

Why this matters in production

Many latency-sensitive inference services call gc.disable() to avoid unpredictable GC pauses on the request path. With the cyclic collector off, the per-call cycles created here are never collected. They accumulate for the life of the process, and because these wrapper objects sit on top of array buffers, the native memory they pin accumulates too — i.e. steady, unbounded RSS growth proportional to the number of predictions served.

We hit exactly this in a long-running scoring service (Python 3.13 / numpy 2.x / xgboost 2.x) that scores one application at a time against dozens of boosters per request.

Reproduction

Standalone script (no XGBoost internals, just the public API). It fingerprints how many objects a single request creates that are only reclaimable by gc.collect() (i.e. cyclic), and then runs a sustained loop with the GC disabled to show RSS growth.

import gc, os, platform, sys
import numpy as np
import xgboost

N_FEATURES = 1228
N_BOOSTERS = 40
N_ROWS = 1
ITERS = 150
EVERY = 30
GC_ENABLED = False   # False = reproduce the leak (GC off); True = control (flat)

_FEATURE_TYPES = ["float"] * N_FEATURES

def rss_mb():
    try:
        import psutil
        return psutil.Process().memory_info().rss / (1024 * 1024)
    except Exception:
        with open(f"/proc/{os.getpid()}/statm") as f:
            pages = int(f.read().split()[1])
        return pages * os.sysconf("SC_PAGE_SIZE") / (1024 * 1024)

def make_boosters(n):
    rng = np.random.default_rng(0)
    x = rng.standard_normal((256, N_FEATURES)).astype(np.float64)
    y = (x[:, 0] > 0).astype(np.float64)
    train = xgboost.DMatrix(x, label=y, feature_types=_FEATURE_TYPES, enable_categorical=False)
    params = {"max_depth": 4, "objective": "binary:logistic", "nthread": 1}
    return [xgboost.train(params, train, num_boost_round=15) for _ in range(n)]

def main():
    arr = np.random.default_rng(1).standard_normal((N_ROWS, N_FEATURES)).astype(np.float64)
    boosters = make_boosters(N_BOOSTERS)

    def run_once():
        dmatrix = xgboost.DMatrix(arr, feature_types=_FEATURE_TYPES, enable_categorical=False)
        return [b.predict(dmatrix, validate_features=False) for b in boosters]

    print(f"python {platform.python_version()} | numpy {np.__version__} | xgboost {xgboost.__version__}")

    # Fingerprint: how many cyclic-only objects one request creates.
    run_once(); gc.collect()
    gc.disable()
    before = len(gc.get_objects())
    run_once()
    live = len(gc.get_objects()) - before
    cyclic = gc.collect()
    print(f"[fingerprint] one request -> live_delta≈{live}, cyclic_only(reclaimed by gc.collect)={cyclic}")

    # Sustained loop with GC disabled (like the app).
    gc.collect()
    gc.enable() if GC_ENABLED else gc.disable()
    base_rss = rss_mb()
    print(f"GC {'ENABLED (control)' if GC_ENABLED else 'DISABLED (like app)'}")
    for i in range(1, ITERS + 1):
        run_once()
        if i % EVERY == 0:
            print(f"iter={i:>4}  d_rss={rss_mb() - base_rss:+.1f} MB  objects={len(gc.get_objects())}")

if __name__ == "__main__":
    main()

What you observe:

• With GC_ENABLED = False (matches a real service that disabled the GC): cyclic_only is a fixed, non-zero count for every request, objects climbs monotonically, and d_rss grows steadily across iterations.
• With GC_ENABLED = True (control): RSS stays flat because the cyclic collector keeps sweeping up the per-call class cycles — at the cost of the GC pauses services turn off in the first place.
• After the patch: cyclic_only drops to ~0 for the from_array_interface path (the per-call instance is not in a cycle), so RSS stays flat even with the GC disabled.

The fix

Move the wrapper class out of the function to module scope as _ArrayInterfaceProxy, created once at import. The per-call _ArrayInterfaceProxy() instance is not part of a cycle — it references the module-level class, which does not reference the instance back — so it is freed immediately by reference counting, no GC required. The class body (array-interface / cuda-array-interface getters and setters, shape, size) is unchanged, so there is no behavioral or API change.

Affects 2.x too — please also fix on the 2.x line

This is not new to 3.x. The same in-function class Array has existed since the 2.0 line (release_2.0.0 / release_2.1.0), where the function lived in python-package/xgboost/core.py; it was later moved (unchanged) to _data_utils.py in 3.0. So every 2.x and 3.x release has this per-call cycle.

We are currently pinned to xgboost 2.x, so it would be very helpful if this fix could also be backported to the maintained 2.x release branch (the change is a trivial, behavior-preserving hoist of the same class in core.py). Happy to open a companion PR against the 2.x branch if that's preferred.

[trivialfis]

Suggested change

	"""Wrapper type for communicating with numpy and cupy.
	Defined at module scope rather than inside :py:func:`from_array_interface` on
	purpose. A class defined in a function body is a brand-new object on every call
	and participates in reference cycles (via ``__dict__``, ``__mro__`` and its
	property descriptors), so it cannot be reclaimed by reference counting alone.
	Since ``from_array_interface`` runs on every prediction, recreating the class
	per call produces cyclic garbage that only the cyclic GC can free, leading to
	unbounded memory growth in long-running services that disable the GC. Hoisting
	the class here means it is created once at import; the per-call instance is not
	part of a cycle and is freed immediately by reference counting.
	"""
	"""Wrapper type for communicating with numpy and cupy."""