Use bounded-step curvature for Tweedie objectives

[RAMitchell]

Motivation

The Poisson, compound Poisson-Gamma Tweedie, and Gamma objectives form one continuous power-variance family under the log-mean parameterization.

The Taylor Hessian can produce very large leaf updates when its local curvature is small. The bounded Halley curvature added to Poisson in #12431 fixes the leaf-update instability, but its ordinary G^2/H score can still overestimate the loss reduction used to justify a split.

This PR uses a bounded-step pseudo-Hessian across the complete Poisson-Tweedie-Gamma segment. It retains the exact gradient and does not change the tree builder, leaf formula, split-gain formula, or public parameters.

Proposed curvature

Let

mu_i = exp(f_i)
A_i = w_i * y_i * mu_i^(1-p)
B_i = w_i * mu_i^(2-p)

The exact gradient and proposed curvature are

g_i = B_i - A_i
h_i = (p*A_i + (3-p)*B_i) / 3

The endpoints are:

Objective	Power	Curvature
count:poisson	1	w * (y + 2*mu) / 3
reg:tweedie	1 <= p < 2	w * (p*y*mu^(1-p) + (3-p)*mu^(2-p)) / 3
reg:gamma	2	w * (2*y/mu + 1) / 3

At p=1.5, this is exactly the existing Taylor Hessian.

Derivation and guarantees

For one leaf, put A=sum(A_i), B=sum(B_i), and R=A/B. The exact constant-leaf optimum is log(R). For an affine row-additive curvature H_c=c*A+(1-c)*B, the quadratic reduction in loss units is

q_c / B = (R-1)^2 / (2 * (1 + c*(R-1))).

Calibrating this expression to the exact optimized leaf-loss reduction through cubic order uniquely gives c=p/3 and the curvature above.

For nonnegative labels and weights, 1 <= p <= 2, and the unregularized full-step problem:

• the induced leaf update satisfies abs(d) <= 3;
• a full step decreases the exact leaf loss, as does any shrinkage 0 < eta <= 1;
• the quadratic node score lower-bounds both the reduction realized by its own update and the exact optimized reduction; and
• these lower bounds are preserved by the child-minus-parent split calculation.

The last property is important for XGBoost: the curvature controls both the leaf update -G/H and split score G^2/H. This is an optimized-score guarantee, not a claim that the quadratic is a pointwise MM majorizer.

Previous synthetic results

A paired stress study evaluated powers {1, 1.1, 1.5, 1.9, 2}, four signal/initialization conditions, and 30 datasets per condition. Each method trained 50 depth-three histogram trees with eta=1, min_child_weight=0, and all regularization disabled.

The values below are median round-50 mean unit deviance divided by round-zero deviance; lower is better. "Taylor Newton" is a common comparator over the whole family, not the current Poisson implementation from #12431.

Power	Bounded-step	Taylor Newton	Halley	Bounded-step vs Newton	Increasing rounds: bounded/Newton
1.0	0.238	8.69e24	0.236	120/120 lower	0 / 729
1.1	0.244	0.280	0.241	120/120 lower	0 / 461
1.5	0.313	0.313	0.313	120/120 identical	0 / 0
1.9	0.454	0.540	0.457	120/120 lower	0 / 1,891
2.0	0.430	1.41e19	0.430	120/120 lower	0 / 128

Across the 480 off-midpoint datasets, bounded-step curvature finished below Taylor Newton in every paired run. Bounded-step and Halley were close: bounded-step won 291/480 paired endpoints, and their condition-level median endpoints differed by at most about 1.75%.

The large endpoint values are deliberately adversarial unit-step stability results; they should not be interpreted as expected production magnitudes.

A separate Poisson study evaluated 36 candidate partitions for each of 1,000 synthetic parents:

Method	Exact top-one agreement	Mean oracle-gain regret	Parents optimistic vs exact	Parents optimistic vs own update
Taylor Newton	79.3%	2.90%	54.9%	58.2%
Halley	75.0%	4.94%	25.9%	47.3%
Bounded-step	78.8%	3.78%	0.0%	0.0%

Bounded-step curvature therefore retained split ranking close to Newton while eliminating score overstatement in this corpus, consistent with the analytical split certificate.

Implementation

• Update the Poisson, Tweedie, and Gamma elementwise gradient functors.
• Keep their registrations, validation, metrics, and public interfaces separate.
• Remove the now-unnecessary GammaDeviance wrapper.
• Add an explicit reg:tweedie power-1 endpoint regression check.
• Update expected gradient-pair values for all three objectives.

Validation

• ./build/testxgboost --gtest_filter='Objective.*': 32/32 passed.
• Unregularized stump checks converged to the aggregate optimum at powers 1, 1.1, 1.5, 1.9, and 2.

Limitations

The analytical split certificate assumes the unregularized full-step problem. The synthetic study is a mechanism and stability test, not a held-out generalization benchmark or a hyperparameter-tuned comparison.

[Copilot]

🔵 Needs a closer look

The broad training-behavior change and its mathematical split-gain guarantees warrant final human validation.

Pull request overview

Updates Poisson, Tweedie, and Gamma objectives to use bounded-step pseudo-Hessian curvature while retaining exact gradients and existing APIs.

Changes:

• Implements family-consistent bounded curvature across all three objectives.
• Removes the obsolete GammaDeviance wrapper.
• Updates gradient-pair expectations and adds Tweedie’s Poisson-endpoint coverage.

File summaries

File	Description
src/objective/poisson_obj.h	Applies bounded Poisson curvature.
src/objective/tweedie_obj.h	Implements generalized Tweedie curvature.
src/objective/gamma_obj.h	Applies Gamma endpoint curvature and simplifies helpers.
src/objective/gamma_obj.cc	Inlines removed wrapper functionality.
tests/cpp/objective/test_regression_obj.cc	Updates expected Hessians and tests power 1.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 0
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.