Skip to content

Fix batched Gaussian state accumulation - #1051

Open
tandede wants to merge 1 commit into
nerfstudio-project:mainfrom
tandede:fix-batched-strategy-gaussian-indices
Open

Fix batched Gaussian state accumulation#1051
tandede wants to merge 1 commit into
nerfstudio-project:mainfrom
tandede:fix-batched-strategy-gaussian-indices

Conversation

@tandede

@tandede tandede commented Aug 24, 2026

Copy link
Copy Markdown

Summary

Fix DefaultStrategy state accumulation when non-packed rasterization outputs include leading batch dimensions.

For the original [C, N] visibility mask, torch.where(sel)[1] identifies the Gaussian dimension. With batched inputs, however, the mask is shaped [..., C, N], so coordinate 1 can be a batch or camera coordinate instead. Gradients and visibility counts are consequently accumulated into the wrong Gaussians without raising an error.

Approach

Use the last coordinate returned by torch.where(sel) as the Gaussian ID. The final mask dimension is always N, independent of the number of leading batch and camera dimensions.

The existing masked gradient and radius selection remains unchanged, so every visible batch-camera observation continues to contribute independently to the accumulated gradient and visibility count. The observations are not collapsed across batches because doing so would change the averaging semantics used by the densification strategy.

Regression coverage

Add a CPU regression with:

  • 2 batches
  • 2 cameras
  • 3 Gaussians
  • different visible Gaussians and gradient magnitudes in each batch-camera pair

Before the fix, the strategy assigns observations according to camera coordinates, producing counts [2, 2, 0] and accumulated gradients [8, 12, 0]. With the fix, observations are assigned by Gaussian index, producing the expected counts [2, 1, 1] and gradients [10, 4, 6].

Test results

  • pytest -q tests/test_strategy.py: 2 passed, 2 skipped
  • pytest -q tests: 539 passed, 342 skipped
  • black --check --diff --required-version 22.3.0 gsplat/strategy/default.py tests/test_strategy.py
  • git diff --check

The skipped tests require CUDA or native CUDA kernels and are unrelated to this CPU state-accumulation path.

Fixes #854

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Possible wrong results with batch

1 participant