Skip to content

Field report: INT8/INT4 text encoder and body-denoiser quantization #9

Description

@ebowwa

Summary

I tested three quantization directions against pinned ARDY and LLM2Vec revisions on Modal GPUs:

  1. bitsandbytes INT8 for the LLM2Vec text encoder;
  2. bitsandbytes NF4 INT4 for the LLM2Vec text encoder;
  3. symmetric per-output-channel, weight-only INT8 for selected body-denoiser transformer weights.

The text-encoder INT8 path is the strongest deployment candidate. INT4 materially reduces memory but caused action-dependent semantic motion changes. Body-denoiser MLP INT8 looked excellent in a one-seed probe and then failed 13/24 long-horizon comparisons. These results support an optional INT8 encoder path, but do not establish a production INT8 ARDY motion model.

The ARDY baseline below is called native precision, not BF16: the released checkpoint loads major protected denoiser modules, including the root stage, as FP32.

Reproducibility

Common inputs:

  • ARDY source: 693f74d13b3d04a0a22ce127ee79c929dd89756b
  • ARDY model: nvidia/ARDY-G1-RP-25FPS-Horizon8@334a8a9cdbafc962dcd304c26f31e6b17a869355
  • Llama base: 53346005fb0ef11d3b6a83b12c895cca40156b6c
  • MNTP adapter: 31474e395ada192e8ed1586db6be79fb3b70c9c0
  • supervised adapter: baa8ebf04a1c2500e61288e7dad65e8ae42601a7
  • motions: 250 frames / 10 seconds at 25 FPS
  • diffusion: four steps
  • actions: rest, walk, guard, jab, kick, hit reaction, fall, recovery
  • comparisons reused identical prompts and seeds between modes

1. Text encoder INT8

On an A100, with the unchanged native-precision motion model:

Mode Load + generation Motion generation Allocated Peak allocated
BF16 encoder 195.222 s 13.798 s 15.006 GiB 15.025 GiB
INT8 encoder 108.659 s 13.312 s 8.511 GiB 9.330 GiB

This reduced steady allocation by 6.495 GiB (43.3%) and measured peak by 5.695 GiB (37.9%). The speed improvement came principally from encoder loading; ARDY diffusion time was effectively unchanged.

Embedding cosine was 0.992471–0.994200, but motion error remained action-dependent:

Action Embedding cosine qpos MAE qpos RMSE qpos max error
rest .993649 .009336 .013614 .061610
walk .994050 .014033 .029851 .280121
guard .993254 .027882 .048362 .193491
jab .992471 .010051 .023914 .347752
kick .993383 .033467 .048329 .204833
hit reaction .994200 .074779 .152488 1.182481
fall .994017 .158847 .306165 1.429512
recovery .992916 .026467 .037632 .230958

The high embedding cosine did not predict motion equivalence: fall and hit reaction diverged most. This path still needs multi-seed visual/contact acceptance before becoming a default.

2. Text encoder INT4 NF4

The exact Llama + merged MNTP + supervised stack loaded with NF4, BF16 compute, and double quantization.

Encoder-only probe:

Mode Allocated Peak MNTP load Encode
BF16 14.143 GiB 14.579 GiB 22.470 s 1.049 s
INT8 7.648 GiB 8.467 GiB 85.510 s .707 s
INT4 NF4 4.497 GiB 5.301 GiB 45.075 s .257 s

Full comparison:

Mode Load + generation Generation Allocated Peak
BF16 encoder 195.207 s 35.373 s 15.006 GiB 15.025 GiB
INT4 encoder 100.428 s 22.643 s 5.360 GiB 6.164 GiB

INT4 reduced full-stack allocation by 9.646 GiB (64.3%), but embedding cosine fell to .888557–.926366. Fall changed semantically: the native-precision-conditioned motion was already airborne/falling at frame 71, while INT4 remained standing and later reached a different ground pose. INT4 should therefore remain experimental and action-gated, not a default.

Rendered media and complete W&B result

The public W&B run contains configuration, timing/memory telemetry, the per-action table, all eight rendered four-panel MP4 comparisons, and the review artifact:

3. Body-denoiser weight-only INT8

The root denoiser, normalization, input/output projections, conditioning projections, positional embeddings, output heads, activations, and biases stayed floating point. Only body-transformer attention/MLP weights were eligible. The root weights' digest remained unchanged.

This implementation explicitly dequantized weights during every forward. It tests numerical tolerance and serialized size only—not a genuine INT8 kernel, reduced live VRAM, latency, or cost.

Initial one-action/one-seed 10-second probe:

Target Cosine MAE RMSE Max Decision
MLP only .998794 .019275 .030385 .262399 advance to rendered validation
attention only .998060 .021734 .038531 .590259 secondary candidate
attention + MLP .875612 .116219 .315760 2.148622 reject

The full MLP-only gate used all eight actions and three seed bases, for 24 matched 10-second comparisons. It failed 13/24. The worst case, hit reaction seed 20262728, had:

  • motion cosine .277877;
  • final root error 3.1715 m;
  • mean posed-joint error 1.3216 m;
  • foot-contact agreement .652.

Runtime/memory for 24 generations:

Mode Time Peak allocated Peak reserved
native precision 46.369 s .889 GiB .898 GiB
explicit-dequant MLP INT8 47.117 s .810 GiB .918 GiB

The candidate was 1.61% slower and did not establish a production memory/cost win. Protecting root weights was insufficient because perturbed body latents feed subsequent diffusion iterations and alter the final root trajectory.

Rendered media and complete W&B results

The recovery run contains the complete 24-row metric table, all paired front/side videos, contact sheets including the worst hit-reaction failure, runtime/memory telemetry, and the review artifact:

Conclusions / proposed upstream direction

  • Add an optional, explicitly experimental bitsandbytes INT8 encoder-loading mode.
  • Keep native precision as the reference and require rendered, action-level acceptance.
  • Cache embeddings with a key that fingerprints encoder weights, both adapters, and preprocessing (addressed separately in Version text embedding cache keys #8).
  • Do not advertise NF4 INT4 as equivalent or default.
  • Do not optimize or promote the unchanged all-layer body-MLP INT8 scheme.
  • If body quantization continues, perform per-layer sensitivity testing and restore the most sensitive one or two MLP layers to floating point before rerunning the same multi-seed gate.
  • Only after a candidate passes quality gates should it move to a genuine kernel-backed implementation (for example TorchAO or TensorRT) and be benchmarked for cold start, warm latency, peak VRAM, and cost.

I am filing this as a field report/RFC rather than a code PR because the body experiment is a negative result and the encoder path still needs an upstream-friendly implementation and acceptance policy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions