You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I tested three quantization directions against pinned ARDY and LLM2Vec revisions on Modal GPUs:
bitsandbytes INT8 for the LLM2Vec text encoder;
bitsandbytes NF4 INT4 for the LLM2Vec text encoder;
symmetric per-output-channel, weight-only INT8 for selected body-denoiser transformer weights.
The text-encoder INT8 path is the strongest deployment candidate. INT4 materially reduces memory but caused action-dependent semantic motion changes. Body-denoiser MLP INT8 looked excellent in a one-seed probe and then failed 13/24 long-horizon comparisons. These results support an optional INT8 encoder path, but do not establish a production INT8 ARDY motion model.
The ARDY baseline below is called native precision, not BF16: the released checkpoint loads major protected denoiser modules, including the root stage, as FP32.
actions: rest, walk, guard, jab, kick, hit reaction, fall, recovery
comparisons reused identical prompts and seeds between modes
1. Text encoder INT8
On an A100, with the unchanged native-precision motion model:
Mode
Load + generation
Motion generation
Allocated
Peak allocated
BF16 encoder
195.222 s
13.798 s
15.006 GiB
15.025 GiB
INT8 encoder
108.659 s
13.312 s
8.511 GiB
9.330 GiB
This reduced steady allocation by 6.495 GiB (43.3%) and measured peak by 5.695 GiB (37.9%). The speed improvement came principally from encoder loading; ARDY diffusion time was effectively unchanged.
Embedding cosine was 0.992471–0.994200, but motion error remained action-dependent:
Action
Embedding cosine
qpos MAE
qpos RMSE
qpos max error
rest
.993649
.009336
.013614
.061610
walk
.994050
.014033
.029851
.280121
guard
.993254
.027882
.048362
.193491
jab
.992471
.010051
.023914
.347752
kick
.993383
.033467
.048329
.204833
hit reaction
.994200
.074779
.152488
1.182481
fall
.994017
.158847
.306165
1.429512
recovery
.992916
.026467
.037632
.230958
The high embedding cosine did not predict motion equivalence: fall and hit reaction diverged most. This path still needs multi-seed visual/contact acceptance before becoming a default.
2. Text encoder INT4 NF4
The exact Llama + merged MNTP + supervised stack loaded with NF4, BF16 compute, and double quantization.
Encoder-only probe:
Mode
Allocated
Peak
MNTP load
Encode
BF16
14.143 GiB
14.579 GiB
22.470 s
1.049 s
INT8
7.648 GiB
8.467 GiB
85.510 s
.707 s
INT4 NF4
4.497 GiB
5.301 GiB
45.075 s
.257 s
Full comparison:
Mode
Load + generation
Generation
Allocated
Peak
BF16 encoder
195.207 s
35.373 s
15.006 GiB
15.025 GiB
INT4 encoder
100.428 s
22.643 s
5.360 GiB
6.164 GiB
INT4 reduced full-stack allocation by 9.646 GiB (64.3%), but embedding cosine fell to .888557–.926366. Fall changed semantically: the native-precision-conditioned motion was already airborne/falling at frame 71, while INT4 remained standing and later reached a different ground pose. INT4 should therefore remain experimental and action-gated, not a default.
Rendered media and complete W&B result
The public W&B run contains configuration, timing/memory telemetry, the per-action table, all eight rendered four-panel MP4 comparisons, and the review artifact:
The root denoiser, normalization, input/output projections, conditioning projections, positional embeddings, output heads, activations, and biases stayed floating point. Only body-transformer attention/MLP weights were eligible. The root weights' digest remained unchanged.
This implementation explicitly dequantized weights during every forward. It tests numerical tolerance and serialized size only—not a genuine INT8 kernel, reduced live VRAM, latency, or cost.
Initial one-action/one-seed 10-second probe:
Target
Cosine
MAE
RMSE
Max
Decision
MLP only
.998794
.019275
.030385
.262399
advance to rendered validation
attention only
.998060
.021734
.038531
.590259
secondary candidate
attention + MLP
.875612
.116219
.315760
2.148622
reject
The full MLP-only gate used all eight actions and three seed bases, for 24 matched 10-second comparisons. It failed 13/24. The worst case, hit reaction seed 20262728, had:
motion cosine .277877;
final root error 3.1715 m;
mean posed-joint error 1.3216 m;
foot-contact agreement .652.
Runtime/memory for 24 generations:
Mode
Time
Peak allocated
Peak reserved
native precision
46.369 s
.889 GiB
.898 GiB
explicit-dequant MLP INT8
47.117 s
.810 GiB
.918 GiB
The candidate was 1.61% slower and did not establish a production memory/cost win. Protecting root weights was insufficient because perturbed body latents feed subsequent diffusion iterations and alter the final root trajectory.
Rendered media and complete W&B results
The recovery run contains the complete 24-row metric table, all paired front/side videos, contact sheets including the worst hit-reaction failure, runtime/memory telemetry, and the review artifact:
Add an optional, explicitly experimental bitsandbytes INT8 encoder-loading mode.
Keep native precision as the reference and require rendered, action-level acceptance.
Cache embeddings with a key that fingerprints encoder weights, both adapters, and preprocessing (addressed separately in Version text embedding cache keys #8).
Do not advertise NF4 INT4 as equivalent or default.
Do not optimize or promote the unchanged all-layer body-MLP INT8 scheme.
If body quantization continues, perform per-layer sensitivity testing and restore the most sensitive one or two MLP layers to floating point before rerunning the same multi-seed gate.
Only after a candidate passes quality gates should it move to a genuine kernel-backed implementation (for example TorchAO or TensorRT) and be benchmarked for cold start, warm latency, peak VRAM, and cost.
I am filing this as a field report/RFC rather than a code PR because the body experiment is a negative result and the encoder path still needs an upstream-friendly implementation and acceptance policy.
Summary
I tested three quantization directions against pinned ARDY and LLM2Vec revisions on Modal GPUs:
The text-encoder INT8 path is the strongest deployment candidate. INT4 materially reduces memory but caused action-dependent semantic motion changes. Body-denoiser MLP INT8 looked excellent in a one-seed probe and then failed 13/24 long-horizon comparisons. These results support an optional INT8 encoder path, but do not establish a production INT8 ARDY motion model.
The ARDY baseline below is called native precision, not BF16: the released checkpoint loads major protected denoiser modules, including the root stage, as FP32.
Reproducibility
Common inputs:
693f74d13b3d04a0a22ce127ee79c929dd89756bnvidia/ARDY-G1-RP-25FPS-Horizon8@334a8a9cdbafc962dcd304c26f31e6b17a86935553346005fb0ef11d3b6a83b12c895cca40156b6c31474e395ada192e8ed1586db6be79fb3b70c9c0baa8ebf04a1c2500e61288e7dad65e8ae42601a71. Text encoder INT8
On an A100, with the unchanged native-precision motion model:
This reduced steady allocation by 6.495 GiB (43.3%) and measured peak by 5.695 GiB (37.9%). The speed improvement came principally from encoder loading; ARDY diffusion time was effectively unchanged.
Embedding cosine was
0.992471–0.994200, but motion error remained action-dependent:The high embedding cosine did not predict motion equivalence: fall and hit reaction diverged most. This path still needs multi-seed visual/contact acceptance before becoming a default.
2. Text encoder INT4 NF4
The exact Llama + merged MNTP + supervised stack loaded with NF4, BF16 compute, and double quantization.
Encoder-only probe:
Full comparison:
INT4 reduced full-stack allocation by 9.646 GiB (64.3%), but embedding cosine fell to
.888557–.926366. Fall changed semantically: the native-precision-conditioned motion was already airborne/falling at frame 71, while INT4 remained standing and later reached a different ground pose. INT4 should therefore remain experimental and action-gated, not a default.Rendered media and complete W&B result
The public W&B run contains configuration, timing/memory telemetry, the per-action table, all eight rendered four-panel MP4 comparisons, and the review artifact:
3. Body-denoiser weight-only INT8
The root denoiser, normalization, input/output projections, conditioning projections, positional embeddings, output heads, activations, and biases stayed floating point. Only body-transformer attention/MLP weights were eligible. The root weights' digest remained unchanged.
This implementation explicitly dequantized weights during every forward. It tests numerical tolerance and serialized size only—not a genuine INT8 kernel, reduced live VRAM, latency, or cost.
Initial one-action/one-seed 10-second probe:
The full MLP-only gate used all eight actions and three seed bases, for 24 matched 10-second comparisons. It failed 13/24. The worst case, hit reaction seed
20262728, had:.277877;3.1715 m;1.3216 m;.652.Runtime/memory for 24 generations:
The candidate was 1.61% slower and did not establish a production memory/cost win. Protecting root weights was insufficient because perturbed body latents feed subsequent diffusion iterations and alter the final root trajectory.
Rendered media and complete W&B results
The recovery run contains the complete 24-row metric table, all paired front/side videos, contact sheets including the worst hit-reaction failure, runtime/memory telemetry, and the review artifact:
Conclusions / proposed upstream direction
I am filing this as a field report/RFC rather than a code PR because the body experiment is a negative result and the encoder path still needs an upstream-friendly implementation and acceptance policy.