[model] support Nanbeige4.2 Looped Transformer for LoRA/full SFT

[4teven]

Summary

• Add Nanbeige4.2 Looped Transformer support (model_type=nanbeige, trust_remote_code) for LoRA and full SFT.
• Register nanbeige / nanbeige_nothink chat templates and WebUI model entries for Base/Instruct.
• Patch transformers>=5 rope_scaling injection so remote modeling_nanbeige.py can initialize RoPE correctly.
• Add LoRA, QLoRA, and full SFT example configs.

Notes

• Nanbeige4.2 reuses 22 layers with num_loops=2 (roughly 2x forward compute vs a plain 22-layer model).
• Full SFT example uses DeepSpeed ZeRO-3; multi-GPU is recommended.
• QLoRA example is preferred on smaller GPUs.

Test plan

• Load config/tokenizer with trust_remote_code=true, use_fast_tokenizer=false
• LlamaFactory LoRA (4-bit) load + short forward/backward (finite loss, LoRA grads present)
• Template encode checks for nanbeige / nanbeige_nothink
• Multi-GPU full SFT with examples/train_full/nanbeige42_full_sft.yaml — validated on the production GPU cluster after fixing the transformers>=5.x remote-code incompatibilities (see commit db026093 and the follow-up comment: _tied_weights_keys list->dict crash during checkpoint save, and rope_scaling["type"] KeyError in standalone from_pretrained).

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[4teven]

Follow-up fix: Nanbeige4.2 remote code vs transformers>=5.x

While running training on the production GPU cluster, two crashes showed up that are not bugs in LlamaFactory itself, but incompatibilities between Nanbeige4.2's trust_remote_code files and the transformers>=5.x refactors.

Problem 1: checkpoint save crashes

AttributeError: 'list' object has no attribute 'keys'
  at remove_tied_weights_from_state_dict -> _get_tied_weight_keys

The remote modeling_nanbeige.py declares _tied_weights_keys as List[str] (old convention). transformers>=5.x refactored tied-weights handling to expect Dict[str, str] (target -> source) and calls .keys() on it during remove_tied_weights_from_state_dict, so every checkpoint save crashes.

Problem 2: standalone from_pretrained crashes

KeyError: 'type'
  at NanbeigeAttention._init_rope -> scaling_type = self.config.rope_scaling["type"]

The remote _init_rope reads rope_scaling["type"], but transformers>=5.x injects {"rope_type": "default", ...} (renamed key). This breaks standalone scripts like test_embed.py that call AutoModelForCausalLM.from_pretrained directly.

Fixes in this commit

• src/llamafactory/model/patcher.py: two compatibility shims guarded by model_type == "nanbeige".
  • patch_config: normalizes rope_scaling (rope_type -> type / None) at model load time.
  • patch_model: converts _tied_weights_keys from list to dict at model load time.
• scripts/patch_nanbeige_remote_code.py: one-shot idempotent patcher for the cached remote code on GPU clusters, for standalone scripts that bypass LlamaFactory.

Note

These shims are workarounds for outdated trust_remote_code and will become no-ops once the upstream Nanbeige HF repo (Nanbeige/Nanbeige4.2-3B) ships transformers>=5.x-compatible remote code (_tied_weights_keys as dict, _init_rope reading rope_type). Each shim is annotated with this context so it can be removed cleanly later.