Skip to content

[Bug] Qwen3-next mtp topk>1 accuracy issue #18831

Description

@hanming-lu

Checklist

  • I searched related issues but found no solution.
  • The bug persists in the latest version.
  • Issues without environment info and a minimal reproducible demo are hard to resolve and may receive no feedback.
  • If this is not a bug report but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
  • Please use English. Otherwise, it will be closed.

Describe the bug

GDN model spec dec topk > 1 path has bug and can generate garbage output. The easiest repro test is this: https://github.com/sgl-project/sglang/blob/main/test/registered/4-gpu-models/test_qwen3_next_models_mtp.py#L30

The threshold for mtp should be the same as non-mtp, at "kl_div": 0.0025, instead of currently 0.008.

Reproduction

  1. update kl threshold to 0.0025, same as non-mtp threshold https://github.com/sgl-project/sglang/blob/main/test/registered/4-gpu-models/test_qwen3_next_models_mtp.py#L30
  2. run TestQwen3NextMTP

Environment

main default environment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions