qwen3 tokenizer think token

[00HS]

In the Hugging Face Qwen3-8B code, I see the following:

text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
enable_thinking=True # Switches between thinking and non-thinking modes. Default is True.
)

From the docs, it looks like enable_thinking toggles thinking vs. non-thinking modes.
When I train with torchtune, the tokenizer seems to inherit from the Qwen2 tokenizer.

My questions are:

Does this mean the default setting already includes thinking mode?

At inference time, is there a way to disable/remove thinking mode when using this tokenizer?

[happypanda94]

The issue seems to be closed, but I do not really see any answer. I am also trying to finetune Qwen3-8B full (I already completed a few epochs), when I try to infer it starts thinking. How do I disable it? Is there a way to do so in the yaml config files? If thinking is on by default, should I have disabled it during fine tuning as well? If so, how to do it in the config?

[00HS]

@happypanda94 I was able to stop the thinking behavior during inference by setting enable_thinking=False, but I’m still unsure how it affects fine-tuning.
I’ll reopen the issue to hear from others who may know more about this.

[gabrielhuang]

Can confirm the issue is still present. HF tokenizer always adds tokens, enable thinking only changes the end (enable_thinking=False forces an empty tags). Torchtune never adds tokens

HF

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B", trust_remote_code=True)

# Simple test conversation
messages = [
    {"role": "user", "content": "What is 2+2?"},
    {"role": "assistant", "content": "Think: I need to add 2 and 2.\n\nAct: answer(4)"}
]

# Test with enable_thinking=True
print("\n1. With enable_thinking=True:")
text_with_thinking = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True
)
print(repr(text_with_thinking[-300:]))

# Test with enable_thinking=False
print("\n2. With enable_thinking=False:")
text_without_thinking = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False

================================================================================
HUGGINGFACE TOKENIZER TESTS
================================================================================

1. With enable_thinking=True:
'<|im_start|>user\nWhat is 2+2?<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nThink: I need to add 2 and 2.\n\nAct: answer(4)<|im_end|>\n<|im_start|>assistant\n'

2. With enable_thinking=False:
'<|im_start|>user\nWhat is 2+2?<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nThink: I need to add 2 and 2.\n\nAct: answer(4)<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n'

Torchtune

    tokenizer = qwen3_tokenizer(
        path=f'{home}/models/Qwen3-8B/vocab.json',
        merges_file=f'{home}/models/Qwen3-8B/merges.txt'
    )

    # Same test conversation as HF tokenizer test, using TorchTune Message objects
    test_messages = [
        Message(role="user", content="What is 2+2?"),
        Message(role="assistant", content="Think: I need to add 2 and 2.\n\nAct: answer(4)")
    ]

    print("\nTorchTune Message objects:")
    for i, msg in enumerate(test_messages):
        print(f"  {i}: role={msg.role}, content={repr(msg.content[:50])}...")

    # Tokenize using TorchTune's tokenize_messages method
    # This is what TorchTune actually does during training
    tokens, mask = tokenizer.tokenize_messages(
        test_messages,
    )

    tokens_list = tokens.tolist() if hasattr(tokens, 'tolist') else list(tokens)
    print(f"\nTorchTune tokenized output:")
    print(f"  Total tokens: {len(tokens_list)}")
    print(f"  First 20 tokens: {tokens_list[:20]}")
    print(f"  Last 20 tokens: {tokens_list[-20:]}")

    # Decode back
    decoded = tokenizer.decode(tokens_list)
    print(f"\nFull decoded text (last 300 chars):")
    print(repr(decoded[-300:]))

================================================================================
TORCHTUNE TOKENIZER DIRECT TEST
================================================================================

TorchTune Message objects:
  0: role=user, content=[{'type': 'text', 'content': 'What is 2+2?'}]...
  1: role=assistant, content=[{'type': 'text', 'content': 'Think: I need to add 2 and 2.\n\nAct: answer(4)'}]...

TorchTune tokenized output:
  Total tokens: 34
  First 20 tokens: [151644, 872, 198, 3838, 374, 220, 17, 10, 17, 30, 151645, 198, 151644, 77091, 198, 38687, 25, 358, 1184, 311]
  Last 20 tokens: [198, 38687, 25, 358, 1184, 311, 912, 220, 17, 323, 220, 17, 382, 2414, 25, 4226, 7, 19, 8, 151645]

Full decoded text (last 300 chars):
'<|im_start|>user\nWhat is 2+2?<|im_end|>\n<|im_start|>assistant\nThink: I need to add 2 and 2.\n\nAct: answer(4)<|im_end|>'

✓ No <think> tags in TorchTune tokenization (as expected)

[krammnic]

Hey! I will investigate