Qwen3 not stopping generation after lora finetuning

[dittops]

Reminder

• I have read the above rules and searched the existing issues.

System Info

LLaMA-Factory.git@6a584b40928fb6d69e22c1403db226eb04358a30#egg=llamafactory
Python 3.11.12
Ubunutu 22.04

Nvidia A100 80GB

Reproduction

I'm training Qwen/Qwen3-4B-Base with language adapters. For SFT full parameter fine-tuning, the generation is coming out as expected. But when I train the same data and model with Lora, the generation is not stopping. Here is my train config

### model
model_name_or_path: Qwen/Qwen3-4B-Base
trust_remote_code: true
flash_attn: fa2

### method
stage: sft
do_train: true
finetuning_type: lora
lora_rank: 16
lora_target: all
deepspeed: examples/deepspeed/ds_z2_config.json

### dataset
dataset: alpaca_hindi
template: qwen3
overwrite_cache: true
preprocessing_num_workers: 16
dataloader_num_workers: 4

### output
output_dir: saves/qwen3-4b-base/lora/sft-hindi-v2
logging_steps: 10
plot_loss: true
overwrite_output_dir: true
save_only_model: true
report_to: wandb  # choices: [none, wandb, tensorboard, swanlab, mlflow]

### train
per_device_train_batch_size: 16
gradient_accumulation_steps: 1
learning_rate: 1.0e-4
num_train_epochs: 5.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000
resume_from_checkpoint: null

Others

No response

[dittops]

Instead of end token, it generates another token. And this is not consistent, when I reload I can see some other token.

[hiyouga]

You can try template: default for base models

[dittops]

I have tried the template: default for Qwen 3, and it is still the same. But if I change the model to LLama 3b base, then it's working fine.

[hiyouga]

maybe the eos token was wrong in qwen3 base model. You can use additional_target: embed_tokens to fine-tune the embedding tokens when using lora

[dittops]

I have tried enabling that. Still getting the same issue

### model
model_name_or_path: Qwen/Qwen3-4B-Base
trust_remote_code: true
flash_attn: fa2

### method
stage: sft
do_train: true
finetuning_type: lora
lora_rank: 16
lora_target: all
additional_target: embed_tokens
deepspeed: examples/deepspeed/ds_z2_config.json

### dataset
dataset: alpaca_en
template: default
# cutoff_len: 2048
# max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 16
dataloader_num_workers: 4

### output
output_dir: saves/qwen3-4b-base/lora/sft-test
logging_steps: 10
save_steps: 5000
plot_loss: true
overwrite_output_dir: true
save_only_model: false
report_to: wandb  # choices: [none, wandb, tensorboard, swanlab, mlflow]

### train
per_device_train_batch_size: 16
gradient_accumulation_steps: 1
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000
resume_from_checkpoint: null

[hixulei]

遇到了一样的问题

[hiyouga]

Try setting the eos token in the tokenizer config to <|endoftext|> and using default template if you are fine-tuning base model using lora
https://huggingface.co/Qwen/Qwen3-4B-Base/blob/main/tokenizer_config.json#L232

[SelenoChannel]

Try setting the eos token in the tokenizer config to <|endoftext|> and using default template if you are fine-tuning base model using lora https://huggingface.co/Qwen/Qwen3-4B-Base/blob/main/tokenizer_config.json#L232

It worked! Thanks!