默认参数建议：sft阶段是否考虑将学习率调小

[bayma-1]

Reminder

• I have read the README and searched the existing issues.

System Info

• llamafactory version: 0.8.4.dev0
• Platform: Linux-5.4.0-60-generic-x86_64-with-glibc2.17
• Python version: 3.8.19
• PyTorch version: 2.2.2+cu121 (GPU)
• Transformers version: 4.41.2
• Datasets version: 2.19.1
• Accelerate version: 0.31.0
• PEFT version: 0.11.1
• TRL version: 0.8.6
• GPU type: Tesla V100-PCIE-32GB
• DeepSpeed version: 0.14.2

Reproduction

model

model_name_or_path: path_to_Qwen2-7B-Instruct

method

stage: sft
do_train: true
finetuning_type: full
deepspeed: examples/deepspeed/ds_z3_config.json

dataset

dataset: alpaca_data_en_52k
template: qwen
cutoff_len: 1024
max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 16

output

output_dir: saves/qwen2/full/sft
logging_steps: 1
save_steps: 50000
plot_loss: true
overwrite_output_dir: true

train

per_device_train_batch_size: 1
gradient_accumulation_steps: 2
learning_rate: 1.0e-5
max_steps: 1000
#num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
fp16: true
ddp_timeout: 180000000

eval

val_size: 0.1
per_device_eval_batch_size: 1
eval_strategy: steps
eval_steps: 500

Expected behavior

【感谢查看】

1. 以下是我使用 qwen2-7b-instruct 模型在默认参数的基础上，只修改学习率得到的 loss 曲线。发现学习率为 1e-5 的时候收敛效果要明显好于 1e-4。
   
2. 为了验证 1e-4 的学习率是否对于 qwen2-7b-instruct 模型是更大的，我查阅了 qwen2 的论文，发现在 sft 阶段使用的学习率在 7e-6 到 7e-7 之间。
   
3. 请问是否考虑在 sft 阶段调小默认学习率？

Others

No response

[hiyouga]

fixed