Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

如果我想把训练后的模型权重由32bit转化为16bit #4731

Closed
1 task done
Suiji12 opened this issue Jul 9, 2024 · 2 comments
Closed
1 task done

如果我想把训练后的模型权重由32bit转化为16bit #4731

Suiji12 opened this issue Jul 9, 2024 · 2 comments
Labels
solved This problem has been already solved

Comments

@Suiji12
Copy link

Suiji12 commented Jul 9, 2024

Reminder

  • I have read the README and searched the existing issues.

System Info

用galore进行全参微调，运行：llamafactory-cli train examples/extras/galore/llama3_full_sft.yaml发现模型的权重是32bit的，请问如何将他们转化为16bit的模型权重？

Reproduction

或者能否在yaml文件中直接指定？

Expected behavior

No response

Others

No response
@github-actions github-actions bot added the pending This problem is yet to be addressed label Jul 9, 2024
@hiyouga
Copy link
Owner

hiyouga commented Jul 10, 2024

导出时候添加参数 infer_dtype: float16

@hiyouga hiyouga added solved This problem has been already solved and removed pending This problem is yet to be addressed labels Jul 10, 2024
@hiyouga hiyouga mentioned this issue Jul 10, 2024
Closed
1 task
@Suiji12
Copy link
Author

Suiji12 commented Jul 10, 2024

感谢您的回复

xtchen96 pushed a commit to xtchen96/LLaMA-Factory that referenced this issue Jul 17, 2024
@hiyouga @xtchen96
fix hiyouga#4731
ff575f7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
solved This problem has been already solved
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hiyouga @Suiji12