New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

关于pretrain_yuan_13B.sh微调相关问题 #14

Closed

jiangliqin opened this issue Mar 29, 2022 · 2 comments

jiangliqin commented Mar 29, 2022 •

edited

Loading

1.CHECKPOINT_PATH中gpt3_case11_300B应该有基础模型吧？这个有点困惑，请帮忙解答下
2.关于data_path_aug.txt数据配置文件，开头是采样比例，接着是具体数据文件，请问数据的格式怎样的，如果只需要在下游一个具体任务上微调只有一个数据文件，就是1 /xxxxx.txt吗？ #9
请帮忙解答上面几个问题，谢谢~

Contributor

zhaoxudong01-ieisystem commented Apr 6, 2022 •

edited

Loading

gpt3_case11_300B是预训练的130亿参数的模型。
数据文件是通过tools/preprocess_data_cn.py对txt文件处理得到的
只在校友任务上做微调，可以参考tasks目录下对不同任务的数据处理，与训练过程的数据格式有所不同

1.CHECKPOINT_PATH中gpt3_case11_300B应该有基础模型吧？这个有点困惑，请帮忙解答下 2.关于data_path_aug.txt数据配置文件，开头是采样比例，接着是具体数据文件，请问数据的格式怎样的，如果只需要在下游一个具体任务上微调只有一个数据文件，就是1 /xxxxx.txt吗？ #9 请帮忙解答上面几个问题，谢谢~

Author

jiangliqin commented Apr 6, 2022

请问gpt3_case11_300B有开源计划吗？

Shawn-IEITSystems closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment