Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【PaddlePaddle Hackathon】21、在 Paddle 中实现基于 DBTree 拓扑的 AllReduce #35989

Closed
TCChenlong opened this issue Sep 23, 2021 · 1 comment

Comments

@TCChenlong
Copy link
Contributor

(此 ISSUE 为 PaddlePaddle Hackathon 活动的任务 ISSUE,更多详见PaddlePaddle Hackathon

【任务说明】

  • 任务标题:在 Paddle 中实现基于 DBTree 拓扑的 AllReduce

  • 技术标签:深度学习框架,C++,通信拓扑

  • 任务难度:困难

  • 详细描述: DBTree 主要思想是利用二叉树中大约一半节点是叶子节点的特性,通过将叶子节点变换为非叶子节点,得到两颗二叉树,每个节点在其中一颗二叉树上是叶子节点,在另一颗二叉树上是非叶子节点。这种方法理论上是能够提供比 Ring 算法更低的延迟。此任务的目标是在 Paddle 分布式训练框架中,实现 DBTree 结构的 AllReduce,除使用 GPU 进行训练外,最好也能支持其他异构硬件。

    NCCL 参考:https://developer.nvidia.com/blog/massively-scale-deep-learning-training-nccl-2-4/

【提交内容】

  • 任务提案

  • 任务 PR 到 Paddle

  • 相关技术文档

  • 任务单测文件

【技术要求】

  • 了解 Paddle 分布式训练框架

  • 熟练掌握 C++ 、Python

  • 熟悉模型训练和集合通信实现、DBTree 通信算法

@paddle-bot-old
Copy link

您好,我们已经收到了您的问题,会安排技术人员尽快解答您的问题,请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时,您也可以通过查看官网API文档常见问题历史IssueAI社区来寻求解答。祝您生活愉快~

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the APIFAQGithub Issue and AI community to get the answer.Have a nice day!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants