【Hackathon 6th Article No.6】稀疏计算的使用指南&学习心得 #869
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ticle/【Hackathon 6th Article No.6】 稀疏计算的使用指南&学习心得.md
…0240329_guide_to_using_sparseoperation.md
zhwesky2010
reviewed
Apr 17, 2024
|
|
||
| ## 3.稀疏格式介绍 | ||
|
|
||
| PaddlePaddle框架提供了对稀疏计算的高效支持。它允许开发者利用稀疏数据的特性，通过特定的操作来优化模型训练。这些操作不仅包括对稀疏矩阵和稀疏张量的直接操作，还包括针对稀疏数据的特殊优化算法，如稀疏梯度下降等。 |
There was a problem hiding this comment.
关于这个稀疏格式，最好通过图形化的方式来讲解，让读者能够清晰易懂的看明白
| **SparseCOO**格式通过坐标列表（**COOrdinates**）来存储非零元素的位置和值，适合于随机分布的非零元素。而SparseCSR格式则通过压缩稀疏行（**Compressed Sparse Row**）的形式存储数据，适合于行内元素较为集中的情况。 | ||
|
|
||
| ## 4.创建稀疏张量 | ||
| 在**PaddlePaddle**中，我们可以通过创建**paddle.sparse.SparseTensor**对象来指定稀疏格式，并指定相应的非零元素和坐标信息。 |
| ``` | ||
| ## 5. 稀疏张量的操作 | ||
|
|
||
| PaddlePaddle支持对稀疏张量进行多种操作，如加法、乘法等。这些操作通常比直接对稠密张量进行操作更高效。 |
There was a problem hiding this comment.
介绍一下当前支持哪些稀疏运算，让读者对paddle的稀疏运算能力有一个了解
| class SparseResNet(nn.Layer): | ||
| def __init__(self): | ||
| super(SparseResNet, self).__init__() | ||
| self.conv1 = nn.Conv2D(...) |
|
|
||
| # 定义损失函数和优化器 | ||
| criterion = nn.CrossEntropyLoss() | ||
| optimizer = paddle.optimizer.Adam(parameters=model.parameters()) |
|
|
||
| **PaddlePaddle**支持了多种稀疏计算操作，包括但不限于矩阵乘法、稀疏张量的加法、转置等。这使得我们可以使用稀疏**Tensor**来构建和训练各种经典神经网络模型。 | ||
|
|
||
| 例如，在3D点云处理任务中，我们通常会使用稀疏卷积网络来处理稀疏的点云数据。**PaddlePaddle**通过支持稀疏卷积操作，使得我们可以方便地构建稀疏卷积网络模型。 |
| import paddle.sparse as sparse | ||
|
|
||
| # 假设我们有一个稀疏的输入tensor | ||
| sparse_input = sparse.SparseTensor(...) |
There was a problem hiding this comment.
这行代码不对，没有sparse.SparseTensor
| sparse_input = sparse.SparseTensor(...) | ||
|
|
||
| # 定义Sparse Transformer模型 | ||
| class SparseTransformer(nn.Layer): |
There was a problem hiding this comment.
这些代码看起来和稀疏Transformer没什么关系，没有用到相关的稀疏API
|
【Hackathon 6th】优秀稿件征集与传播：按照活动规则，阶段一已于 5 月 15 日 截止，研发不再审核稿件 PR |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.