-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
新IR Python API适配升级 #58067
Comments
【报名】:2、3、12、13、16、18、78-82 |
【报名】:105, 106, 166 |
【报名】:4, 5, 6, 7, 8, 9 |
【报名】:97, 98, 99, 100, 209, 210, 211 |
【报名】:133,134,135 |
【报名】:50-70 |
【报名】: |
【报名】: |
【报名】:231-232 |
【报名】:229,230,208 |
【报名】:202、205、254、262-264、267-268 |
【报名】: 298、300、302-306、309 |
【报名】:270、271、272、287、289、291、294、296、297 |
【报名】:311、312、301、290、259、188、163、165 |
【报名】183 |
认领 88、164 |
新IR Python API适配升级 已基本完成,感谢参与的小伙伴们!
欢迎继续参与快乐开源的其他任务! |
一、BackGround 📚
⭐️ 提交PR 模版 ⭐️:
注意事项:
PR 样板间 🥇
🏅 【PIR API adaptor No.54-56】Migrate some ops into pir
🏅 【PIR API adaptor No.24-26】Migrate paddle.bitwise_not/bitwise_or/bitwise_xor into pir
🏅 【PIR API adaptor No.139】 Migrate logsumexp into pir
🏅 【PIR API adaptor No.224】 Migrate paddle.tan
🏅 【PIR API adaptor No.27、29、102 】 Migrate bmm/broadcast_tensors/histogram into pir
奖励一览:
二、Task 🙋🏻♀️
请大家以 comment 的形式认领任务,如:
多个任务之间需要使用中文顿号分隔,报名多个连续任务可用横线表示,如 2-5
PR 提交格式:在 PR 的标题中以 【PIR API adaptor No.xxx】 开头,注明任务编号
本期需要升级的 API 如下:
python/paddle/nn/layer/loss.py
python/paddle/tensor/ops.py
python/paddle/tensor/ops.py
python/paddle/nn/functional/vision.py
@enkilee #59661
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
paddle.Tensor.argmin
python/paddle/tensor/search.py
@0x45f #57909
p1python/paddle/tensor/search.py
@enkilee #59661
python/paddle/tensor/manipulation.py
python/paddle/tensor/manipulation.py
python/paddle/tensor/ops.py
python/paddle/tensor/ops.py
python/paddle/nn/initializer/assign.py
@MarioLulab #60114
python/paddle/nn/initializer/Bilinear.py
@MarioLulab #60114
python/paddle/tensor/ops.py
python/paddle/tensor/math.py
python/paddle/tensor/ops.py
python/paddle/incubate/optimizer/modelaverage.py
python/paddle/nn/layer/loss.py
python/paddle/tensor/random.py
python/paddle/nn/functional/common.py
python/paddle/tensor/linalg.py
@GreatV #58692
python/paddle/tensor/logic.py
python/paddle/tensor/logic.py
python/paddle/tensor/logic.py
python/paddle/tensor/linalg.py
python/paddle/vision/ops.py
python/paddle/tensor/manipulation.py
python/paddle/distribution/distribution.py
python/paddle/distribution/normal.py
python/paddle/distribution/uniform.py
python/paddle/tensor/math.py
@changeyoung98 #60619
python/paddle/nn/layer/activation.py
python/paddle/nn/layer/vision.py
python/paddle/amp/debugging.py
python/paddle/tensor/linalg.py
python/paddle/tensor/linalg.py
python/paddle/nn/functional/common.py
python/paddle/nn/clip.py
python/paddle/tensor/creation.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/tensor/ops.py
python/paddle/tensor/manipulation.py
python/paddle/tensor/linalg.py
python/paddle/nn/functional/loss.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/vision/ops.py
@enkilee #58955
python/paddle/vision/ops.py
@MarioLulab #60168
python/paddle/tensor/linalg.py
@zrr1999 #58384
python/paddle/tensor/creation.py
python/paddle/nn/functional/extension.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/distribution/dirichlet.py
python/paddle/tensor/linalg.py
python/paddle/vision/ops.py
python/paddle/tensor/linalg.py
python/paddle/tensor/linalg.py
python/paddle/tensor/linalg.py
python/paddle/nn/layer/activation.py
python/paddle/tensor/creation.py
python/paddle/tensor/logic.py
@zrr1999 #58287
python/paddle/tensor/math.py
python/paddle/tensor/manipulation.py
@longranger2
python/paddle/tensor/creation.py
python/paddle/nn/functional/flash_attention.py
python/paddle/tensor/manipulation.py
@enkilee #59091
python/paddle/tensor/math.py
@enkilee #58718
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/nn/layer/common.py
python/paddle/nn/functional/loss.py
python/paddle/nn/initializer/dirac.py
@MarioLulab #59911
python/paddle/incubate/nn/layer/fused_transformer.py
python/paddle/incubate/nn/functional/fused_transformer.py
python/paddle/incubate/nn/functional/fused_layer_norm.py
python/paddle/incubate/nn/layer/fused_dropout_add.py
python/paddle/incubate/nn/layer/fused_transformer.py
python/paddle/incubate/nn/layer/fused_linear.py
@enkilee #58911
python/paddle/incubate/nn/functional/fused_matmul_bias.py
@enkilee #58911
python/paddle/incubate/nn/functional/fused_matmul_bias.py
@enkilee #58911
python/paddle/incubate/nn/functional/fused_transformer.py
@enkilee #58911
python/paddle/incubate/nn/functional/fused_rotary_position_embedding.py
@enkilee #58911
python/paddle/incubate/operators/softmax_mask_fuse.py
python/paddle/nn/functional/extension.py
@enkilee #58792
python/paddle/vision/ops.py
python/paddle/incubate/operators/graph_khop_sampler.py
python/paddle/incubate/operators/graph_reindex.py
python/paddle/incubate/operators/graph_sample_neighbors.py
python/paddle/geometric/sampling/neighbors.py
python/paddle/nn/functional/vision.py
python/paddle/nn/layer/norm.py
python/paddle/nn/layer/activation.py
python/paddle/nn/layer/activation.py
python/paddle/nn/layer/activation.py
python/paddle/nn/layer/activation.py
python/paddle/tensor/math.py
python/paddle/tensor/linalg.py
python/paddle/nn/layer/loss.py
python/paddle/nn/layer/loss.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/incubate/nn/loss.py
python/paddle/tensor/attribute.py
python/paddle/tensor/manipulation.py
python/paddle/tensor/manipulation.py
python/paddle/tensor/search.py
python/paddle/tensor/search.py
2. paddle.nn.InstanceNorm2D
3. paddle.nn.InstanceNorm3D
python/paddle/nn/layer/norm.py
python/paddle/tensor/math.py
python/paddle/tensor/logic.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/nn/layer/loss.py
python/paddle/tensor/math.py
python/paddle/tensor/search.py
python/paddle/nn/functional/common.py
python/paddle/optimizer/lamb.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/nn/quant/quantized_linear.py
python/paddle/nn/functional/loss.py
python/paddle/nn/layer/activation.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/tensor/logic.py
python/paddle/tensor/logic.py
python/paddle/tensor/logic.py
python/paddle/tensor/math.py
python/paddle/nn/layer/activation.py
python/paddle/tensor/creation.py
python/paddle/tensor/math.py
python/paddle/tensor/linalg.py
python/paddle/tensor/linalg.py
python/paddle/tensor/linalg.py
python/paddle/nn/functional/loss.py
python/paddle/incubate/nn/functional/masked_multihead_attention.py
python/paddle/tensor/search.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/vision/ops.py
python/paddle/tensor/linalg.py
python/paddle/tensor/linalg.py
python/paddle/nn/layer/activation.py
python/paddle/tensor/creation.py
python/paddle/tensor/search.py
python/paddle/tensor/linalg.py
python/paddle/tensor/math.py
python/paddle/tensor/linalg.py
python/paddle/tensor/stat.py
@Liyulingyue #58889
python/paddle/tensor/math.py
python/paddle/nn/layer/loss.py
@ooooo-create
python/paddle/vision/ops.py
python/paddle/nn/functional/input.py
python/paddle/nn/functional/norm.py
@MarioLulab
2. paddle.nn.Pad2D
3. paddle.nn.Pad3D
4. paddle.nn.ZeroPad2D
python/paddle/nn/layer/common.py
python/paddle/nn/layer/vision.py
@MarioLulab
python/paddle/tensor/math.py
2. paddle.nn.AvgPool1D
python/paddle/nn/layer/pooling.py
2. paddle.nn.AvgPool3D
python/paddle/nn/layer/pooling.py
python/paddle/nn/layer/activation.py
python/paddle/vision/ops.py
python/paddle/vision/ops.py
python/paddle/tensor/manipulation.py
python/paddle/tensor/linalg.py
python/paddle/tensor/random.py
python/paddle/vision/ops.py
python/paddle/tensor/ops.py
python/paddle/geometric/reindex.py
python/paddle/nn/layer/activation.py
paddle.Tensor.floor_mod
python/paddle/tensor/math.py
p1python/paddle/tensor/math.py
python/paddle/tensor/manipulation.py
python/paddle/optimizer/rmsprop.py
2. paddle.nn.LSTM
3. paddle.nn.SimpleRNN
python/paddle/nn/layer/rnn.py
@changeyoung98 #60180
python/paddle/vision/ops.py
python/paddle/vision/ops.py
python/paddle/tensor/manipulation.py
@enkilee #59091
python/paddle/nn/layer/activation.py
@xiaoyewww #59313
python/paddle/regularizer.py
@MarioLulab #60292
python/paddle/tensor/math.py
python/paddle/tensor/manipulation.py
python/paddle/tensor/manipulation.py
python/paddle/tensor/search.py
@MarioLulab #60014
python/paddle/geometric/math.py
python/paddle/geometric/math.py
python/paddle/geometric/math.py
python/paddle/geometric/math.py
python/paddle/nn/layer/activation.py
python/paddle/incubate/operators/graph_send_recv.py
python/paddle/geometric/message_passing/send_recv.py
python/paddle/geometric/message_passing/send_recv.py
python/paddle/geometric/message_passing/send_recv.py
python/paddle/nn/functional/extension.py
python/paddle/tensor/manipulation.py
python/paddle/nn/layer/loss.py
python/paddle/regularizer.py
python/paddle/tensor/math.py
python/paddle/tensor/ops.py
python/paddle/tensor/linalg.py
python/paddle/nn/layer/activation.py
python/paddle/nn/layer/activation.py
python/paddle/nn/layer/activation.py
python/paddle/tensor/manipulation.py
python/paddle/nn/layer/common.py
python/paddle/tensor/math.py
python/paddle/tensor/manipulation.py
python/paddle/nn/layer/loss.py
python/paddle/nn/functional/loss.py
python/paddle/nn/layer/distance.py
python/paddle/tensor/linalg.py
python/paddle/tensor/linalg.py
python/paddle/nn/layer/activation.py
python/paddle/nn/layer/norm.py
python/paddle/tensor/manipulation.py
@xiaoyewww
python/paddle/tensor/ops.py
python/paddle/nn/layer/activation.py
python/paddle/nn/functional/extension.py
python/paddle/nn/layer/activation.py
python/paddle/tensor/manipulation.py
python/paddle/tensor/linalg.py
python/paddle/tensor/linalg.py
python/paddle/tensor/creation.py
python/paddle/tensor/creation.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/tensor/manipulation.py
python/paddle/nn/layer/common.py
python/paddle/nn/initializer/uniform.py
python/paddle/nn/initializer/kaiming.py
@MarioLulab #59419
python/paddle/nn/initializer/kaiming.py
python/paddle/nn/initializer/xavier.py
python/paddle/nn/initializer/xavier.py
python/paddle/tensor/manipulation.py
2. paddle.nn.MaxUnPool2D
python/paddle/nn/layer/pooling.py
python/paddle/nn/layer/pooling.py
python/paddle/tensor/manipulation.py
python/paddle/incubate/nn/functional/variable_length_memory_efficient_attention.py
python/paddle/text/viterbi_decode.py
python/paddle/nn/layer/loss.py
python/paddle/nn/layer/loss.py
python/paddle/geometric/sampling/neighbors.py
python/paddle/vision/ops.py
python/paddle/vision/ops.py
python/paddle/tensor/math.py
python/paddle/optimizer/lr.py
python/paddle/incubate/nn/layer/fused_transformer.py
python/paddle/nn/layer/loss.py
python/paddle/tensor/math.py
@enkilee #59600
python/paddle/nn/functional/activation.py
python/paddle/nn/layer/loss.py
@MarioLulab #60291
python/paddle/audio/functional/functional.py
python/paddle/tensor/logic.py
python/paddle/distribution/laplace.py
python/paddle/optimizer/lbfgs.py
python/paddle/incubate/optimizer/lbfgs.py
python/paddle/tensor/math.py
python/paddle/tensor/math.py
python/paddle/optimizer/lr.py
python/paddle/nn/layer/norm.py
python/paddle/tensor/math.py
python/paddle/audio/features/layers.py
python/paddle/distribution/lognormal.py
python/paddle/incubate/optimizer/lookahead.py
python/paddle/tensor/stat.py
python/paddle/audio/functional/functional.py
python/paddle/audio/functional/functional.py
python/paddle/audio/features/layers.py
python/paddle/audio/features/layers.py
python/paddle/nn/layer/activation.py
python/paddle/nn/layer/loss.py
python/paddle/nn/layer/loss.py
python/paddle/nn/layer/transformer.py
python/paddle/tensor/math.py
@xiaoyewww #59469
python/paddle/tensor/math.py
@xiaoyewww
python/paddle/tensor/stat.py
python/paddle/tensor/math.py
@xiaoyewww
python/paddle/tensor/math.py
@xiaoyewww
python/paddle/nn/functional/loss.py
python/paddle/nn/initializer/orthogonal.py
python/paddle/tensor/linalg.py
python/paddle/nn/layer/loss.py
@MarioLulab #60291
python/paddle/tensor/creation.py
python/paddle/audio/functional/functional.py
@MarioLulab #60172
python/paddle/tensor/stat.py
python/paddle/vision/transforms/transforms.py
python/paddle/tensor/attribute.py
python/paddle/geometric/reindex.py
python/paddle/distribution/transform.py
python/paddle/tensor/manipulation.py
python/paddle/tensor/math.py
python/paddle/distribution/transform.py
python/paddle/nn/layer/loss.py
@MarioLulab #60291
python/paddle/nn/layer/activation.py
python/paddle/nn/layer/norm.py
python/paddle/audio/features/layers.py
python/paddle/distribution/transform.py
python/paddle/distribution/transform.py
python/paddle/tensor/math.py
python/paddle/tensor/manipulation.py
@MarioLulab #60014
python/paddle/nn/layer/transformer.py
python/paddle/tensor/math.py
python/paddle/nn/layer/loss.py
@MarioLulab #60291
python/paddle/nn/layer/loss.py
@MarioLulab #60291
python/paddle/nn/layer/common.py
python/paddle/tensor/math.py
python/paddle/tensor/stat.py
三、Tutorial 💼
对Python API进行适配升级,实现在 PIR 模式下,在 Python 端调用底层 C++ 端 PIR API。
以组网 api mean 举例:
升级前:
升级后:
再比如concat:
升级前:
升级后:
本任务中,需要关注的有三种模式判断:
为了提高 Python 端的代码复用,提升代码简洁性降低维护成本,我们对 C++ 端 PIR API 和动态图 API 接口进行了统一。正如上述 mean 所示,
_C_ops.mean
可以表示在 PIR 或者动态图下的 C++ 接口,所以为了能够在 PIR 模式下调用 C++ 接口,只需要将in_dygraph_mode()
改为in_dynamic_or_pir_mode()
即可,这样就实现了 Python 端支持 PIR API 调用。但是在 API concat当中,由于动态图中存在无法给新IR复用的代码,所以使用in_pir_mode()
新开了一个分支。四、单测验证 🔧
需要对 api 相关的单测进行修改以进行验证,单测文件中一般会有两种类型的单测:一种单测继承自
OpTest
,另一种单测继承自TestCase
以
mean
为例,其单测在文件test_mean_op.py
中,具有继承自OpTest
和继承自TestCase
两种类型的单测:1. 对于继承自
OpTest
的单测在单测函数中加入
check_pir = True
,即可实现对新 IR 下 api 的验证2. 对于继承自
TestCase
的单测首先需要确认一下这个单测是静态图模式下的代码,然后在单测函数上加装饰器
test_with_pir_api
运行单测有两种方式,任意一种跑通即可:
ctest -R test_mean_op
来运行五、注意事项 💡
1. 关于 PR 的验收规范
我们适配的 API 对应的单测原则上是应开尽开,PR 描述里需要写清楚相关单测打开情况 (xx/总数),如果有的单测开了有解决不了的 bug,可以把 bug 截图添加到 PR 描述里记录为 TODO。以
paddle.mean
的在test/legacy_test/test_mean_op.py
中的单测验证为例,说明如何统计单测验证的数量:我们以继承自
OpTest
和TestCase
的单测 class 为基本单位,如果下图所示的TestMeanOp
和TestMeanOp_ZeroDim
内的所有单测均正常运行,则单测打开数量计为 22. 关于问题记录和 BUG 修复
由于当前 PIR 还处于机制完善和验证阶段,且单测验证框架也可能存在问题,所以遇到不容易解决的问题先记录到 PR 的描述当中,有 Paddle 团队来解决。
遇到的 bug 可能会普遍出现在其他 API 适配的场景,因此为了降低开发成本,我们维护了一个 bug 修复手册 组网 API PIR 迁移 bug 修复手册 。建议把 bug 问题描述及解决方法在 组网 API PIR 迁移 bug 修复手册 以 comment 的形式进行记录。
看板信息
统计信息
The text was updated successfully, but these errors were encountered: