[Hackthon No.26] #31
Merged
[Hackthon No.26] #31
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
PR 格式检查通过，你的PR 将接受Paddle 专家以及开源社区的review，请及时关注PR 动态 |
shiyutang
requested changes
Mar 15, 2022
There was a problem hiding this comment.
- 整体有较多位置修改，请参考案例：https://github.com/PaddlePaddle/community/blob/master/rfcs/APIs/20200301_api_design_for_quantile.md
- 另外需要恢复误删的 rfcs/APIs/api_design_template.md 文件
Comment on lines
27
to
45
| Pytorch 中有相关的`torch.nn.functional.triplet_margin_loss(anchor, positive, negative, margin=1.0, p=2, eps=1e-06, swap=False, size_average=None, reduce=None, reduction='mean') -> Tensor`和`torch.nn.TripletMarginLoss(margin=1.0, p=2.0, eps=1e-06, swap=False, size_average=None, reduce=None, reduction='mean') -> Tensor` | ||
|
|
||
| 在 pytorch 中，介绍为： | ||
|
|
||
| > Creates a criterion that measures the triplet loss given an input tensors $x 1, x 2, x 3$ and a margin with a value greater than 0 . This is used for measuring a relative similarity between samples. A triplet is composed by a, $p$ and $n$ (i.e., anchor, positive examples and negative examples respectively). The shapes of all input tensors should be $(N, D)$. | ||
| > | ||
| > The distance swap is described in detail in the paper Learning shallow convolutional feature descriptors with triplet losses by V. Balntas, E. Riba et al. | ||
| > The loss function for each sample in the mini-batch is: | ||
| > | ||
| > $$ | ||
| > L(a, p, n)=\max \left\{d\left(a_{i}, p_{i}\right)-d\left(a_{i}, n_{i}\right)+\operatorname{margin}, 0\right\} | ||
| > $$ | ||
| > | ||
| > where | ||
| > | ||
| > $$ | ||
| > d\left(x_{i}, y_{i}\right)=\left\|\mathbf{x}_{i}-\mathbf{y}_{i}\right\|_{p} | ||
| > $$ | ||
|
|
There was a problem hiding this comment.
| 为 paddle 框架中新增计算损失函数的方法 | ||
|
|
||
| # 二、飞桨现状 | ||
| 对飞桨框架目前支持此功能的现状调研，如果不支持此功能，如是否可以有替代实现的API，是否有其他可绕过的方式，或者用其他API组合实现的方式； |
Comment on lines
23
to
24
| 目前paddle缺少相关功能实现。 | ||
| 需要独立设计实现相关的函数 |
There was a problem hiding this comment.
可以补充其他loss的实现方式，例如MSE Loss，从而明确参数规范等
| 目前paddle缺少相关功能实现。 | ||
| 需要独立设计实现相关的函数 | ||
|
|
||
| # 三、业内方案调研 |
There was a problem hiding this comment.
| > $$ | ||
|
|
||
| # 四、对比分析 | ||
| paddle 和 pytorch整体框架相似，故直接采用其方法进行设计。 |
There was a problem hiding this comment.
需要对torch和tf实现对比，二者的对参数的支持，输入维度的支持，距离函数的选择等都有所不同，这些都可以对比，并说明 paddle 的实现中应当如何选择
| # 五、设计思路与实现方案 | ||
|
|
||
| ## 命名与参数设计 | ||
| 参考：[飞桨API 设计及命名规范] |
Comment on lines
55
to
56
| - `paddle.nn.TripletMarginLoss(margin=1.0, p=2.0, epsilon=1e-06, swap=False, reduction='mean', name=None) -> Tensor` | ||
| - `padde.nn.functional.triplet_margin_loss(input, positive, negative, margin=1.0, p=2, epsilon=1e-06, swap=False, reduction='mean', name=None) -> Tensor` |
|
|
||
| 1. 检查 reduction 有效性（同其余 functional loss 中的实现） | ||
| 2. 检查输入的 dtype（含 `anchor`、`positive`、`negative`）（同其余 functional loss 中的实现） | ||
|
|
There was a problem hiding this comment.
- 检查
anchor、positive、negative三者维度是否对齐
|
|
||
| 测试考虑的case如下: | ||
| 1.动态图，静态图，要与np计算下的结果输出需要一致。 | ||
| 2.在swap下，动态图静态图输出结果一致。 |
There was a problem hiding this comment.
|
你的PR有最新反馈，请及时修改。 |
|
如果修改完整，请rerequest-review |
shiyutang
requested changes
Mar 22, 2022
|
|
||
| 2. 计算 | ||
|
|
||
| 1. 使用 `paddle.nn.Pairwisedistance` 分别计算得到正锚点与样本和负锚点与样本的距离。 |
There was a problem hiding this comment.
可以替换为 paddle.linalg.norm 底层和nn.PairwiseDistance 调用相同的 pnorm OP，并且可以直接进行函数调用
|
|
||
| ``` | ||
| padde.nn.functional.triplet_margin_loss(input Tensor[float64 or float32] 维度为[batch_size,dim] | ||
| positive, 1 or -1 Tensor[float64 or float32],维度为[batch_size,dim] negative, 1 or -1 Tensor[float64 or float32],维度为[batch_size,dim] |
There was a problem hiding this comment.
为什么positive和negative需要限制为-1/1的tensor？
| 1. 检查参数 | ||
|
|
||
| 1. 检查 reduction 有效性（同其余 functional loss 中的实现） | ||
| 2. 检查输入的 dtype（含 `input`、`positive`、`negative`）（同其余 functional loss 中的实现） |
There was a problem hiding this comment.
由于输入损失的tensor不一定为（N，Dim）的形状，还需要对输入进行维度转换
|
你的PR已合入community库，请进行后续代码开发，并将代码提交至Paddle仓库。 |
This was referenced Apr 12, 2022
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.