Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PaddlePaddle Hackathon 57 提交 #1128

Merged
merged 17 commits into from
Nov 29, 2021
Prev Previous commit
Next Next commit
Update
deeplaying committed Nov 28, 2021
commit 2341c3de86cac96f4fb3b17806ac803757910cf2
Original file line number Diff line number Diff line change
@@ -7,8 +7,8 @@ https://huggingface.co/sshleifer/tiny-distilbert-base-uncased-finetuned-sst-2-en
import paddle
from paddlenlp.transformers import DistilBertForSequenceClassification, DistilBertTokenizer

model = DistilBertForSequenceClassification.from_pretrained('renmada/distilbert-base-multilingual-cased')
tokenizer = DistilBertTokenizer.from_pretrained('renmada/distilbert-base-multilingual-cased')
model = DistilBertForSequenceClassification.from_pretrained('renmada/sshleifer-tiny-distilbert-base-uncase-finetuned-sst-2-english')
tokenizer = DistilBertTokenizer.from_pretrained('renmada/sshleifer-tiny-distilbert-base-uncase-finetuned-sst-2-english')
inp = 'It is good'
ids = tokenizer.encode(inp)['input_ids']
ids = paddle.to_tensor([ids])
11 changes: 10 additions & 1 deletion docs/model_zoo/transformers.rst
Original file line number Diff line number Diff line change
@@ -241,7 +241,16 @@ Transformer预训练模型汇总
| |``distilbert-base-cased`` | English | 6-layer, 768-hidden, |
| | | | 12-heads, 66M parameters. |
| | | | The DistilBERT model distilled from |
| | | | the BERT model ``bert-base-cased`` |
| +----------------------------------------------------------------------------------+--------------+-----------------------------------------+
| |``distilbert-base-multilingual-cased`` | English | 6-layer, 768-hidden, |
| | | | 12-heads, 200M parameters. |
| | | | The DistilBERT model distilled from |
| | | | the BERT model |
| | | | ``bert-base-multilingual-cased`` |
| +----------------------------------------------------------------------------------+--------------+-----------------------------------------+
| |``sshleifer-tiny-distilbert-base-uncase-finetuned-sst-2-english`` | English | 2-layer, 2-hidden, |
| | | | 2-heads, 50K parameters. |
| | | | The DistilBERT model |
+--------------------+----------------------------------------------------------------------------------+--------------+-----------------------------------------+
|ELECTRA_ |``electra-small`` | English | 12-layer, 768-hidden, |
| | | | 4-heads, _M parameters. |
2 changes: 1 addition & 1 deletion paddlenlp/transformers/xlnet/modeling.py
Original file line number Diff line number Diff line change
@@ -1854,7 +1854,7 @@ def forward(
return_dict=return_dict, )
output = transformer_outputs if not return_dict \
else transformer_outputs["last_hidden_state"]
logits = self.classifier(output)
logits = self.qa_outputs(output)
logits = paddle.transpose(logits, perm=[2, 0, 1])
start_logits, end_logits = paddle.unstack(x=logits, axis=0)
return start_logits, end_logits