Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add model Rembert #1701

Merged
merged 14 commits into from
Apr 14, 2022
Prev Previous commit
Next Next commit
Update modeling.py
Beacontownfc authored Apr 13, 2022

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit d9c8eb34f78ae9374ee2e67f2679b72e147a58bd
2 changes: 1 addition & 1 deletion paddlenlp/transformers/rembert/modeling.py
Original file line number Diff line number Diff line change
@@ -126,7 +126,7 @@ def __init__(self,
layer_norm_eps=1e-12):
super(RemBertEmbeddings, self).__init__()
self.word_embeddings = nn.Embedding(
vocab_size, input_embedding_size, padding_idx=pad_token_id)
vocab_size, input_embedding_size)
self.position_embeddings = nn.Embedding(max_position_embeddings,
input_embedding_size)
self.token_type_embeddings = nn.Embedding(type_vocab_size,