-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added cast op oneDNN kernel for bf16/fp32 datatypes casting(FWD/BWD) #33056
Conversation
Thanks for your contribution! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@jakpiase Hi Please rename the PR title or make it more clear because I will use these titles for release note. |
python/paddle/fluid/tests/unittests/mkldnn/test_cast_mkldnn_op.py
Outdated
Show resolved
Hide resolved
python/paddle/fluid/tests/unittests/mkldnn/test_cast_mkldnn_op.py
Outdated
Show resolved
Hide resolved
python/paddle/fluid/tests/unittests/mkldnn/test_cast_mkldnn_op.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
python/paddle/fluid/tests/unittests/mkldnn/test_cast_mkldnn_op.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@luotao1 Could you please start your review? |
self.check_grad_with_place( | ||
core.CPUPlace(), ["X"], | ||
"Out", | ||
check_dygraph=False, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Paddle/python/paddle/fluid/tests/unittests/op_test.py
Lines 1076 to 1090 in 394d8d4
def check_output_with_place(self, | |
place, | |
atol=0, | |
no_check_set=None, | |
equal_nan=False, | |
check_dygraph=True, | |
inplace_atol=None): | |
self.infer_dtype_from_inputs_outputs(self.inputs, self.outputs) | |
if self.dtype == np.float64 and \ | |
self.op_type not in op_threshold_white_list.NEED_FIX_FP64_CHECK_OUTPUT_THRESHOLD_OP_LIST: | |
atol = 0 | |
if self.is_bfloat16_op(): | |
check_dygraph = False | |
if hasattr(self, 'force_fp32_output') and getattr( |
since you add
check_dygraph = False
when bf16_op in check_output_with_place
, could you add same code in check_grad_with_place
? Thus, you can decrease one approve@jakpiase @lidanqing-intel @jczaja
@wzzju Please note this.
PR types
New features
PR changes
OPs
Describe
Added cast op for 4 types of casts (fp32->fp32, fp32->bf16, bf16->fp32, bf16->bf16) for enabling Word2Vec model. OneDNN kernel can be used both in FWD and BWD.