New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

zero grad issus in encodec? #10

Open

yuzuda283 opened this issue Jun 11, 2024 · 2 comments

yuzuda283 commented Jun 11, 2024

facebookresearch/encodec#25
will this issue have some impact on the training of speechtokenizer?

UkiTenzai commented Oct 12, 2024

好问题毕竟蒸馏时编码器输出的feature的梯度无法反传到前面的卷积层等结构

UkiTenzai commented Oct 12, 2024 •

edited

Loading

我知道了第一层码本在训练时也使用了梯度直通估计，所以feature还是记录有前面encoder的计算图的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment