Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/allow different encoder and decoder feature dimensions in transformer decoder layer #1260

Conversation

ferraric
Copy link
Contributor

@ferraric ferraric commented Oct 1, 2023

Fixes #1235

Thanks @fdtomasi for the detailed issue report, made fixing this quite easy.

@mattdangerw
Copy link
Member

/gcbrun

Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Looks good to me, thanks for the change

@mattdangerw mattdangerw merged commit 84c2122 into keras-team:master Oct 2, 2023
@ferraric
Copy link
Contributor Author

ferraric commented Oct 2, 2023

Awesome! Looks good to me, thanks for the change

You're welcome! Any idea when this might make it into a release?

@mattdangerw
Copy link
Member

You're welcome! Any idea when this might make it into a release?

Good question! The next big release will probably be timed along with the initial Keras 3 pip. That's this year, but might not be the month.

But we can and should cut a small point release in the next couple weeks if the big release is going to drag on a bit.

@ferraric
Copy link
Contributor Author

ferraric commented Oct 6, 2023

You're welcome! Any idea when this might make it into a release?

Good question! The next big release will probably be timed along with the initial Keras 3 pip. That's this year, but might not be the month.

But we can and should cut a small point release in the next couple weeks if the big release is going to drag on a bit.

Thanks for the info. A release soon would be appreciated since I am waiting on this feature. Of course only if possible and not too much overhead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TransformerDecoder with CrossAttention does not work when query and key have different dimensionality
2 participants