Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added a Learning Rate Schedule for the BERT Example #96

Merged
merged 16 commits into from
Apr 11, 2022
Merged

Added a Learning Rate Schedule for the BERT Example #96

merged 16 commits into from
Apr 11, 2022

Conversation

Stealth-py
Copy link
Contributor

Fixes #84
This PR is ready for review!

@google-cla
Copy link

google-cla bot commented Apr 7, 2022

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

For more information, open the CLA check for this pull request.

mattdangerw
mattdangerw requested changes Apr 7, 2022
View reviewed changes
Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Left some comments

.gitignore Outdated Show resolved Hide resolved
examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
@Stealth-py
Copy link
Contributor Author

Thanks for the review @mattdangerw!
I've made the necessary changes and resolved the comments above.

@Stealth-py
Update .gitignore
a85e426 
Added newline
mattdangerw
mattdangerw approved these changes Apr 11, 2022
View reviewed changes
Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few more style nits, then can merge!

examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
examples/bert/run_pretraining.py Outdated Show resolved Hide resolved
@Stealth-py
Copy link
Contributor Author

Thank you! I've resolved the issues and pushed :D

@mattdangerw
Copy link
Member

@Stealth-py thanks! Can you fix formatting errors? https://github.com/keras-team/keras-nlp/runs/5979856948?check_suite_focus=true

@Stealth-py
Copy link
Contributor Author

Yeah, I'm sorry I should've ran it before pushing into the main directory.
It should be fixed now I think.

@mattdangerw
Copy link
Member

@Stealth-py thanks! I will merge now with a few last copy edits.

@Stealth-py
Copy link
Contributor Author

Stealth-py commented Apr 11, 2022 •
edited
Loading

Alright!

@mattdangerw mattdangerw merged commit 5cb1a16 into keras-team:master Apr 11, 2022
@mattdangerw
Copy link
Member

@Stealth-py no need! We generally merge through github which will close. Thanks!

adhadse pushed a commit to adhadse/keras-nlp that referenced this pull request Sep 17, 2022
@Stealth-py @mattdangerw
Added a Learning Rate Schedule for the BERT Example (keras-team#96)
8135935 
* Added a Schedule Class

* Added/Changed decay and increase factors

* Added/Changed decay and increase factors

* Edited the Scheduler, Changes

* calculates steps_per_epoch now

* uncommented the epochs flag

* final few changes/fixes/cleaning up

* Made the necessary changes

* Forgot to add the newline in gitignore, added now

* Update .gitignore

Added newline

* resolved reviews 2.0

* ran format.sh and lint.sh

* Copy edits

Co-authored-by: Matt Watson <[email protected]>
divyashreepathihalli added a commit to divyashreepathihalli/keras-nlp that referenced this pull request Dec 5, 2024
@james77777778 @divyashreepathihalli
Add PaliGemma2 (keras-team#96)
7eb2044 
* Add PaliGemma2 arch

* Enable mixed precision check for PaliGemma

* Add conversion script

* Revert ImageConverter and reduce mem usage in the conversion script

* Remove `compute_output_spec`

* Fix `compute_output_shape` issue for keras 3.1

* Add model cards and update conversion script

* update presets

---------

Co-authored-by: divyashreepathihalli <[email protected]>
divyashreepathihalli added a commit that referenced this pull request Dec 5, 2024
@divyashreepathihalli @james77777778
Adding PaliGemma2 to KerasHub (#1998)
f251ed3 
* Add PaliGemma2 (#96)

* Add PaliGemma2 arch

* Enable mixed precision check for PaliGemma

* Add conversion script

* Revert ImageConverter and reduce mem usage in the conversion script

* Remove `compute_output_spec`

* Fix `compute_output_shape` issue for keras 3.1

* Add model cards and update conversion script

* update presets

---------

Co-authored-by: divyashreepathihalli <[email protected]>

* Update pali_gemma_presets.py - remove mix presets

* Update pali_gemma_presets.py

* Update convert_pali_gemma2_checkpoints.py

---------

Co-authored-by: james77777778 <[email protected]>
divyashreepathihalli added a commit to divyashreepathihalli/keras-nlp that referenced this pull request Dec 5, 2024
@divyashreepathihalli @james77777778
Adding PaliGemma2 to KerasHub (keras-team#1998)
80f1aa2 
* Add PaliGemma2 (keras-team#96)

* Add PaliGemma2 arch

* Enable mixed precision check for PaliGemma

* Add conversion script

* Revert ImageConverter and reduce mem usage in the conversion script

* Remove `compute_output_spec`

* Fix `compute_output_shape` issue for keras 3.1

* Add model cards and update conversion script

* update presets

---------

Co-authored-by: divyashreepathihalli <[email protected]>

* Update pali_gemma_presets.py - remove mix presets

* Update pali_gemma_presets.py

* Update convert_pali_gemma2_checkpoints.py

---------

Co-authored-by: james77777778 <[email protected]>
divyashreepathihalli added a commit that referenced this pull request Dec 5, 2024
@divyashreepathihalli @james77777778
Version bump to 0.18.0 and cherry pick (#2002)
1f11e3f 
* Adding PaliGemma2 to KerasHub (#1998)

* Add PaliGemma2 (#96)

* Add PaliGemma2 arch

* Enable mixed precision check for PaliGemma

* Add conversion script

* Revert ImageConverter and reduce mem usage in the conversion script

* Remove `compute_output_spec`

* Fix `compute_output_shape` issue for keras 3.1

* Add model cards and update conversion script

* update presets

---------

Co-authored-by: divyashreepathihalli <[email protected]>

* Update pali_gemma_presets.py - remove mix presets

* Update pali_gemma_presets.py

* Update convert_pali_gemma2_checkpoints.py

---------

Co-authored-by: james77777778 <[email protected]>

* Version bump to 0.18.0

* Update pali_gemma_presets.py (#2003)

* Update pali_gemma_presets.py

* code reformat

* Adding PaliGemma2 to KerasHub (#1998)

* Add PaliGemma2 (#96)

* Add PaliGemma2 arch

* Enable mixed precision check for PaliGemma

* Add conversion script

* Revert ImageConverter and reduce mem usage in the conversion script

* Remove `compute_output_spec`

* Fix `compute_output_shape` issue for keras 3.1

* Add model cards and update conversion script

* update presets

---------

Co-authored-by: divyashreepathihalli <[email protected]>

* Update pali_gemma_presets.py - remove mix presets

* Update pali_gemma_presets.py

* Update convert_pali_gemma2_checkpoints.py

---------

Co-authored-by: james77777778 <[email protected]>

* Update pali_gemma_presets.py (#2003)

* Update pali_gemma_presets.py

* code reformat

---------

Co-authored-by: james77777778 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattdangerw mattdangerw mattdangerw approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add a proper learning rate schedule to the BERT example
2 participants
@Stealth-py @mattdangerw