Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix] Change eval dataloader to use eval_batch_size #2847

Merged
merged 5 commits into from
Sep 10, 2024

Conversation

akashd-2
Copy link
Contributor

The evaluation dataloader is currently initialized using the train_batch_size which seems to be a bug. This PR fixes this.

akashd-2 and others added 4 commits July 18, 2024 10:41
I think eval is more similar than train here
The former is identical for single-GPU and DDP, but has a higher batch size for DP (which is the expected behaviour).
@tomaarsen
Copy link
Collaborator

Hello!

Apologies for the delay, I've been recovering from a surgery this last month.
Well spotted! I had another look at the other batch sizes as well, and I noticed that I'm using train for the get_test_dataloader currently: I think eval is better (there is no per_device_test_batch_size). This also matches what transformers does.

Additionally, I started using ..._batch_size everywhere instead of per_device_..._batch_size: The former is identical for single-GPU and DDP, but has a higher batch size for DP (which is the expected behaviour). I think after this PR, everything should be as expected again!

  • Tom Aarsen

@tomaarsen tomaarsen changed the title Change eval dataloader to use eval_batch_size [fix] Change eval dataloader to use eval_batch_size Sep 10, 2024
@tomaarsen tomaarsen merged commit 02fb5f8 into UKPLab:master Sep 10, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants