Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Make the torch dataloader support TensorDict #66

Open
vmoens opened this issue Nov 25, 2022 · 0 comments
Open

[Feature Request] Make the torch dataloader support TensorDict #66

vmoens opened this issue Nov 25, 2022 · 0 comments
Assignees
Labels
enhancement New feature or request question Further information is requested

Comments

@vmoens
Copy link

vmoens commented Nov 25, 2022

PyTorch has recently released tensordict, which provides a series of utilities for efficiently process batches of heterogeneous tensor types.
It makes it easy to index multiple tensors at the same time, manipulate their shape, split them etc. It also allows to work efficiently with nested data structures. Have a look at the README, the doc and tutorial.

For context, ysing TensorDict with replay buffers in TorchRL provided us with at least an order of magnitude speed-up during data collection.

Would it be possible to consider a version of the dataloader that would output a TensorDict, instead of a dictionary as in the tutorial?

@vmoens vmoens changed the title [Feature Request] Make torch dataloader support TensorDict [Feature Request] Make the torch dataloader support TensorDict Nov 25, 2022
@rnyak rnyak added this to the Merlin 23.02 milestone Jan 18, 2023
@rnyak rnyak added enhancement New feature or request question Further information is requested labels Jan 18, 2023
@karlhigley karlhigley removed this from the Merlin 23.02 milestone Apr 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants