Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Critical Error While Loading Data #353

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

AlonzoLeeeooo
Copy link

Hi @huxiuhan ,
I've found some critical issues of this code when debugging it, where I propose this PR to fix it. Details are below:

Previous version of the __getitem__ in animatediff/data/dataset.py, line 67-73:

 while True:
            try:
                pixel_values, name = self.get_batch(idx)
                break

            except Exception as e:
                idx = random.randint(0, self.length-1)
pixel_values = self.pixel_transforms(pixel_values)

Fixed version:

pixel_values, name = self.get_batch(idx)
pixel_values = self.pixel_transforms(pixel_values)

The previous version of dataset.py might cause dead loop if the WebVid10M dataset is not downloaded completely. Specifically, if self.get_batch(idx) does not function well, the data loading process will keep excecuting idx = random.randint(0, self.length-1), untill the idx data is available.

This might cause much inconvenicence for two reasons: (1) If one is attempting to debug this code with a tiny subset of WebVid, he might not need the entire dataset to do so; (2) The dead loop prevents us to see where the code issue truly lies.

I would really appreciate it if you can accept this PR, since the issue has been addressed in a series of issues, particularly in #201, #352, #331. Thank you so much in advance for your precious time and attention from your busy schedule.

Best regards,
Chang

Fix critical error in `dataset.py`, which might cause dead loop while running the code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant