Fix Critical Error While Loading Data #353
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi @huxiuhan ,
I've found some critical issues of this code when debugging it, where I propose this PR to fix it. Details are below:
Previous version of the
__getitem__
inanimatediff/data/dataset.py
,line 67-73
:Fixed version:
The previous version of
dataset.py
might cause dead loop if theWebVid10M
dataset is not downloaded completely. Specifically, ifself.get_batch(idx)
does not function well, the data loading process will keep excecutingidx = random.randint(0, self.length-1)
, untill theidx
data is available.This might cause much inconvenicence for two reasons: (1) If one is attempting to debug this code with a tiny subset of
WebVid
, he might not need the entire dataset to do so; (2) The dead loop prevents us to see where the code issue truly lies.I would really appreciate it if you can accept this PR, since the issue has been addressed in a series of issues, particularly in #201, #352, #331. Thank you so much in advance for your precious time and attention from your busy schedule.
Best regards,
Chang