Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/strided dataset for torch and regression models #2624

Open
wants to merge 25 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
aaaa234
feat: adding stride to shifted_dataset
madtoinou Dec 18, 2024
dec1636
feat: adding stride to the sequential dataset
madtoinou Dec 18, 2024
56b53ed
feat: add striding to tabularization
madtoinou Dec 18, 2024
9847469
feat: add stride to RegressioModel fit API
madtoinou Dec 18, 2024
d055824
fix: bug in striding implementation
madtoinou Dec 18, 2024
67fd550
feat: updating the tabularization tests
madtoinou Dec 19, 2024
d615552
fix: bug
madtoinou Dec 19, 2024
1162c2c
Merge branch 'master' into feat/strided_dataset
madtoinou Dec 19, 2024
9ff31a4
feat: adding test for stride in torch datasets
madtoinou Dec 19, 2024
69fbc47
update changelog
madtoinou Dec 19, 2024
911b8c5
fix: missing test and small bug
madtoinou Dec 20, 2024
7c3e5a2
Merge branch 'master' into feat/strided_dataset
madtoinou Dec 30, 2024
e8ac91d
update changelog
madtoinou Dec 30, 2024
c375832
Merge branch 'master' into feat/strided_dataset
madtoinou Dec 31, 2024
198edd5
doc: update some docstrings
madtoinou Jan 3, 2025
a38b9f5
feat: stride is now applied from the end of the series
madtoinou Jan 9, 2025
c546991
feat: added stride support to the horizon based dataset
madtoinou Jan 9, 2025
dd8ab50
feat: updated unit tests
madtoinou Jan 9, 2025
2fc77e0
feat: adding support of stride in the fit method of torch models
madtoinou Jan 9, 2025
1a7e42d
fix: update the test so that the stride is applied from the end
madtoinou Jan 9, 2025
ad6bd31
Merge branch 'master' into feat/strided_dataset
madtoinou Jan 9, 2025
f168f46
fix: add missing defualt value
madtoinou Jan 9, 2025
69fc81b
Merge branch 'feat/strided_dataset' of https://github.com/unit8co/dar…
madtoinou Jan 16, 2025
dd07ed6
Merge branch 'master' into feat/strided_dataset
madtoinou Jan 25, 2025
eb8b502
Merge branch 'feat/strided_dataset' of https://github.com/unit8co/dar…
madtoinou Feb 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: adding test for stride in torch datasets
madtoinou committed Dec 19, 2024
commit 9ff31a475c045e806930f41afd48a7cd9a18c5a9
48 changes: 48 additions & 0 deletions darts/tests/datasets/test_datasets.py
Original file line number Diff line number Diff line change
@@ -1706,6 +1706,54 @@ def test_sequential_training_dataset_invalid_weight(self, ds_cls):
"2000-01-02 00:00:00 - 2000-01-04 00:00:00."
)

@pytest.mark.parametrize(
"config",
[
# (dataset class, whether contains future, future batch index)
madtoinou marked this conversation as resolved.
Show resolved Hide resolved
(PastCovariatesSequentialDataset, None),
(FutureCovariatesSequentialDataset, 1),
(DualCovariatesSequentialDataset, 2),
(MixedCovariatesSequentialDataset, 3),
(SplitCovariatesSequentialDataset, 2),
],
)
def test_sequential_training_dataset_stride(self, config):
madtoinou marked this conversation as resolved.
Show resolved Hide resolved
ds_cls, future_idx = config
icl = 4
ocl = 2
nb_samples = 12
target = self.target1[: icl + ocl + nb_samples - 1]

ds_covs = {}
ds_init_params = set(inspect.signature(ds_cls.__init__).parameters)
for cov_type in ["covariates", "past_covariates", "future_covariates"]:
if cov_type in ds_init_params:
ds_covs[cov_type] = self.cov1

ds_reg = ds_cls(
target_series=target,
input_chunk_length=icl,
output_chunk_length=ocl,
stride=1,
**ds_covs,
)

ds_stride = ds_cls(
target_series=target,
input_chunk_length=icl,
output_chunk_length=ocl,
stride=3,
**ds_covs,
)
assert len(ds_stride) * 3 == len(ds_reg) == nb_samples
# regular dataset with stride=1, every 3rd values should be identical to the dataset with stride=3
for idx, batch_str in enumerate(ds_stride):
for entry_s, entry_r in zip(batch_str, ds_reg[idx * 3]):
if entry_s is not None and entry_r is not None:
np.testing.assert_almost_equal(entry_s, entry_r)
else:
assert entry_s == entry_r

def test_get_matching_index(self):
from darts.utils.data.utils import _get_matching_index