Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add drivers section to Data.toml for autoloading #20

Merged
merged 4 commits into from
Jun 23, 2021
Merged

Conversation

c42f
Copy link
Contributor

@c42f c42f commented Jun 14, 2021

This allows modules which provide data storage drivers to be automatically loaded when DataSets itself is loaded, providing a declarative data environment workflow while bypassing world age issues we'd get from loading these drivers on demand.

This is achieved with a [[drivers]] section in the Data.toml. The example from the tests:

data_config_version = 0

[[datasets]]
description="Test dynamic loading of drivers"
name="dummy_storage_blob"
uuid="785b3cdc-428e-426f-a3f7-3f6ae88a9637"

    [datasets.storage]
    driver="DummyTomlStorage"  # Usage of the driver
    type="Blob"
    data="data_from_dummy_backend"

# Declaration of the driver
[[drivers]]
type="storage"
name="DummyTomlStorage"

    [drivers.module]
    name="DummyStorageBackends"
    uuid="89b7a33a-382e-4698-a931-421b088d35a2"

c42f added 3 commits June 15, 2021 22:29
This allows modules which provide data storage drivers to be
automatically loaded when DataSets itself is loaded, providing a
declarative data environment workflow while bypassing world age issues.
@c42f c42f force-pushed the cjf/driver-autoload branch from 9378aeb to 99b0c4e Compare June 15, 2021 12:30
…torage drivers.

Hopefully this is better behavior when the user's Manifest is
inconsistent with their dataset driver configuration.
@c42f c42f merged commit d918afa into master Jun 23, 2021
@delete-merged-branch delete-merged-branch bot deleted the cjf/driver-autoload branch June 23, 2021 06:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant