Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Zarr3 default and deprecations #1242

Open
wants to merge 75 commits into
base: master
Choose a base branch
from
Open

Make Zarr3 default and deprecations #1242

wants to merge 75 commits into from

Conversation

normanrz
Copy link
Member

@normanrz normanrz commented Jan 27, 2025

Description:

  • Makes Zarr3 the default data format
  • Removes deprecated functions, properties, arguments
  • Changes the writing behavior to include a allow_resize and allow_unaligned argument to make it more robust against accidental concurrency issues
  • MagViews are compressed by default

Issues:

Todos:

  • List all deprecations in the changelog
  • Add tests for new allow_resize behavior
  • Add allow_unaligned arg
  • Fix from_images for unexpected bboxes

Make sure to delete unnecessary points or to check all before merging:

  • Updated Changelog
  • Updated Documentation
  • Added / Updated Tests
  • Updated examples

@normanrz normanrz marked this pull request as draft January 27, 2025 21:40
@@ -264,7 +264,7 @@ def download_and_unpack(

BIOFORMATS_ARGS: list[tuple[str, str, dict, str, int, tuple[int, int, int], int]] = [
(
"https://samples.scif.io/wtembryo.zip",
"https://static.webknossos.org/data/wklibs-samples/wtembryo.zip",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using mirrors of the testdata to avoid dependence on external servers.

@normanrz normanrz marked this pull request as ready for review February 6, 2025 14:17
@normanrz normanrz requested a review from markbader February 6, 2025 14:18
Copy link
Member

@philippotto philippotto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great stuff 👍 I especially like the removal of all the deprecations.

However, I think I'm not yet fully understanding when allow_resize can be specified and when not. see my one comment for that.

.github/workflows/ci.yml Outdated Show resolved Hide resolved
cluster_tools/tests/test_all.py Show resolved Hide resolved
webknossos/Changelog.md Outdated Show resolved Hide resolved
webknossos/Changelog.md Show resolved Hide resolved
webknossos/test.sh Outdated Show resolved Hide resolved
@@ -299,9 +241,8 @@ def _get_mag1_bbox(
def write(
self,
data: np.ndarray,
offset: Optional[Vec3IntLike] = None, # deprecated, relative, in current mag
json_update_allowed: bool = True,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why doesn't the View.write method have a allow_resize parameter (only MagView.write has it currently)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment above. A View doesn't have a reference to its Layer, so it cannot resize the Layer.

webknossos/webknossos/dataset/view.py Outdated Show resolved Hide resolved
webknossos/webknossos/dataset/view.py Show resolved Hide resolved
webknossos/webknossos/dataset/view.py Outdated Show resolved Hide resolved
webknossos/webknossos/dataset/view.py Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Harden concurrency assertions for Zarr writing
2 participants