README.md

yt-napari

A napari plugin for loading data from yt.

This readme provides a brief overview including:

1. Installation
2. Quick Start
3. Contributing

Full documentation is available at yt-napari.readthedocs.io.

Installation

1. (optional) install yt and napari

If you skip this step, the installation in the following section will only install the minimal package requirements for yt or napari, in which case you will likely need to manually install some packages. So if you are new to either package, or if you are installing in a clean environment, it may be simpler to install these packages first.

For napari,

pip install napari[all]

will install napari with the default Qt backend (see here for how to choose between PyQt5 or PySide2).

For yt, you will need yt>=4.0.1 and any of the optional dependencies for your particular workflow. If you know that you'll need more than the base yt install, you can install the full suite of dependent packages with

pip install yt[full]

See the yt documentation for more information. If you're not sure which packages you'll need but don't want the full yt installation, you can proceed to the next step and then install any packages as needed (you will receive error messages when a required package is missing).

2. install yt-napari

You can install the yt-napari plugin with:

pip install yt-napari

If you are missing either yt or napari (or they need to be updated), the above installation will fetch and run a minimal installation for both.

To install the latest development version of the plugin instead, use:

pip install git+https://github.com/data-exp-lab/yt-napari.git

Note that if you are working off the development version, be sure to use the latest documentation for reference: https://yt-napari.readthedocs.io/en/latest/

Quick Start

After installation, there are three modes of using yt-napari:

1. jupyter notebook interaction (jump down)
2. loading a json file from the napari gui (jump down)
3. napari widget plugins (jump down)

jupyter notebook interaction

yt-napari provides a helper class, yt_napari.viewer.Scene that assists in properly aligning new yt selections in the napari viewer when working in a Jupyter notebook.

import napari
import yt
from yt_napari.viewer import Scene
from napari.utils import nbscreenshot

viewer = napari.Viewer()
ds = yt.load("IsolatedGalaxy/galaxy0030/galaxy0030")
yt_scene = Scene()

left_edge = ds.domain_center - ds.arr([40, 40, 40], 'kpc')
right_edge = ds.domain_center + ds.arr([40, 40, 40], 'kpc')
res = (600, 600, 600)

yt_scene.add_region(viewer,
                    ds,
                    ("enzo", "Temperature"),
                    left_edge=left_edge,
                    right_edge=right_edge,
                    resolution=res)

yt_scene.add_region(viewer,
                    ds,
                    ("enzo", "Density"),
                    left_edge=left_edge,
                    right_edge=right_edge,
                    resolution=res)

nbscreenshot(viewer)

yt_scene.add_to_viewer accepts any of the keyword arguments allowed by viewer.add_image. See the full documentation (yt-napari.readthedocs.io) for more examples, including additional helper methods for linking layer appearance.

Additionally, with yt_napari>= v0.2.0, you can use the yt_napari.timeseries module to help sample and load in selections from across datasets.

loading a selection from a yt dataset interactively

yt-napari provides two ways to sample a yt dataset and load in an image layer into a Napari viewer: the yt Reader plugin and json file specification.

using the yt Reader plugin

To use the yt Reader plugin, click on Plugins -> yt-napari: yt Reader. From there, add a region or slice selector then specify a field type and field and bounds to sample between and then simply click "Load":

You can add multiple selections and load them all at once or adjust values and click "Load" again.

using the yt Time Series Reader plugin

To use the yt Time Series Reader plugin, click on Plugins -> yt-napari: yt Time Series Reader. Specify your file matching: use file_pattern to enter glob expressions or use file_list to enter a list of specific files. Then add a slice or region to sample for each matched dataset file (note: be careful of memory here!):

using a json file and schema

yt-napari also provides the ability to load json that contain specifications for loading a file. Properly formatted files can be loaded from the napari GUI as you would load any image file (File->Open). The json file describes the selection process for a dataset as described by a json-schema. The following json file results in similar layers as the above examples:

{"$schema": "https://raw.githubusercontent.com/data-exp-lab/yt-napari/main/src/yt_napari/schemas/yt-napari_0.0.1.json",
 "datasets": [{"filename": "IsolatedGalaxy/galaxy0030/galaxy0030",
               "selections": {"regions": [{
                                "fields": [{"field_name": "Temperature", "field_type": "enzo", "take_log": true},
                                           {"field_name": "Density", "field_type": "enzo", "take_log": true}],
                                "left_edge": [460.0, 460.0, 460.0],
                                "right_edge": [560.0, 560.0, 560.0],
                                "resolution": [600, 600, 600]
                              }]},
               "edge_units": "kpc"
             }]
}

To help in filling out a json file, it is recommended that you use an editor capable of parsing a json schema and displaying hints. For example, in vscode, you will see field suggestions after specifying the yt-napari schema:

See the full documentation at yt-napari.readthedocs.io for a complete specification.

Contributing

Contributions are very welcome! Development follows a fork and pull request workflow. To get started, you'll need a development installation and a testing environment.

development environment

To start developing, fork the repository and clone your fork to get a local copy. You can then install in development mode along with all the extra requirements for developing:

pip install -e .[full,dev]

tests and style checks

Both bug fixes and new features will need to pass the existing test suite and style checks. While both will be run automatically when you submit a pull request, it is helpful to run the test suites locally and run style checks throughout development. For testing, you can use tox to test different python versions on your platform or simply run pytest and rely on the github actions to test the additional python environments.

testing with tox

first install tox with:

pip install tox

And then from the top level of the yt-napari directory, run

tox

Tox will then run a series of tests in isolated environments. In addition to checking the terminal output for test results, the tox run will generate a test coverage report: a coverage.xml file and a htmlcov folder -- to view the results, open htmlcov/index.html in a browser.

testing with pytest

If you prefer a lighter weight test, you can also use pytest directly and rely on the Github CI to test different python versions and systems. To do so, first install pytest and some related plugins:

pip install pytest pytest-qt pytest-cov

Note that if you set up your dev environment with pip install -e .[full,dev] as suggested above, you'll arelady have these dependencies.

To run the tests you can use the pytest command

pytest -v --cov=yt_napari --cov-report=html

Or the taskipy task:

task test

In addition to telling you whether or not the tests pass, the above command will write out a code coverage report to the htmlcov directory. You can open up htmlcov/index.html in a browser and check out the lines of code that were missed by existing tests.

style checks

For style checks, you can use pre-commit to run checks as you develop. To set up pre-commit:

pip install pre-commit
pre-commit install

after which, every time you run git commit, some automatic style adjustments and checks will run. The same style checks will run when you submit a pull request, but it's often easier to catch them early.

After submitting a pull request, the pre-commit.ci bot will run the style checks. If style checks fail, you can have the bot attempt to auto-fix the failures by adding the following in a comment on it's own:

pre-commit.ci autofix

The bot will then commit changes to your pull request after which you will want to run git pull locally to update your local version of the branch before making further changes to the branch.

building documentation locally

Documentation can be built using sphinx in two steps. First, update the api mapping with

sphinx-apidoc -f -o docs/source src/yt_napari/

This will update the rst files in docs/source/ with the latest docstrings in yt_napari. Next, build the html documentation with

make html

updating the pydantic models and schema

The schema versioning follows a major.minor.micro versioning pattern that matches the yt-napari versioning. Each yt-napari release should have an accompanying updated schema file, even if the contents of the schema file have not changed. On-disk schema are stored in src/yt_napari/schemas/, with copies in the documentation at docs/_static.

There are a number of utilities to help automate the management of schema in repo_utilities/. The easiest way to use these utitities is with taskipy from the command line. To list available scripts:

task --list

Before a release, run

task validate_release vX.X.X

where vX.X.X is the version of the upcoming release. This script will run through some checks that ensure:

• the on-disk schema matches the schema generated by the pydantic model
• the schema files in the documentation match the schema files in the package

If any of the checks fail, you will be advised to update the schema using update_schema_docs. If you run without providing a version:

task update_schema_docs

It will simply copy over the existing on-disk schema files to the documentation. If you run with a version:

task update_schema_docs -v vX.X.X

It will write a schema file for the current pydantic model, overwriting any on-disk schema files for the provided version.

updating the sample data

The sample data utilizes another helper script: repo_utilities/update_sample_data.py that you can invoke with taskipy as:

task update_sample_data

To adjust which sample datasets are included, go edit the enabled list in repo_utilities/update_sample_data.py. The names in enabled must match those accepted by yt.load_sample. In addition to enabling a dataset, you may need to adjust the field settings for the sample dataset that you are adding: see the sample_field and log_field dictionaries.

When you run update_sample_data, a number of things happen:

1. The napari plugin manifest is updated. For every dataset in the enabled list, yt_napari/napari.yaml will include 2 entries: a new entry in commands and a new entry in sample_data.
2. For every dataset in the enabled list, a json file will be generated in yt_napari/sample_data/ along with a single yt_napari/sample_data/sample_registry.json. These json files are used for actually loading the sample data.
3. yt_napari/sample_data/_sample_data.py will be rewritten and for every dataset in the enabled list, there will be a corresponding function. The function name maps to the python name in yt_napari/napari.yaml (the plugin manifest file). If yt_napari/sample_data/_sample_data.py is incorrect then the code generation in repo_utilities/update_sample_data.py should be updated, do not edit yt_napari/sample_data/_sample_data.py directly.

License

Distributed under the terms of the BSD-3 license, "yt-napari" is free and open source software

Issues

If you encounter any problems, please file an issue along with a detailed description.

Funding

The yt-napari plugin project was funded with support from the Chan Zuckerberg Initiative through the napari Plugin Accelerator Grants project, Enabling Access To Multi-resolution Data.

---

This napari plugin was generated with Cookiecutter using @napari's cookiecutter-napari-plugin template.