diff --git a/docs/index.rst b/docs/index.rst index 326178cfe..d3eac653d 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -135,6 +135,7 @@ tedana is licensed under GNU Lesser General Public License version 2.1. approach outputs contributing + roadmap api Indices and tables diff --git a/docs/roadmap.rst b/docs/roadmap.rst new file mode 100644 index 000000000..82f9466e6 --- /dev/null +++ b/docs/roadmap.rst @@ -0,0 +1,197 @@ +The ``tedana`` Roadmap +====================== + +Project vision +-------------- + +ME-EPI processing is not well integrated into major preprocessing packages, +yielding duplicated and unmaintained code. +```tedana`` has been developed to address this need and will serve as a central repository +for standard ME-EPI denoising as well as a testing ground for novel ME-EPI denoising methods. +This will jointly reduce the external burden on pipeline maintainers, +facilitate increased ME-EPI adoption, and enable future development in ME-EPI denoising. + +Metrics of success and corresponding milestones +----------------------------------------------- + +We will know that we have been successful in creating ``tedana`` when we have succeeded in providing +several concrete deliverables, which can be broadly categorized into: + +1. *:ref:`Documentation`*, +2. *:ref:`Transparent and Reproducible Processing`*, +3. *:ref:`Testing`*, +4. *:ref:`Workflow Integrations`*, +5. *ref:`Extensions and Improvements to ME-EPI processing`*, and +6. *:ref:`Developing a healthy community`* + +Each deliverable has been synthesized into a milestone that gives the ``tedana`` community a link +between the issues and the high level vision for the project. + +.. _Documentation: + +Documentation +````````````` +**Summary**: +One long-standing concern with ME-EPI denoising has been the availability of +documentation for the method outside of published scientific papers. +To address this, we have created `a ReadTheDocs site`_; +however, there are still several sections either explicitly marked as "# TODO" +or otherwise missing crucial information. + +We are committed to providing helpful documentation for all users of ``tedana``. +One metric of success, then, is to develop documentation that includes: + +1. Motivations for conducting echo time dependent analysis, +2. A collection of key ME-EPI references from the published literature, +3. Tutorials on how to use ``tedana``, +4. The different processing steps that are conducted in each workflow, +5. An up-to-date description of the API, +6. A transparent explanation of the different decisions that are made through the ```tedana`` pipeline, and +7. Where to seek support + +.. _a ReadTheDocs site: https://tedana.readthedocs.io + +**Associated Milestone**: +This milestone will close when the online documentation contains the minimum necessary information +to orient a complete newcomer to ME-EPI, both on the theoretical basis of the method as well as +the practical steps used in ME-EPI denoising. + +.. _Transparent and Reproducible Processing: + +Transparent and reproducible processing +``````````````````````````````````````` +**Summary**: +Alongside the lack of existing documentation, +there is a general unfamiliarity with how selection criteria are applied to individual data sets. +This lack of transparency, combined with the non-deterministic nature of the decomposition, +has generated significant uncertainty when interpreting results. + +In order to build and maintain confidence in ME-EPI processing, +any analysis software---including ``tedana``---must provide enough information such that +the user is empowered to conduct transparent and reproducible analyses. +This will permit clear reporting of the ME-EPI results in published studies +and facilitate a broader conversation in the scientific community on the nature of ME-EPI processing. + +We are therefore committed to making ``tedana`` analysis transparent and reproducible +such that we report back all processing steps applied to any individual data set, +including the specific selection criteria used in making denoising decisions. +This, combined with the reproducibility afforded by seeding all non-deterministic steps, +will enable both increased confidence and better reporting of ME-EPI results. + +A metric of success for ``tedana`` then, should be enhancements to the code such that +1. Non-deterministic steps are made reproducible by enabling access to a "seed value", and +2. The decision process for individual component data is made accessible to the end user. + +**Associated Milestone**: +This milestone will close when when the internal decision making process for +component selection is made accessible to the end user, +and an analysis can be reproduced by an independent researcher who has access to the same data. + +.. _Testing: + +Testing +``````` +**Summary**: +Historically, the lack of testing for ME-EPI analysis pipelines has prevented new +developers from engaging with the code for fear of silently breaking or otherwise degrading +the existing implementation. +Moving forward, we want to grow an active development community, +where developers feel empowered to explore new enhancements to the ``tedana`` code base. + +One means to ensure that new code does not introduce silent failures is through extensive testing. +We are therefore committed to implementing high test coverage at both +the unit test and integration test levels; +that is, both in testing individual functions and broader workflows, respectively. + +A metric of success should thus be: +1. Achieving 90% test coverage for unit tests, as well as +2. Three distinguishable integration tests over a range of possible acquisition conditions. + +**Milestone**: +This milestone will close when we have 90% test coverage for unit tests and +three distinguishable integration tests, +varying number of echos and acquisition type (i.e., task vs. rest). + +.. _Workflow Integrations: + +Workflow integration: AFNI +`````````````````````````` +**Summary**: +Currently, `afni_proc.py`_ distributes an older version of ``tedana``, +around which they have built a wrapper script, `tedana_wrapper.py`_, to ensure compatibility. +AFNI users at this point are therefore not accessing the latest version of ``tedana``. +We will grow our user base if ``tedana`` can be accessed through AFNI, +and we are therefore committed to supporting native integration of ``tedana`` in AFNI. + +.. _afni_proc.py: https://afni.nimh.nih.gov/pub/dist/doc/program_help/afni_proc.py.html +.. _tedana_wrapper.py: https://github.com/afni/afni/blob/a3288abefb66bc7c76e98fdf13425ab48651bf36/src/python_scripts/afni_python/tedana_wrapper.py + +One metric of success, therefore, will be if we can demonstrate sufficient stability and support +such that the ``afni_proc.py`` maintainers are willing to switch to ``tedana`` as the recommended +method of accessing ME-EPI denoising in AFNI. +We will aim to aid in this process by increasing compatibility between ``tedana`` +and the ```afni_proc.py`` workflow, eliminating the need for an additional wrapper script. +For example, ``tedana`` could directly accept BRIK/HEAD files, +facilitating interoperability with other AFNI pipelines. + +**Milestone**: + This milestone will close when ``tedana`` is stable enough such that the recommended default in + ``afni_proc.py`` is to access ME-EPI denoising via ``pip install tedana``, + rather than maintaining the alternative version that is currently used. + + +Workflow integration: BIDS +`````````````````````````` +**Summary**: +Currently, the BIDS ecosystem has limited support for ME-EPI processing. +We will grow our user base if ``tedana`` is integrated into existing BIDS Apps and +therefore accessible to members of the BIDS community. +One promising opportunity is if ``tedana`` can be used natively in `FMRIPrep`_. +Some of the work is not required at this repository, but other changes will need to happen here; +for example, making sure the outputs are BIDS compliant. + +A metric of success, then, will be +1. Fully integrating ``tedana`` into ``FMRIPrep``, and +2. Making ``tedana`` outputs compliant with the `BIDS derivatives specification`_. + +.. _FMRIPrep: https://github.com/poldracklab/fmriprep +.. _BIDS derivatives specification: https://docs.google.com/document/d/1Wwc4A6Mow4ZPPszDIWfCUCRNstn7d_zzaWPcfcHmgI4/edit + +**Milestone**: +This milestone will close when the denoising steps of ``tedana`` are stable enough +to integrate into ``FMRIPrep`` and the ```FMRIPrep`` project is updated to process ME-EPI scans. + +.. _Extensions and Improvements to ME-EPI processing: + +Method extensions & improvements +```````````````````````````````` +**Summary**: +Overall, each of the listed deliverables will support a broader goal: +to improve on ME-EPI processing itself. +This is an important research question and will advance the state-of-the-art in ME-EPI processing. + +A metric of success here would be +* *EITHER* integrating a new decomposition method, beyond ICA +* *OR* validating new selection criteria. + +**Milestone**: +This milestone will close when the codebase is stable enough to integrate novel methods +into ``tedana``, and that happens! + +.. _Developing a healthy community: + +Developing a healthy community +`````````````````````````````` +**Summary**: +In developing ``tedana``, we are committed to fostering a healthy community. +A healthy community is one in which the maintainers are happy and not overworked, +and which empowers users to contribute back to the project. +By making ``tedana`` stable and well-documented, with enough modularity to integrate improvements, +we will enable new contributors to feel that their work is welcomed. + +We therefore have one additional metric of success: +1. An outside contributor integrates an improvement to ME-EPI denoising. + +**Milestone**: +This milestone will probably never close, +but will serve to track issues related to building and supporting the ``tedana`` community.