From d64efff5823b5a3fc16344f0ca664c5e1ce9a5c1 Mon Sep 17 00:00:00 2001 From: paugier Date: Fri, 5 Jan 2024 12:13:21 +0100 Subject: [PATCH] Doc: convert all doc/*.rst files to .md --- doc/faq.md | 42 ++++-- doc/install.md | 274 +++++++++++++++++++++++++++++++++++++ doc/install.rst | 246 --------------------------------- doc/test_bench_profile.md | 34 +++++ doc/test_bench_profile.rst | 39 ------ doc/to_do.md | 19 +++ doc/to_do.rst | 24 ---- doc/tutorials.md | 23 ++++ doc/tutorials.rst | 21 --- 9 files changed, 377 insertions(+), 345 deletions(-) create mode 100644 doc/install.md delete mode 100644 doc/install.rst create mode 100644 doc/test_bench_profile.md delete mode 100644 doc/test_bench_profile.rst create mode 100644 doc/to_do.md delete mode 100644 doc/to_do.rst create mode 100644 doc/tutorials.md delete mode 100644 doc/tutorials.rst diff --git a/doc/faq.md b/doc/faq.md index 390f4cb3d..7dedd4e37 100644 --- a/doc/faq.md +++ b/doc/faq.md @@ -2,44 +2,56 @@ ## Applications: Can I use FluidSim for -:::\{admonition} Wall bounded / multiphase / reactive flows ...? Maybe. The -built-in {mod}`solvers ` excels solving within periodic domains -with pseudospectral solvers. However, FluidSim is a framework, and this allows -FluidSim to interface with third-party solvers. See for instance -{mod}`fluidsim.base.basilisk`, {mod}`fluidsim.base.dedalus` and -[snek5000](https://snek5000.readthedocs.io). ::: +:::{admonition} Wall bounded / multiphase / reactive flows ...? + +Maybe. The built-in {mod}`solvers ` excels solving within +periodic domains with pseudospectral solvers. However, FluidSim is a framework, +and this allows FluidSim to interface with third-party solvers. See for +instance {mod}`fluidsim.base.basilisk`, {mod}`fluidsim.base.dedalus` and +[snek5000](https://snek5000.readthedocs.io). + +::: ## Troubleshooting installation issues Make sure you read the [installation guide](install) carefully. -:::\{admonition} Permission denied while running `pip install fluidsim` from PyPI +:::{admonition} Permission denied while running `pip install fluidsim` from PyPI + or `make develop` inside the repository. This means you are probably using the Python bundled with the system and as a user you are restricted from installing -packages into it. If this is so, create a [virtual environment]. ::: +packages into it. If this is so, create a [virtual environment]. + +::: + +:::{admonition} *No module named pip* or `distutils.errors.DistutilsError` -:::\{admonition} *No module named pip* or `distutils.errors.DistutilsError` Package manager `pip` was not installed into your environment or is too old. The following commands should help: -``` +```sh python -m ensurepip python -m pip install --upgrade pip setuptools wheel ``` ::: -:::\{admonition} System freezes or becomes unresponsive as fluidsim starts to -build extensions By default `pythran` extensions try to use `gcc` and this is a +:::{admonition} System freezes or becomes unresponsive as fluidsim starts to build extensions + +By default `pythran` extensions try to use `gcc` and this is a CPU and memory intensive compilation. Instead `pythran` can be configured to use `clang`. See {ref}`pythranrc` for more details. Additionally to reduce the load during installation is to configure certain -{ref}`build specific environment variables `. ::: +{ref}`build specific environment variables `. +::: + +:::{admonition} `ModuleNotFoundError: No module named 'fluidsim_core. ...` -:::\{admonition} `ModuleNotFoundError: No module named 'fluidsim_core. ...` FluidSim depends on {mod}`fluidsim_core` and both follow the same versioning. Make sure the versions match if you had used `pip install` or `conda install`. For -developers, `make develop` should install both as editable installations. ::: +developers, `make develop` should install both as editable installations. + +::: [virtual environment]: https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment diff --git a/doc/install.md b/doc/install.md new file mode 100644 index 000000000..6b5df0327 --- /dev/null +++ b/doc/install.md @@ -0,0 +1,274 @@ +# Installation + +FluidSim is part of the FluidDyn project and requires Python >= 3.6. Some issues +regarding the installation of Python and Python packages are discussed in +[the main documentation of the project](http://fluiddyn.readthedocs.org/en/latest/install.html). + +We don't upload wheels on PyPI, so the simplest and fastest procedure is to use +`conda` (no compilation needed). Alternatively, one can compile fluidsim and +fluidfft using `pip`. + +## Installing the conda-forge packages with conda or mamba + +The fluidsim packages are in the conda-forge channels, so if it is not already +done, one needs to add it: + +```sh +conda config --add channels conda-forge +``` + +If you just want to run sequential simulations and/or analyze the results of +simulations, you can just install the fluidsim package: + +```sh +conda install fluidsim +``` + +For parallel simulations using MPI, let's create a dedicated environment: + +```sh +conda create -n env_fluidsim ipython fluidsim "fluidfft[build=mpi*]" "h5py[build=mpi*]" +``` + +The environment can then be activated with `conda activate env_fluidsim`. + +## Build/install using pip + +We recommend installing with a recent version of pip so you might want to run +`pip install pip -U` before anything (if you use conda, `conda update pip`). Then, +fluidsim can be installed with: + +```sh +pip install fluidsim +``` + +However, one have to note that (i) fluidsim builds are sensible to environment +variables (see below) and (ii) fluidsim can optionally use +[fluidfft](http://fluidfft.readthedocs.io) for pseudospectral solvers. Fluidsim +and fluidfft can be both installed with the command: + +```sh +pip install fluidsim[fft] +``` + +Moreover, fluidfft builds can also be tweaked so you could have a look at +[fluidfft documentation](http://fluidfft.readthedocs.io/en/latest/install.html). + +(env-vars)= + +## Environment variables and build configuration + +### Build time configuration + +- `FLUIDDYN_NUM_PROCS_BUILD` + +Fluidsim binaries are builds in parallel. This speedups the build process a lot on +most computers. However, it can be a very bad idea on computers with not enough +memory. If you encounter problems, you can force the number of processes used +during the build using the environment variable `FLUIDDYN_NUM_PROCS_BUILD`: + +```sh +export FLUIDDYN_NUM_PROCS_BUILD=2 +``` + +- Customize compilers to build extensions, if the defaults do not work for you, + either using the environment variables: + + - `MPICXX`: for Cython extensions in `fluidfft` (default: `mpicxx`) + - `CC`: command to generate object files in `fluidsim` + - `LDSHARED`: command to link and generate shared libraries in `fluidsim` + - `CARCH`: to cross compile (default: `native`) + +- `DISABLE_PYTHRAN` disables compilation with Pythran at build time. + +- `FLUIDSIM_TRANSONIC_BACKEND` + + "pythran" by default, it can be set to "python", "numba" or "cython". + +You can also configure the installation of fluidsim by creating the file +`~/.fluidsim-site.cfg` (or `site.cfg` in the repo directory) and modify it to fit +your requirements before the installation with pip: + +```sh +wget https://foss.heptapod.net/fluiddyn/fluidsim/-/raw/branch/default/site.cfg.default -O ~/.fluidsim-site.cfg +``` + +### Runtime configuration + +Fluidsim is also sensitive to the environment variables: + +- `FLUIDSIM_PATH`: path where the simulation results are saved. + + In Unix systems, you can for example put this line in your `~/.bashrc`: + + ```sh + export FLUIDSIM_PATH=$HOME/Data + ``` + +- `FLUIDDYN_PATH_SCRATCH`: working directory (can be useful on some clusters). + +## Warning about re-installing fluidsim and fluidfft with new build options + +If fluidsim has already been installed and you want to recompile with new +configuration values, you need to really recompile fluidsim and not just reinstall +an already produced wheel. To do this, use: + +```sh +pip install fluidsim --no-binary fluidsim -v +``` + +`-v` toggles the verbose mode of pip so that we see the compilation log and can +check that everything goes well. + +(pythranrc)= + +## About using Pythran to compile functions + +We choose to use the Python compiler +[Pythran](https://github.com/serge-sans-paille/pythran) for some functions of the +operators. Our microbenchmarks show that the performances are as good as what we +are able to get with Fortran or C++! + +But it implies that building Fluidsim requires a C++11 compiler (for example GCC +4.9 or clang). + +:::{note} + +If you don't want to use Pythran and C++ to speedup Fluidsim, you can +use the environment variable `DISABLE_PYTHRAN`. + +::: + +:::{warning} + +To reach good performance, we advice to try to put in the file +`~/.pythranrc` the lines (it seems to work well on Linux, see the +[Pythran documentation](https://pythran.readthedocs.io)): + +```bash +[pythran] +complex_hook = True +``` + +::: + +:::{warning} + +The compilation of C++ files produced by Pythran can be long and can +consume a lot of memory. If you encounter any problems, you can try to use clang +(for example with `conda install clangdev`) and to enable its use in the file +`~/.pythranrc` with: + +```bash +[compiler] +CXX=clang++ +CC=clang +``` + +::: + +## MPI simulations and mpi4py! + +Fluidsim can use [mpi4py](http://mpi4py.scipy.org) (which depends on a MPI +implementation) for MPI simulations. + +:::{warning} + +If the system has multiple MPI libraries, it is adviced to +explicitly mention the MPI command. For instance to use Intel MPI: + +```sh +CC=mpiicc pip install mpi4py --no-binary mpi4py +``` + +::: + +## About h5py and HDF5_MPI + +FluidSim is able to use h5py built with MPI support. + +:::{warning} + +Prebuilt installations (for e.g. via h5py wheels) lacks MPI support. +Most of the time, this is what you want. However, you can install h5py from source +and link it to a hdf5 built with MPI support, as follows: + +```bash +CC="mpicc" HDF5_MPI="ON" HDF5_DIR=/path/to/parallel-hdf5 pip install --no-deps --no-binary=h5py h5py +python -c 'import h5py; h5py.run_tests()' +``` + +In some cases you need to set C_INCLUDE_PATH variable before h5py installation. +For example on Debian stretch: + +```bash +export C_INCLUDE_PATH=/usr/include/openmpi/ +CC="mpicc" HDF5_MPI="ON" HDF5_DIR=/path/to/parallel-hdf5 pip install --no-deps --no-binary=h5py h5py +``` + +See the [h5py documentation](http://docs.h5py.org/en/latest/build.html) for more +details. + +::: + +## Installing from the repository + +:::{note} + +A good base to install Fluidsim from source can be to create and +activate a conda environment with: + +```sh +conda create -y -n env-fluidsim -c conda-forge "fluidfft=*=*openmpi*" pythran clangdev mako +conda activate env-fluidsim +``` + +::: + +For fluidsim, we use the revision control software Mercurial and the main +repository is hosted [here](https://foss.heptapod.net/fluiddyn/fluidsim) in +Heptapod. Download the source with something like: + +```sh +hg clone https://foss.heptapod.net/fluiddyn/fluidsim +``` + +If you are new with Mercurial and Heptapod, you can also read +[this short tutorial](http://fluiddyn.readthedocs.org/en/latest/mercurial_heptapod.html). + +For particular installation setup, copy the default configuration file to +`site.cfg`: + +```sh +cp site.cfg.default site.cfg +``` + +and modify it to fit your requirements. + +Build/install in development mode, by running from the top-level directory: + +```sh +cd lib && pip install -e .; cd .. +pip install -e . +``` + +:::{note} + +To install from source in a conda environment, it is actually necessary +to disable the isolated build by running the command +`pip install -e . --no-build-isolation`. + +::: + +To install Fluidsim with all optional dependencies and all capacities: + +```sh +pip install -e .[full] +``` + +### Run the tests + +You can run some unit tests by running `make tests` (shortcut for +`fluidsim-test -v`) or `make tests_mpi` (shortcut for +`mpirun -np 2 fluidsim-test -v`). Alternatively, you can also run `pytest` from +the root directory or from any of the source directories. diff --git a/doc/install.rst b/doc/install.rst deleted file mode 100644 index 16ea95069..000000000 --- a/doc/install.rst +++ /dev/null @@ -1,246 +0,0 @@ -Installation -============ - -FluidSim is part of the FluidDyn project and requires Python >= 3.6. Some -issues regarding the installation of Python and Python packages are discussed -in `the main documentation of the project -`_. - -We don't upload wheels on PyPI, so the simplest and fastest procedure is to -use ``conda`` (no compilation needed). Alternatively, one can compile fluidsim -and fluidfft using ``pip``. - -Installing the conda-forge packages with conda or mamba -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The fluidsim packages are in the conda-forge channels, so if it is not already -done, one needs to add it:: - - conda config --add channels conda-forge - -If you just want to run sequential simulations and/or analyze the results of -simulations, you can just install the fluidsim package:: - - conda install fluidsim - -For parallel simulations using MPI, let's create a dedicated environment:: - - conda create -n env_fluidsim ipython fluidsim "fluidfft[build=mpi*]" "h5py[build=mpi*]" - -The environment can then be activated with ``conda activate env_fluidsim``. - -Build/install using pip -~~~~~~~~~~~~~~~~~~~~~~~ - -We recommend installing with a recent version of pip so you might want to -run ``pip install pip -U`` before anything (if you use conda, ``conda update -pip``). Then, fluidsim can be installed with:: - - pip install fluidsim - -However, one have to note that (i) fluidsim builds are sensible to environment -variables (see below) and (ii) fluidsim can optionally use -`fluidfft `_ for pseudospectral solvers. -Fluidsim and fluidfft can be both installed with the command:: - - pip install fluidsim[fft] - -Moreover, fluidfft builds can also be tweaked so you could have a look at -`fluidfft documentation -`_. - -.. _env_vars: - -Environment variables and build configuration -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Build time configuration -........................ - -- ``FLUIDDYN_NUM_PROCS_BUILD`` - -Fluidsim binaries are builds in parallel. This speedups the build process a lot -on most computers. However, it can be a very bad idea on computers with not -enough memory. If you encounter problems, you can force the number of processes -used during the build using the environment variable -``FLUIDDYN_NUM_PROCS_BUILD``:: - - export FLUIDDYN_NUM_PROCS_BUILD=2 - -- Customize compilers to build extensions, if the defaults do not work for you, - either using the environment variables: - - - ``MPICXX``: for Cython extensions in ``fluidfft`` (default: ``mpicxx``) - - ``CC``: command to generate object files in ``fluidsim`` - - ``LDSHARED``: command to link and generate shared libraries in ``fluidsim`` - - ``CARCH``: to cross compile (default: ``native``) - -- ``DISABLE_PYTHRAN`` disables compilation with Pythran at build time. - -- ``FLUIDSIM_TRANSONIC_BACKEND`` - - "pythran" by default, it can be set to "python", "numba" or "cython". - -You can also configure the installation of fluidsim by creating the file -``~/.fluidsim-site.cfg`` (or ``site.cfg`` in the repo directory) and modify it -to fit your requirements before the installation with pip:: - - wget https://foss.heptapod.net/fluiddyn/fluidsim/-/raw/branch/default/site.cfg.default -O ~/.fluidsim-site.cfg - -Runtime configuration -..................... - -Fluidsim is also sensitive to the environment variables: - -- ``FLUIDSIM_PATH``: path where the simulation results are saved. - - In Unix systems, you can for example put this line in your ``~/.bashrc``:: - - export FLUIDSIM_PATH=$HOME/Data - -- ``FLUIDDYN_PATH_SCRATCH``: working directory (can be useful on some clusters). - -Warning about re-installing fluidsim and fluidfft with new build options -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If fluidsim has already been installed and you want to recompile with new -configuration values, you need to really recompile fluidsim and not just -reinstall an already produced wheel. To do this, use:: - - pip install fluidsim --no-binary fluidsim -v - -``-v`` toggles the verbose mode of pip so that we see the compilation log and -can check that everything goes well. - -.. _pythranrc: - -About using Pythran to compile functions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -We choose to use the Python compiler `Pythran -`_ for some functions of the -operators. Our microbenchmarks show that the performances are as good as what -we are able to get with Fortran or C++! - -But it implies that building Fluidsim requires a C++11 compiler (for example -GCC 4.9 or clang). - -.. note:: - - If you don't want to use Pythran and C++ to speedup Fluidsim, you can use the - environment variable ``DISABLE_PYTHRAN``. - -.. warning:: - - To reach good performance, we advice to try to put in the file - ``~/.pythranrc`` the lines (it seems to work well on Linux, see the `Pythran - documentation `_): - - .. code:: bash - - [pythran] - complex_hook = True - -.. warning:: - - The compilation of C++ files produced by Pythran can be long and can consume - a lot of memory. If you encounter any problems, you can try to use clang (for - example with ``conda install clangdev``) and to enable its use in the file - ``~/.pythranrc`` with: - - .. code:: bash - - [compiler] - CXX=clang++ - CC=clang - -MPI simulations and mpi4py! -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Fluidsim can use `mpi4py `_ (which depends on a MPI -implementation) for MPI simulations. - -.. warning:: - - If the system has multiple MPI libraries, it is adviced to explicitly - mention the MPI command. For instance to use Intel MPI:: - - CC=mpiicc pip install mpi4py --no-binary mpi4py - -About h5py and HDF5_MPI -~~~~~~~~~~~~~~~~~~~~~~~ - -FluidSim is able to use h5py built with MPI support. - -.. warning:: - - Prebuilt installations (for e.g. via h5py wheels) lacks MPI support. - Most of the time, this is what you want. However, you can install h5py - from source and link it to a hdf5 built with MPI support, as follows: - - .. code:: bash - - $ CC="mpicc" HDF5_MPI="ON" HDF5_DIR=/path/to/parallel-hdf5 pip install --no-deps --no-binary=h5py h5py - $ python -c 'import h5py; h5py.run_tests()' - - In some cases you need to set C_INCLUDE_PATH variable before h5py - installation. For example on Debian stretch: - - .. code:: bash - - $ export C_INCLUDE_PATH=/usr/include/openmpi/ - $ CC="mpicc" HDF5_MPI="ON" HDF5_DIR=/path/to/parallel-hdf5 pip install --no-deps --no-binary=h5py h5py - - See the `h5py documentation - `_ for more details. - -Installing from the repository -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. note:: - - A good base to install Fluidsim from source can be to create and activate a - conda environment with:: - - conda create -y -n env-fluidsim -c conda-forge "fluidfft=*=*openmpi*" pythran clangdev mako - conda activate env-fluidsim - -For fluidsim, we use the revision control software Mercurial and the main -repository is hosted `here `_ in -Heptapod. Download the source with something like:: - - hg clone https://foss.heptapod.net/fluiddyn/fluidsim - -If you are new with Mercurial and Heptapod, you can also read `this short -tutorial -`_. - -For particular installation setup, copy the default configuration file to -``site.cfg``:: - - cp site.cfg.default site.cfg - -and modify it to fit your requirements. - -Build/install in development mode, by running from the top-level directory:: - - cd lib && pip install -e .; cd .. - pip install -e . - -.. note:: - - To install from source in a conda environment, it is actually necessary to - disable the isolated build by running the command ``pip install -e . - --no-build-isolation``. - -To install Fluidsim with all optional dependencies and all capacities:: - - pip install -e .[full] - -Run the tests! -.............. - -You can run some unit tests by running ``make tests`` (shortcut for -``fluidsim-test -v``) or ``make tests_mpi`` (shortcut for ``mpirun -np 2 -fluidsim-test -v``). Alternatively, you can also run ``pytest`` from the root -directory or from any of the source directories. diff --git a/doc/test_bench_profile.md b/doc/test_bench_profile.md new file mode 100644 index 000000000..98768b682 --- /dev/null +++ b/doc/test_bench_profile.md @@ -0,0 +1,34 @@ +# Testing, benchmarks and profiling + +Fluidsim comes with command-line tools for testing, benchmarking and profiling. +Here are useful commands: + +```bash +fluidsim -h +``` + +## Testing + +```bash +fluidsim-test -h +fluidsim-test -v +``` + +## Benchmarks + +```bash +fluidsim-bench -h +fluidsim-bench -s ns2d +``` + +```bash +fluidsim-bench-analysis -h +``` + +## Profiling + +```bash +fluidsim-profile -h +fluidsim-profile -s ns2d +fluidsim-profile -p -sf /tmp/fluidsim_profile/result_bench_ns2d_512x512_2017-11-19_22-19-26_8772.json +``` diff --git a/doc/test_bench_profile.rst b/doc/test_bench_profile.rst deleted file mode 100644 index 30a2ea546..000000000 --- a/doc/test_bench_profile.rst +++ /dev/null @@ -1,39 +0,0 @@ -Testing, benchmarks and profiling -================================= - -Fluidsim comes with command-line tools for testing, benchmarking and -profiling. Here are useful commands: - -.. code-block:: bash - - fluidsim -h - -Testing -------- - -.. code-block:: bash - - fluidsim-test -h - fluidsim-test -v - -Benchmarks ----------- - -.. code-block:: bash - - fluidsim-bench -h - fluidsim-bench -s ns2d - -.. code-block:: bash - - fluidsim-bench-analysis -h - - -Profiling ---------- - -.. code-block:: bash - - fluidsim-profile -h - fluidsim-profile -s ns2d - fluidsim-profile -p -sf /tmp/fluidsim_profile/result_bench_ns2d_512x512_2017-11-19_22-19-26_8772.json diff --git a/doc/to_do.md b/doc/to_do.md new file mode 100644 index 000000000..c5e7caf67 --- /dev/null +++ b/doc/to_do.md @@ -0,0 +1,19 @@ +# To do list + +## Long term + +- better integration with + + - paraview + - VAPOR is the Visualization and Analysis Platform for Ocean, Atmosphere, and + Solar Researchers () + - VisIt is a free \[and open-source\], interactive parallel visualization and + graphical analysis tool + () + +## Inline to do items + +```{eval-rst} +.. todolist:: + +``` diff --git a/doc/to_do.rst b/doc/to_do.rst deleted file mode 100644 index 36e76697a..000000000 --- a/doc/to_do.rst +++ /dev/null @@ -1,24 +0,0 @@ -To do list -========== - -Long term ---------- - -- better integration with - - * paraview - - * VAPOR is the Visualization and Analysis Platform for Ocean, - Atmosphere, and Solar Researchers (https://www.vapor.ucar.edu/) - - * VisIt is a free [and open-source], interactive parallel - visualization and graphical analysis tool - (https://wci.llnl.gov/simulation/computer-codes/visit) - - -Inline to do items ------------------- - -.. todolist:: - - diff --git a/doc/tutorials.md b/doc/tutorials.md new file mode 100644 index 000000000..05b813135 --- /dev/null +++ b/doc/tutorials.md @@ -0,0 +1,23 @@ +# Tutorials + +Most of these tutorials have been produced by Ipython notebook. + +```{toctree} +--- +maxdepth: 2 +--- +ipynb/tuto_user +ipynb/tuto_dev +ipynb/executed/taylor-green +``` + +## Parametric study with fluidsim.solvers.ns3d.strat + +```{toctree} +--- +maxdepth: 2 +--- +ipynb/executed/parametric_study_ns3dstrat/0_run_simulations +ipynb/executed/parametric_study_ns3dstrat/1_analyze_1_simul +ipynb/executed/parametric_study_ns3dstrat/2_compare_simulations +``` diff --git a/doc/tutorials.rst b/doc/tutorials.rst deleted file mode 100644 index f8adc0812..000000000 --- a/doc/tutorials.rst +++ /dev/null @@ -1,21 +0,0 @@ -Tutorials -========= - -Most of these tutorials have been produced by Ipython notebook. - -.. toctree:: - :maxdepth: 2 - - ipynb/tuto_user - ipynb/tuto_dev - ipynb/executed/taylor-green - -Parametric study with fluidsim.solvers.ns3d.strat -------------------------------------------------- - -.. toctree:: - :maxdepth: 2 - - ipynb/executed/parametric_study_ns3dstrat/0_run_simulations - ipynb/executed/parametric_study_ns3dstrat/1_analyze_1_simul - ipynb/executed/parametric_study_ns3dstrat/2_compare_simulations