diff --git a/docs/howto/charm-to-poetry.rst b/docs/howto/charm-to-poetry.rst new file mode 100644 index 000000000..5b2d3655d --- /dev/null +++ b/docs/howto/charm-to-poetry.rst @@ -0,0 +1,108 @@ +.. _howto-migrate-to-poetry: + +Migrate from the Charm plugin to the Poetry plugin +================================================== + +Many charms use `Poetry`_ to manage their Python projects. For these charms, Charmcraft +has a :ref:`craft_parts_poetry_plugin`. Migrating from the Charm plugin provides some +benefits, such as no longer having to maintain a ``requirements.txt`` file. If the +charm to be migrated does not currently use poetry, refer to the +`Poetry documentation `_ for instructions +on how to use poetry for a Python project. + +Update ``charmcraft.yaml`` +-------------------------- + +The first step is to update ``charmcraft.yaml`` to include the correct parts definition. +Depending on the history of a specific charm, it may not have an explicitly-included +``parts`` section determining how to build the charm. In this case, a ``parts`` section +can be created as follows: + +.. code-block:: yaml + + parts: + my-charm: # This can be named anything you want + plugin: poetry + source: . + +Select compatible versions of ``pip`` and ``poetry`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Poetry plugin requires at least `pip 22.3 +`_, released in October 2022. If the +charm's base uses an older version of pip, a newer version can be installed in the +build environment using a dependency part. Likewise, a charm may require a newer +version of Poetry than is available in the distribution's repositories. The following +``parts`` section can be used in place of the section above to upgrade pip and Poetry +for charms that build on Ubuntu 22.04 or earlier: + +.. code-block:: yaml + :emphasize-lines: 2-9,11 + + parts: + poetry-deps: + plugin: nil + build-packages: + - curl + override-build: | + /usr/bin/python3 -m pip install pip==24.2 + curl -sSL https://install.python-poetry.org | python3 - + ln -sf $HOME/.local/bin/poetry /usr/local/bin/poetry + my-charm: # This can be named anything you want + after: [poetry-deps] + plugin: poetry + source: . + +Add optional dependency groups +------------------------------ + +If the charm has optional `dependency groups`_ that should be included when creating +the virtual environment, the ``poetry-with`` key can be used to include those groups +when creating the virtual environment. + +.. note:: + This is useful and encouraged, though not mandatory, for keeping track of + library dependencies, as covered in the next section. For an example, see + `postgresql-operator`_. + +Include charm library dependencies +---------------------------------- + +Unlike the Charm plugin, the Poetry plugin does not install the dependencies for +included charmlibs. If any of the charm libraries used have PYDEPS, these will +need to be added to the charm's dependencies, potentially as their own +`dependency group `_. + +To find these dependencies, check each library file for its ``PYDEPS``. A command +that can find these is:: + + find lib -name "*.py" -exec awk '/PYDEPS = \[/,/\]/' {} + + +If run from the base directory of a charm, this will show all the PYDEPS declarations +from all loaded charm libs. + +Include extra files +------------------- + +A Poetry plugin only includes the contents of the ``src`` and ``lib`` directories +as well as the generated virtual environment. If other files were previously included +from the main directory, they can be included again using the +:ref:`craft_parts_dump_plugin`: + +.. code-block:: yaml + :emphasize-lines: 5-9 + + parts: + my-charm: # This can be named anything you want + plugin: poetry + source: . + version-file: + plugin: dump + source: . + stage: + - charm_version + + +.. _dependency groups: https://python-poetry.org/docs/managing-dependencies/#dependency-groups +.. _Poetry: https://python-poetry.org +.. _postgresql-operator: https://github.com/canonical/postgresql-operator/blob/3c7c783d61d4bee4ce64c190a9f7d4a78048e4e7/pyproject.toml#L22-L35 diff --git a/docs/howto/charm-to-python.rst b/docs/howto/charm-to-python.rst new file mode 100644 index 000000000..cf530aa71 --- /dev/null +++ b/docs/howto/charm-to-python.rst @@ -0,0 +1,129 @@ +.. _howto-migrate-to-python: + +Migrate from the Charm plugin to the Python plugin +================================================== + +The Python plugin in Charmcraft offers a faster, stricter means of packing an operator +charm with a virtual environment. This guide shows how to migrate from a charm using +the default Charm plugin to using the Python plugin. + +Update ``charmcraft.yaml`` +-------------------------- + +The first step is to update ``charmcraft.yaml`` to include the correct parts definition. +Depending on the history of a specific charm, it may not have an explicitly-included +``parts`` section determining how to build the charm. In this case, a ``parts`` section +can be created as follows: + +.. code-block:: yaml + + parts: + my-charm: # This can be named anything you want + plugin: python + source: . + python-requirements: + - requirements.txt # Or whatever your requirements file is called. + +Select a compatible version of ``pip`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Python plugin requires at least `pip 22.3`_, released in October 2022. If the +charm's base uses an older version of pip, a newer version can be installed in the +build environment using a dependency part. The following ``parts`` section can be +used in place of the section above to upgrade pip for charms that build on Ubuntu +22.04 or earlier: + +.. code-block:: yaml + :emphasize-lines: 2-5,7 + + parts: + python-deps: + plugin: nil + override-build: | + /usr/bin/python3 -m pip install pip==24.2 + my-charm: # This can be named anything you want + after: [python-deps] + plugin: python + source: . + python-requirements: + - requirements.txt # Or whatever your requirements file is called. + +Flatten ``requirements.txt`` +---------------------------- + +One difference between the Python plugin and the Charm plugin is that the Python +plugin does not install dependencies, so the ``requirements.txt`` file must be a +complete set of packages needed in the charm's virtual environment. + +.. note:: + There are several tools for creating an exhaustive ``requirements.txt`` file. + Charmcraft works with any as long as it generates a requirements file that ``pip`` + understands. Because different versions of packages may have different + dependencies, it is recommended that the requirements file be generated using a + tool that will lock the dependencies to specific versions. + A few examples include: + + - `uv export `_ + - `pip-compile `_ + - `pip freeze `_ + +A basic ``requirements.txt`` file for a charm with no dependencies other than the +Operator framework may look something like:: + + ops==2.17.0 + pyyaml==6.0.2 + websocket-client==1.8.0 + +To check that the virtual environment for the charm would be valid, activate an +empty virtual environment and then run:: + + pip install --no-deps -r requirements.txt + pip check + +Include charm library dependencies +---------------------------------- + +Unlike the Charm plugin, the Python plugin does not install the dependencies +for included charmlibs. If any of the charm libraries used have PYDEPS, these will +need to be added to a requirements file as well. + +.. note:: + All requirements files are included in the same ``pip`` command to prevent + conflicting requirements from overriding each other. However, this means + that a charm will fail to build if it has conflicting requirements. A single + ``requirements.txt`` file, while not mandatory, is recommended. + +To find these dependencies, check each library file for its ``PYDEPS``. A command +that can find these is:: + + find lib -name "*.py" -exec awk '/PYDEPS = \[/,/\]/' {} + + +If run from the base directory of a charm, this will show all the PYDEPS declarations +from all loaded charm libs, which can be used to help generate the input for a tool +that generates ``requirements.txt``. + +Include extra files +------------------- + +The Python plugin only includes the contents of the ``src`` and ``lib`` directories +as well as the generated virtual environment. If other files were previously included +from the main directory, they can be included again using the +:ref:`craft_parts_dump_plugin`: + +.. code-block:: yaml + :emphasize-lines: 7-11 + + parts: + my-charm: # This can be named anything you want + plugin: python + source: . + python-requirements: + - requirements.txt # Or whatever your requirements file is called. + version-file: + plugin: dump + source: . + stage: + - charm_version + + +.. _pip 22.3: https://pip.pypa.io/en/stable/news/#v22-3 diff --git a/docs/howto/index.rst b/docs/howto/index.rst new file mode 100644 index 000000000..46ccff52b --- /dev/null +++ b/docs/howto/index.rst @@ -0,0 +1,10 @@ +.. _howto: + +How-To +****** + +.. toctree:: + :maxdepth: 2 + + charm-to-poetry + charm-to-python diff --git a/docs/index.rst b/docs/index.rst index b0bd33862..4e9931d9f 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -10,9 +10,20 @@ Most of Charmcraft's documentation is available there. :maxdepth: 1 :hidden: + howto/index reference/index explanation/index +.. grid:: 1 1 2 2 + + .. grid-item-card:: `Tutorial `_ + + **Get started** with a hands-on introduction to Charmcraft + + .. grid-item-card:: :ref:`How-to guides ` + + **Step-by-step guides** covering key operations and common tasks + .. grid:: 1 1 2 2 :reverse: diff --git a/docs/reference/plugins/python_plugin.rst b/docs/reference/plugins/python_plugin.rst index 911c94c23..3b9c8a256 100644 --- a/docs/reference/plugins/python_plugin.rst +++ b/docs/reference/plugins/python_plugin.rst @@ -49,7 +49,6 @@ During the build step, the plugin performs the following actions: 4. It copies any existing ``src`` and ``lib`` directories from your charm project into the final charm. - Example -------