Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pip inspect command #11245

Merged
merged 4 commits into from
Jul 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/html/cli/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ pip
pip_install
pip_uninstall
pip_inspect
pip_list
pip_show
pip_freeze
Expand Down
33 changes: 33 additions & 0 deletions docs/html/cli/pip_inspect.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
.. _`pip inspect`:

===========
pip inspect
===========

.. versionadded:: 22.2

sbidoul marked this conversation as resolved.
Show resolved Hide resolved

Usage
=====

.. tab:: Unix/macOS

.. pip-command-usage:: inspect "python -m pip"

.. tab:: Windows

.. pip-command-usage:: inspect "py -m pip"


Description
===========

.. pip-command-description:: inspect

The format of the JSON output is described in :doc:`../reference/inspect-report`.


Options
=======

.. pip-command-options:: inspect
1 change: 1 addition & 0 deletions docs/html/reference/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,5 @@ build-system/index
requirement-specifiers
requirements-file-format
installation-report
inspect-report
```
217 changes: 217 additions & 0 deletions docs/html/reference/inspect-report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
# `pip inspect` JSON output specification

```{versionadded} 22.2
```

The `pip inspect` command produces a detailed JSON report of the Python
environment, including installed distributions.

## Specification

The report is a JSON object with the following properties:

- `version`: the string `0`, denoting that the inspect command is an experimental
feature. This value will change to `1`, when the feature is deemed stable after
gathering user feedback (likely in pip 22.3 or 23.0). Backward incompatible changes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should commit to this… Versions numbers recorded in documentation likely this generally get stale without being updated.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its good to give a sense of when we intend the feature to become stable, and I'm motivated to follow this up in the coming month so it's unlikely that I'll forget. So I'd be in favor of keeping it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And, we're using "likely" to give us some leeway, in case we do miss this deadline in some way. :)

may be introduced in version `1` without notice. After that, it will change only if
and when backward incompatible changes are introduced, such as removing mandatory
fields or changing the semantics or data type of existing fields. The introduction of
backward incompatible changes will follow the usual pip processes such as the
deprecation cycle or feature flags. Tools must check this field to ensure they support
the corresponding version.

- `pip_version`: a string with the version of pip used to produce the report.
sbidoul marked this conversation as resolved.
Show resolved Hide resolved

- `installed`: an array of [InspectReportItem](InspectReportItem) representing the
distribution packages that are installed.

- `environment`: an object describing the environment where the installation report was
generated. See [PEP 508 environment
markers](https://peps.python.org/pep-0508/#environment-markers) for more information.
Values have a string type.

(InspectReportItem)=

An `InspectReportItem` is an object describing an installed distribution package with
the following properties:

- `metadata`: the metadata of the distribution, converted to a JSON object according to
the [PEP 566
transformation](https://www.python.org/dev/peps/pep-0566/#json-compatible-metadata).

- `metadata_location`: the location of the metadata of the installed distribution. Most
of the time this is the `.dist-info` directory. For legacy installs it is the
`.egg-info` directory.

```{warning}
This field may not necessary point to a directory, for instance, in the case of older
`.egg` installs.
```

- `direct_url`: Information about the direct URL that was used for installation, if any,
using the [direct
URL](https://packaging.python.org/en/latest/specifications/direct-url/) data
structure. In most case, this field corresponds to the `direct_url.json` metadata,
except for legacy editable installs, where it is emulated.

- `requested`: `true` if the `REQUESTED` metadata is present, `false` otherwise. This
field is only present for modern `.dist-info` installations.

```{note}
The `REQUESTED` metadata may not be generated by all installers.
It is generated by pip since version 20.2.
```

- `installer`: the content of the `INSTALLER` metadata, if present and not empty.

## Example

Running the ``pip inspect`` command, in an environment where `pip` is installed in
editable mode and `packaging` is installed as well, will produce an output similar to
this (metadata abriged for brevity):

```json
{
"version": "0",
"pip_version": "22.2.dev0",
"installed": [
{
"metadata": {
"metadata_version": "2.1",
"name": "pyparsing",
"version": "3.0.9",
"summary": "pyparsing module - Classes and methods to define and execute parsing grammars",
"description_content_type": "text/x-rst",
"author_email": "Paul McGuire <[email protected]>",
"classifier": [
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"Intended Audience :: Information Technology",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: PyPy",
"Typing :: Typed"
],
"requires_dist": [
"railroad-diagrams ; extra == \"diagrams\"",
"jinja2 ; extra == \"diagrams\""
],
"requires_python": ">=3.6.8",
"project_url": [
"Homepage, https://github.com/pyparsing/pyparsing/"
],
"provides_extra": [
"diagrams"
],
"description": "..."
},
"metadata_location": "/home/me/.virtualenvs/demoenv/lib/python3.8/site-packages/pyparsing-3.0.9.dist-info",
"installer": "pip",
"requested": false
},
{
"metadata": {
"metadata_version": "2.1",
"name": "packaging",
"version": "21.3",
"platform": [
"UNKNOWN"
],
"summary": "Core utilities for Python packages",
"description_content_type": "text/x-rst",
"home_page": "https://github.com/pypa/packaging",
"author": "Donald Stufft and individual contributors",
"author_email": "[email protected]",
"license": "BSD-2-Clause or Apache-2.0",
"classifier": [
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"License :: OSI Approved :: BSD License",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: PyPy"
],
"requires_dist": [
"pyparsing (!=3.0.5,>=2.0.2)"
],
"requires_python": ">=3.6",
"description": "..."
},
"metadata_location": "/home/me/.virtualenvs/demoenv/lib/python3.8/site-packages/packaging-21.3.dist-info",
"installer": "pip",
"requested": true
},
{
"metadata": {
"metadata_version": "2.1",
"name": "pip",
"version": "22.2.dev0",
"summary": "The PyPA recommended tool for installing Python packages.",
"home_page": "https://pip.pypa.io/",
"author": "The pip developers",
"author_email": "[email protected]",
"license": "MIT",
"classifier": [
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Topic :: Software Development :: Build Tools",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: PyPy"
],
"requires_python": ">=3.7",
"project_url": [
"Documentation, https://pip.pypa.io",
"Source, https://github.com/pypa/pip",
"Changelog, https://pip.pypa.io/en/stable/news/"
],
"description": "..."
},
"metadata_location": "/home/me/pip/src/pip.egg-info",
"direct_url": {
"url": "file:///home/me/pip/src",
"dir_info": {
"editable": true
}
}
}
],
"environment": {
"implementation_name": "cpython",
"implementation_version": "3.8.10",
"os_name": "posix",
"platform_machine": "x86_64",
"platform_release": "5.13-generic",
"platform_system": "Linux",
"platform_version": "...",
"python_full_version": "3.8.10",
"platform_python_implementation": "CPython",
"python_version": "3.8",
"sys_platform": "linux"
}
}
```
2 changes: 2 additions & 0 deletions news/11245.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add ``pip inspect`` command to obtain the list of installed distributions and other
information about the Python environment, in JSON format.
5 changes: 5 additions & 0 deletions src/pip/_internal/commands/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@
"FreezeCommand",
"Output installed packages in requirements format.",
),
"inspect": CommandInfo(
"pip._internal.commands.inspect",
"InspectCommand",
"Inspect the python environment.",
),
"list": CommandInfo(
"pip._internal.commands.list",
"ListCommand",
Expand Down
97 changes: 97 additions & 0 deletions src/pip/_internal/commands/inspect.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
import logging
from optparse import Values
from typing import Any, Dict, List

from pip._vendor.packaging.markers import default_environment
from pip._vendor.rich import print_json

from pip import __version__
from pip._internal.cli import cmdoptions
from pip._internal.cli.req_command import Command
from pip._internal.cli.status_codes import SUCCESS
from pip._internal.metadata import BaseDistribution, get_environment
from pip._internal.utils.compat import stdlib_pkgs
from pip._internal.utils.urls import path_to_url

logger = logging.getLogger(__name__)


class InspectCommand(Command):
"""
Inspect the content of a Python environment and produce a report in JSON format.
"""

ignore_require_venv = True
usage = """
%prog [options]"""

def add_options(self) -> None:
self.cmd_opts.add_option(
"--local",
action="store_true",
default=False,
help=(
"If in a virtualenv that has global access, do not list "
"globally-installed packages."
),
)
self.cmd_opts.add_option(
"--user",
dest="user",
action="store_true",
default=False,
help="Only output packages installed in user-site.",
)
self.cmd_opts.add_option(cmdoptions.list_path())
self.parser.insert_option_group(0, self.cmd_opts)

def run(self, options: Values, args: List[str]) -> int:
logger.warning(
"pip inspect is currently an experimental command. "
"The output format may change in a future release without prior warning."
)

cmdoptions.check_list_path_option(options)
dists = get_environment(options.path).iter_installed_distributions(
local_only=options.local,
user_only=options.user,
skip=set(stdlib_pkgs),
)
output = {
"version": "0",
"pip_version": __version__,
"installed": [self._dist_to_dict(dist) for dist in dists],
"environment": default_environment(),
# TODO tags? scheme?
pradyunsg marked this conversation as resolved.
Show resolved Hide resolved
}
print_json(data=output)
return SUCCESS

def _dist_to_dict(self, dist: BaseDistribution) -> Dict[str, Any]:
res: Dict[str, Any] = {
"metadata": dist.metadata_dict,
"metadata_location": dist.info_location,
}
# direct_url. Note that we don't have download_info (as in the installation
# report) since it is not recorded in installed metadata.
direct_url = dist.direct_url
if direct_url is not None:
res["direct_url"] = direct_url.to_dict()
else:
# Emulate direct_url for legacy editable installs.
editable_project_location = dist.editable_project_location
if editable_project_location is not None:
res["direct_url"] = {
"url": path_to_url(editable_project_location),
"dir_info": {
"editable": True,
},
}
# installer
installer = dist.installer
if dist.installer:
res["installer"] = installer
# requested
if dist.installed_with_dist_info:
res["requested"] = dist.requested
return res
4 changes: 4 additions & 0 deletions src/pip/_internal/metadata/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,10 @@ def installer(self) -> str:
return cleaned_line
return ""

@property
def requested(self) -> bool:
return self.is_file("REQUESTED")

@property
def editable(self) -> bool:
return bool(self.editable_project_location)
Expand Down
Loading