Ability to block a transitive dependency

[mortenlj]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Feature Request

It would in some cases be useful to block a transitive dependency from being installed.

There are a couple use cases that I can think of:

1. Using a small part of a larger library, where the library pulls in some dependencies that are only needed for the parts you are not using.
2. Replacing a transitive dependency with an alternative implementation

The first one is why I started looking for this feature, because we use Prospector to run various code quality tools. One of the tools Prospector pulls in is PyLint, but we are not using PyLint and since PyLint is GPL we would rather avoid pulling it in causing licensing questions.

The second one could be things like the PIL -> Pillow transition that happened some years ago, or similar situations where a new library intends to take the place of an older, deprecated library. This kind of thing is more common in other ecosystems (logging in JVM is a prime example), but can be very useful in some situations.

Both of these use cases would be supported if you could somehow tell poetry to ignore packages with a certain name when processing the dependency graph.

[dimbleby]

You should ask prospector to treat pylint as an optional extra, if that's the behaviour you want.

[mortenlj]

You should ask prospector to treat pylint as an optional extra, if that's the behaviour you want.

In an ideal world, where every project marked all dependencies as optional if it is possible to use parts of their project without that dependency, this would be feasible. In reality, working with extras is just cumbersome enough that if 90% of your users are going to need a specific dependency, you're not going to make that an optional extra. Solving this use case in the "package manager" has a much higher bang-for-bucks ratio.

The second use case mentioned above could also be solved by making every project that depend on package A change to allow using package B instead, but I think that's even less realistic. Solving it in the "package manager" is way more efficient and practical.

[ralbertazzi]

Solving it in the "package manager" is way more efficient and practical.

Interesting, I would find fixing the problem at the source way more efficient and practical than having to exclude transitive dependencies 😄 @mortenlj you seem to be the only person who ever raised this point, which means that it's unlikely this will ever get implemented if you don't provide a change request. Otherwise, I feel this issue will get closed as unplanned

[mortenlj]

Interesting, I would find fixing the problem at the source way more efficient and practical than having to exclude transitive dependencies

For 1 package: sure
For 2 packages: again, sure
For 10 packages: maybe
For 100 packages: not so much

It boils down to how likely this situation is, and in other ecosystems it is so common that the tools support it. Both Gradle and Maven allow it in JVM, Go allows using replace rules to switch out modules in go.mod. Cargo includes support for patching the dependency graph as part of Cargo.toml.

On the other hand, if I'm the only person in all of Python who has ever needed this (and even I can't remember which project this was anymore), I guess it's fair to close it as unplanned. 🤷

[jgangemi]

i could use this functionality as well. i have a package that needlessly declares sphinx-rtd-theme as a dependency, yet i'm forced to download/package it up as part of my application.

[cbornet]

On the other hand, if I'm the only person in all of Python who has ever needed this (and even I can't remember which project this was anymore), I guess it's fair to close it as unplanned.

You're not alone. I also need this 😃.
3 👍 on the issue so far. Maybe this could be considered ?
This is a common feature in other package managers (eg Java: maven, gradle, ...)

[realitycheck]

@ralbertazzi consider the following use case:

"My" library has a dependency to library "A" which provides "import A" package, then in a wild there is a library "A+" which provides "import A" also, though with a different api or purpose, so that library is a part of some dependency to the project with is using the "My" library.

So I as an author of "My" library would like to have as a precaution an ability to indicate via package manager that the "My" library is incompatible with "A+" library and they cannot be installed both into the same environment.

[colindean]

I'm finding a need for this presently as I want to install sentence-transformers which depends on torch>=1.6.0, which pulls in a bunch of GPU compute packages from the nvidia-* family.

Ultimately, I think I'm headed down the track of manually specifying the torch CPU packages à la #4231 (comment).

[garyd203]

A concrete example. python-jose has ecdsa as a default dependency, but if cryptography is installed (eg. by specifying the cryptography extra, or as an independent dependency) then at runtime that is used instead of ecdsa. Given that ecdsa now has a "wont fix" security alert, it'd be nice to not even ship it.

So as a user of python-jose, my current course of action would be:

1. Find a way to rejig the dependencies for python-jose so that ecdsa is not installed if the cryptography extra is supplied, in such a way that existing use cases are not affected (is this even possible?). Hopefully their packaging specification is clean and up-to-date enough that it's easy to make my change.
2. Hope that the maintainer of python-jose is still active 🤞
3. Work with the maintainer of python-jose to accept my patch to their packaging system.
4. Wait for a new version of python-jose to be released.
5. Update my dependencies to use the patched version of python-jose.
6. Hope that all my direct dependencies which transitively use python-jose don't have a misconfigured version spec which blocks them from using the patched version.

Obviously it'd be much simpler and more reliable for me if I could just change my own dependencies today, and move on.

Aside: My experience is that the Python community seems to get most of this library management process right, which may be why it's not a common issue for us. But in general, if one of the goals of a dependency management tool like poetry is to allow the user to control what libraries they are using, then that should include not using a particular (version of a) library

[swssl]

@garyd203 +1 and thanks for bringing this up. For me it was also the ecdsa package that let me find this issue.

[carlinix]

@garyd203 +1 I'm running on the same situation.

[cbornet]

Another example where that would be useful: run-llama/llama_index#11896
Here a dependency pulls another dependency transitively that could have been optional and that have a problematic license.
It would have been useful to be able to exclude that dependency and not wait for a fixed version.

[cbornet]

But thinking about it again, I believe this should be supported in Python packaging first.

[heimalne]

A case where idealism clashes with reality. This is indeed a feature that is needed for real-world usage, where sometimes you can't just update to a newer version or wait for a fix to be made upstream.

[auxsvr]

Another case is to prevent installation of brotlipy when brotli is installed, as both are imported as brotli and their APIs are incompatible.

[finswimmer]

This is unlikely to get implemented as long as there is no standard in the python ecosystem.

There are some discussions around about the ability to declare alternative dependencies e.g. https://discuss.python.org/t/require-any-of-several-alternative-package-dependencies/26293 If there is finally a PEP for it, it might be possible together with #7051 to block a transitive dependency.

[garyd203]

@finswimmer I'm not sure that this should be dependent on upstream support for alternative dependencies.

There are some discussions around about the ability to declare alternative dependencies e.g. https://discuss.python.org/t/require-any-of-several-alternative-package-dependencies/26293 If there is finally a PEP for it,

My understanding of that discussion is that the proposal is for a package (eg. Prospector, for the OP) to specify its dependencies using that technique, and then I get to control which one of those transitive dependencies I bring in.

But the point of the proposal in this discussion is that i get to control which transitive dependencies I bring in even when another package has done a bad job of specifying dependencies. That is, I want to fully control my transitive dependencies via some poetry mechanism, so that I am not blocked by the choices of 3rd party maintainers that don't suit me.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.