Yanked versions listed in ERROR: Could not find a version that satisfies the requirement, but don't get installed

[pilosus]

Description

Let's say a package has versions published: ..., 1.47.0, 1.47.2, 1.48.0rc1, 1.48.0, 1.48.1, ...
Version 1.48.0 is yanked. So installing package>1.47.2,<1.48.1 without pre-releases should end up with an error No matching distribution found. It works as expected. But in addition to that in the output (from versions: ...) I see yanked version 1.48.0 still being listed. It seems to be somewhat misleading to me. If it's listed, why didn't it get installed?

Expected behavior

Yanked versions or versions not satisfying env markers, should not be listed in the output for no matching dist error:

ERROR: No matching distribution found for [...]

pip version

22.3.1

Python version

3.11.1

OS

x86_64 GNU/Linux 5.15.0-58-generic

How to Reproduce

1. pip install 'grpcio>1.47.2,<1.48.1'

Output

$ pip install 'grpcio>1.47.2,<1.48.1'
ERROR: Could not find a version that satisfies the requirement grpcio<1.48.1,>1.47.2 (from versions: 0.4.0a0, 0.4.0a1, 0.4.0a2, 0.4.0a3, 0.4.0a4, 0.4.0a5, 0.4.0a6, 0.4.0a7, 0.4.0a8, 0.4.0a13, 0.4.0a14, 0.5.0a0, 0.5.0a1, 0.5.0a2, 0.9.0a0, 0.9.0a1, 0.10.0a0, 0.11.0b0, 0.11.0b1, 0.12.0b0, 0.13.0, 0.13.1rc1, 0.13.1, 0.14.0rc1, 0.14.0, 0.15.0, 1.0.0rc1, 1.0.0rc2, 1.0.0, 1.0.1rc1, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.1.3, 1.2.0, 1.2.1, 1.3.0, 1.3.5, 1.4.0, 1.6.0, 1.6.3, 1.7.0, 1.7.3, 1.8.1, 1.8.2, 1.8.3, 1.8.4, 1.8.6, 1.9.0rc1, 1.9.0rc2, 1.9.0rc3, 1.9.0, 1.9.1, 1.10.0rc2, 1.10.0, 1.10.1rc1, 1.10.1rc2, 1.10.1, 1.11.0rc1, 1.11.0rc2, 1.11.0, 1.11.1rc1, 1.11.1, 1.12.0rc1, 1.12.0, 1.12.1, 1.13.0rc1, 1.13.0rc2, 1.13.0rc3, 1.13.0, 1.14.0rc1, 1.14.0rc2, 1.14.0, 1.14.1, 1.14.2rc1, 1.14.2, 1.15.0rc1, 1.15.0, 1.16.0rc1, 1.16.0, 1.16.1, 1.17.0, 1.17.1, 1.18.0, 1.19.0, 1.20.0rc1, 1.20.0rc2, 1.20.0rc3, 1.20.0, 1.20.1, 1.21.0rc1, 1.21.1rc1, 1.21.1, 1.22.0rc1, 1.22.0, 1.22.1, 1.23.0rc1, 1.23.0, 1.23.1, 1.24.0rc1, 1.24.0, 1.24.1, 1.24.3, 1.25.0rc1, 1.25.0, 1.26.0rc1, 1.26.0, 1.27.0rc1, 1.27.0rc2, 1.27.1, 1.27.2, 1.28.0rc1, 1.28.0rc2, 1.28.1, 1.29.0, 1.30.0, 1.31.0, 1.32.0, 1.33.1, 1.33.2, 1.34.0rc1, 1.34.0, 1.34.1, 1.35.0rc1, 1.35.0, 1.36.0rc1, 1.36.0, 1.36.1, 1.37.0rc1, 1.37.0, 1.37.1, 1.38.0rc1, 1.38.0, 1.38.1, 1.39.0rc1, 1.39.0, 1.40.0rc1, 1.40.0, 1.41.0rc2, 1.41.0, 1.41.1, 1.42.0rc1, 1.42.0, 1.43.0rc1, 1.43.0, 1.44.0rc1, 1.44.0rc2, 1.44.0, 1.45.0rc1, 1.45.0, 1.46.0rc1, 1.46.0rc2, 1.46.0, 1.46.1, 1.46.3, 1.46.5, 1.47.0rc1, 1.47.0, 1.47.2, 1.48.0rc1, 1.48.0, 1.48.1, 1.48.2, 1.49.0rc1, 1.49.0rc3, 1.49.0, 1.49.1, 1.50.0rc1, 1.50.0, 1.51.0rc1, 1.51.0, 1.51.1, 1.52.0rc1)
ERROR: No matching distribution found for grpcio<1.48.1,>1.47.2

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[notatallshaw]

Its probably a little tricky because it is possible for pip to install a Yanked version, e.g. if you specify package==1.48.0

I imagine the easiest solution if someone wants to make a PR would be to tage the version as yanked in the error message, e.g 1.48.0(Yanked!). Then it would at least make clear to the user that version might not be a candidate?

[pilosus]

I do understand that you can install a yanked version. I guess the whole point of yanking (as opposed to removing the package) is not to break people's code if they use the yanked version already, but on the other hand is to make sure other people without exact version preferences (like in my example above) don't start using the yanked version.

Basically, the only problem with the current pip's behaviour is inconsistency: if my version specifiers don't let me to install the yanked version, then it shouldn't be listed in the output either.

Marking versions explicitly in the error as yanked may also work. But may be as a separate section in the outout? Like it works for requires-python marker:

$ python --version
Python 3.7.16

$ pip --version
pip 22.3.1 

# aio-request is cpython >=3.8
$ pip install --dry-run 'aio-request>0.1.0<0.1.7' 
ERROR: Ignored the following versions that require a different python version: 0.0.1 Requires-Python >=3.8; [...] 0.1.9a4 Requires-Python >=3.8
ERROR: Could not find a version that satisfies the requirement aio-request>0.1.0<0.1.7 (from versions: none)
ERROR: No matching distribution found for aio-request>0.1.0<0.1.7

So, in my example with yanked it could be like this then:

ERROR: Ignored the following versions that have been yanked: 1.48.0.
ERROR: Could not find a version that satisfies the requirement grpcio<1.48.1,>1.47.2 (from versions: 0.4.0a0, ...)
ERROR: No matching distribution found for grpcio<1.48.1,>1.47.2

[notatallshaw]

I'm not confident it's easy to determine if something was ignored because it was yanked, if pip has to backtrack it could be in one state it considers the yanked package and in another state it doesn't consider the yanked package.

Either way though I'm sure Pip maintainers (though I am not one myself so I'm guessing) would seriously consider a PR for this.

[pfmoore]

Either way though I'm sure Pip maintainers (though I am not one myself so I'm guessing) would seriously consider a PR for this.

If we can improve the messages pip gives, that would definitely be welcome, yes. However, I think you're right (disclaimer - I haven't looked at this part of the code in a while) that it would be quite hard in practice to get the information. Particularly as the specification allows for only individual files in a release to be yanked, so saying "version X is yanked" isn't entirely accurate (even though I think PyPI only allows yanking of whole releases).

TBH, the best way for these sorts of details to get worked out is for someone to create a PR. We're not going to get much further just talking about the problem. To that end, I've marked this as "Awaiting PR".

[pilosus]

Ok, I will try to draft a PR then

[pradyunsg]

I'd prefer to clean up this logic TBH -- avoiding presenting the entire list of versions unless someone uses an increased verbosity.

The tricky thing with this situation is that we know that users use pip install foo== as a "trick" to find the latest version by looking at this error message. I reckon the right thing to do would improve this error message to include the counts of how many releases were found as well as the highest and lowest releases at the default verbosity (i.e. info level); with the entire list being presented at increase verbosity (i.e. verbose level).

[ddelange]

Hi 👋 Your idea sounds great @pradyunsg, more info for the users would be great (to somehow allow filtering out yanked versions).

Instead of users having to parse the output of this "trick", do you think it would be a nice solution to include a verbose available_versions in the json output of pip install --ignore-installed --no-deps --dry-run --report - foo ? Then the format of the "trick" can stay the same.

[ddelange]

I've opened #12224 and #12225 👍