Conflict with Pipenv When Executing Bin Files

[KerickHowlett]

Describe the bug
Right away I wish to front load this discussion by saying I don't know if this is indeed a bug with rtx or if it's just my unfamiliarity with the Python ecosystem or a Lovecraftian fusion of both. 😅

With that said, I had recently encountered a bizarre and frustrating situation while attempting to run pytest on a project I had been developing on at work today when this weird bug crept up:

It kept claiming the app's dependencies (e.g., mock, freezegun) were not installed. Thinking it odd but not unheard of, I just shrugged my shoulders and attempted to resync my Pipenv environment. It told me everything was already installed. I then attempted to a hard, clean re-install by deleting the app's local .venv directory before installing everything again.

After some time, it finished installing, so I tried again -- same error!

After some troubleshooting, I realized that all of my installs were being directed towards the rtx (shim?) directory, which is located within my $HOME directory; however, it was running pytest from .venv/bin/pytest, so it had been expecting the dependencies/modules to be located in the project's virtual environment directory. As I'm sure you already guessed, none of them were there.

I had tried a number of things from running rtx shell and pipenv shell --always a number of times, and even tried reshelling my fish.

None of it worked!

Although, I did end up getting it to work, even if I feel to perhaps be nothing more workaround: I adjusted the configuration settings of my project's local .rtx.toml file to have the listed [tools] entry for python to now look like this:

[tools]
python = { version = '3.8.17', virtualenv = '.venv' }

Before, it only had the version number.

And even now, I have to install packages for it using python -m pipenv install <options> after making sure that it's actually using the correct .venv/bin/python path this time.

To Reproduce
Not quite sure how to reproduce it, exactly.

The only things I can think of to mention are having this as your best fitting Python/pipenv project look something like this:

[env]
PIP_NO_CACHE_DIR = 'false'

PIPENV_VENV_IN_PROJECT = 'true'
PIPENV_VERBOSITY = '-1'

PYTEST_THEME = 'github-dark'
PYTEST_MODE = 'dark'

PYTHONDONTWRITEBYTECODE = 'true'
PYTHONFAULTHANDLER = 'true'
PYTHONUNBUFFERED = 'true'

PYTHONPATH = "{{env.PWD}}"
TESTPATH = '{{env.PWD}}/tests/unittests'

[tools]
python = "3.8.17"

I also made it a point to have things like pytest only installed in my respective project's respective virtual environment directories and never globally; however, I did end up having pipenv installed globally, because I thought that'd be safe to do so without it leading to any problems. With that said, even after uninstalling it, I still have to use python -m pipenv just to make sure the dependencies/modules are coming from the right place.

Expected behavior
A clear and concise description of what you expected to happen.

rtx doctor output

rtx version:
  2023.9.1 macos-arm64 (c6d59d7 2023-09-13)

build:
  Target: aarch64-apple-darwin
  Features: DEFAULT, NATIVE_TLS, OPENSSL
  Built: Wed, 13 Sep 2023 14:18:58 +0000
  Rust Version: rustc 1.72.0 (5680fa18f 2023-08-23)
  Profile: release

shell:
  /opt/homebrew/bin/fish
  fish, version 3.6.1

rtx data directory:
  /Users/ooo998/.local/share/rtx

rtx environment variables:
  RTX_SHELL=fish
  RTX_PYTHON_DEFAULT_PACKAGES_FILE=/Users/ooo998/.default-python-packages

settings:
  {"always_keep_download": "false", "always_keep_install": "false", "asdf_compat": "false", "disable_default_shorthands": "false", "disable_tools": "[]", "experimental": "false", "jobs": "4", "legacy_version_file": "true", "legacy_version_file_disable_tools": "[]", "log_level": "INFO", "missing_runtime_behavior": "warn", "plugin_autoupdate_last_check_duration": "10080", "raw": "false", "trusted_config_paths": "[]", "verbose": "false", "yes": "false"}

config files:
  /Users/ooo998/.config/rtx/config.toml
  /Users/ooo998/projects/capitalone/raas/bank-fraud-case-manager/.rtx.toml

plugins:
  go       (core)
  java     (core)
  node     (core)
  python   (core)
  ruby     (core)

toolset:
  [email protected], node@latest

No problems found

[jdx]

I figured out the issue here: list_bin_paths adds the bin directory to PATH when activating the virtualenv. This has an unintended side effect which is that it creates a shim for every bin in that directory.

What needs to happen is PATH should be modified without making all of these shims.

[KerickHowlett]

Apologies for the slow reply; I've been crazy busy lately.

At any case, thank you so much for looking into it and for the fast fix! :)