Exposed credentials

[clod6677]

Environment:

• pip version: pip 20.0.2
• Python version: Python 3.8.1
• OS: Ubuntu 18.04.3 LTS

Execution in a conda env. Installing a package though pip from a private repo (using user + token) located in a subdirectory.

Description
pip expose the credentials in case of Authentication failure. pip logs print the content of the env variable WRONG_TOKEN (see below WRONG_TOKEN_EXPOSED)

Expected behavior
Supplied credentials not shown in the logs nor in the trace-back, even if the authentication fails.

How to Reproduce

1. Get package from '...'
   conda create -n test python=3
2. Then run '...'
   pip install git+https://:${WRONG_TOKEN}@myrepo.git@${LIBS_SHA}#subdirectory=pkg/my-package
3. An error occurs.

Output

Collecting git+https://<my-user>:****@my-repo.git@#subdirectory=pkg/my-package
  Cloning https://<my-user>:****@my-repo.git to /tmp/pip-req-build-dklyop9v
  Running command git clone -q 'https://<my-user>:****@my-repo.git' /tmp/pip-req-build-dklyop9v
  remote: HTTP Basic: Access denied
  fatal: Authentication failed for 'https://<my-user>:[email protected]/'
ERROR: Command errored out with exit status 128: git clone -q 'https://<my-user>:****@my-repo.git' /tmp/pip-req-build-dklyop9v Check the logs for full command output.```

[uranusjr]

The culprit is pip._internal.utils.subprocess:call_subprocess, specifically this line:

pip/src/pip/_internal/utils/subprocess.py

Line 178 in ce1e0f4

log_subprocess("Running command %s", command_desc)

Which is called by pip._internal.vcs.versioncontrol:VersionControl.run_command:

pip/src/pip/_internal/vcs/versioncontrol.py

Lines 652 to 690 in ce1e0f4

	@classmethod
	def run_command(
	cls,
	cmd, # type: Union[List[str], CommandArgs]
	show_stdout=True, # type: bool
	cwd=None, # type: Optional[str]
	on_returncode='raise', # type: str
	extra_ok_returncodes=None, # type: Optional[Iterable[int]]
	command_desc=None, # type: Optional[str]
	extra_environ=None, # type: Optional[Mapping[str, Any]]
	spinner=None, # type: Optional[SpinnerInterface]
	log_failed_cmd=True # type: bool
	):
	# type: (...) -> Text
	"""
	Run a VCS subcommand
	This is simply a wrapper around call_subprocess that adds the VCS
	command name, and checks that the VCS is available
	"""
	cmd = make_command(cls.name, *cmd)
	try:
	return call_subprocess(cmd, show_stdout, cwd,
	on_returncode=on_returncode,
	extra_ok_returncodes=extra_ok_returncodes,
	command_desc=command_desc,
	extra_environ=extra_environ,
	unset_environ=cls.unset_environ,
	spinner=spinner,
	log_failed_cmd=log_failed_cmd)
	except OSError as e:
	# errno.ENOENT = no such file or directory
	# In other words, the VCS executable isn't available
	if e.errno == errno.ENOENT:
	raise BadCommand(
	'Cannot find command %r - do you have '
	'%r installed and in your '
	'PATH?' % (cls.name, cls.name))
	else:
	raise # re-raise exception if a different error occurred

This ties back to #7711, we really need to move away from calling call_subprocess in VCS code :(

[deveshks]

Hi @clod6677

This is fixed in #7969 since we now use a custom call_subprocess for invoking VCS subcommands, which doesn't contain the Running command log string.

$ pip install git+https://:${WRONG_TOKEN}@myrepo.git@${LIBS_SHA}#subdirectory=pkg/my-package
Collecting git+https://:****@#subdirectory=pkg/my-package
  Cloning https://:****@ to /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pip-req-build-togw6ylx
ERROR: Command errored out with exit status 128: git clone -q 'https://:****@' /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pip-req-build-togw6ylx Check the logs for full command output

Can you verify this as well after installing pip from the latest master and running the original command in the issue. (You can do pip install git+https://github.com/pypa/pip.git#egg=master for this)

[sbidoul]

The attempted fix for this had the interesting side effect of completely hiding all vcs errors from the user, except the exit code ? See also #9234. So we'll probably need to revert partially, and I think this particular exposed token issue has to be reported to the git project.

[pradyunsg]

(annoying "keeping discussions easier to keep track of" note) Can we create a new issue, put down all the context there, and continue the discussion on what to do there?

[sbidoul]

@pradyunsg discussion is taking place in #9234. I just wanted to cross-link here and mention that, IMO, this particular issue is not a pip issue.