IOError expanding sdist with unicode filenames

[jaraco]

• Pip version: Unknown
• Python version: 2.7
• Operating System: Windows

Description:

Originally reported here, it seems that when attempting to install a package that contains unicode filenames, it will fail with an IOError:

D:\pydevenv\win32>pip install -U CherryPy
Collecting CherryPy
  Using cached CherryPy-7.1.0.tar.gz
Exception:
Traceback (most recent call last):
  File "d:\devtool\py27x86\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "d:\devtool\py27x86\lib\site-packages\pip\commands\install.py", line 299, in run
    requirement_set.prepare_files(finder)
  File "d:\devtool\py27x86\lib\site-packages\pip\req\req_set.py", line 370, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "d:\devtool\py27x86\lib\site-packages\pip\req\req_set.py", line 587, in _prepare_file
    session=self.session, hashes=hashes)
  File "d:\devtool\py27x86\lib\site-packages\pip\download.py", line 810, in unpack_url
    hashes=hashes
  File "d:\devtool\py27x86\lib\site-packages\pip\download.py", line 653, in unpack_http_url
    unpack_file(from_path, location, content_type, link)
  File "d:\devtool\py27x86\lib\site-packages\pip\utils\__init__.py", line 605, in unpack_file
    untar_file(filename, location)
  File "d:\devtool\py27x86\lib\site-packages\pip\utils\__init__.py", line 577, in untar_file
    with open(path, 'wb') as destfp:
IOError: [Errno 22] invalid mode ('wb') or filename: 'd:\\users\\me\\appdata\\local\\temp\\pip-build-ef4yey\\CherryPy\\cherrypy/test/static/\xd0\xa1\xd0\xbb\xd0\xb0\xd0\xb2\xd0\xb0 \xd0\xa3\xd0\xba\xd1\x80\xd0\xb0\xd1\x97\xd0\xbd\xd1\x96.html'

Possibly related to #2501 or #1989 or #3038 or #1698 or #1473 or #2222, although none of those mention an IOError.

I'll see if I can replicate it with a specific pip version.

[jaraco]

I'm not able to replicate the issue with pip 8.1.2, so I suspect the user was using an older version of pip:

C:\Users\jaraco> C:\Users\jaraco\.virtualenvs\issue3894\Scripts\python -m pip --version
pip 8.1.2 from C:\Users\jaraco\.virtualenvs\issue3894\lib\site-packages (python 2.7)
C:\Users\jaraco> C:\Users\jaraco\.virtualenvs\issue3894\Scripts\python -m pip install cherrypy==7.1.0
Collecting cherrypy==7.1.0
  Using cached CherryPy-7.1.0.tar.gz
Collecting six (from cherrypy==7.1.0)
  Downloading six-1.10.0-py2.py3-none-any.whl
Building wheels for collected packages: cherrypy
  Running setup.py bdist_wheel for cherrypy ... done
  Stored in directory: C:\Users\jaraco\AppData\Local\pip\Cache\wheels\48\bb\36\837f964b8aa03a1a7ef0cfb47f42dbcc73b2fe
e745276d2c10
Successfully built cherrypy
Installing collected packages: six, cherrypy
Successfully installed cherrypy-7.1.0 six-1.10.0
C:\Users\jaraco> C:\Users\jaraco\.virtualenvs\issue3894\Scripts\python -m pip uninstall -y cherrypy
Uninstalling CherryPy-7.1.0:
  Successfully uninstalled CherryPy-7.1.0
C:\Users\jaraco> C:\Users\jaraco\.virtualenvs\issue3894\Scripts\python -m pip install --no-use-wheel cherrypy==7.1.0
DEPRECATION: --no-use-wheel is deprecated and will be removed in the future.  Please use --no-binary :all: instead.
Collecting cherrypy==7.1.0
  Using cached CherryPy-7.1.0.tar.gz
Requirement already satisfied (use --upgrade to upgrade): six in c:\users\jaraco\.virtualenvs\issue3894\lib\site-package
s (from cherrypy==7.1.0)
Skipping bdist_wheel for cherrypy, due to binaries being disabled for it.
Installing collected packages: cherrypy
  Running setup.py install for cherrypy ... done
Successfully installed cherrypy-7.1.0

[marunguy]

I already reported this issue to this site.
#3882

I use pip 8.1.2 version.
This issue is tested on native python environment not virtualenv.
And this issue is reproduced in many PCs in my company and home.

D:\pydevenv\win32>ver

Microsoft Windows [Version 10.0.10586]

D:\pydevenv\win32>python
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getdefaultencoding()
'ascii'
>>> sys.getfilesystemencoding()
'mbcs'
>>> exit()

D:\pydevenv\win32>pip --version
pip 8.1.2 from d:\devtool\py27x86\lib\site-packages (python 2.7)

D:\pydevenv\win32>pip install -U CherryPy
Collecting CherryPy
  Using cached CherryPy-7.1.0.tar.gz
Exception:
Traceback (most recent call last):
  File "d:\devtool\py27x86\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "d:\devtool\py27x86\lib\site-packages\pip\commands\install.py", line 299, in run
    requirement_set.prepare_files(finder)
  File "d:\devtool\py27x86\lib\site-packages\pip\req\req_set.py", line 370, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "d:\devtool\py27x86\lib\site-packages\pip\req\req_set.py", line 587, in _prepare_file
    session=self.session, hashes=hashes)
  File "d:\devtool\py27x86\lib\site-packages\pip\download.py", line 810, in unpack_url
    hashes=hashes
  File "d:\devtool\py27x86\lib\site-packages\pip\download.py", line 653, in unpack_http_url
    unpack_file(from_path, location, content_type, link)
  File "d:\devtool\py27x86\lib\site-packages\pip\utils\__init__.py", line 605, in unpack_file
    untar_file(filename, location)
  File "d:\devtool\py27x86\lib\site-packages\pip\utils\__init__.py", line 577, in untar_file
    with open(path, 'wb') as destfp:
IOError: [Errno 22] invalid mode ('wb') or filename: 'd:\\users\\me\\appdata\\local\\temp\\pip-build-twlz5z\\CherryPy\\cherrypy/test/static/\xd0\xa1\xd0\xbb\xd0\xb0\xd0\xb2\xd0\xb0 \xd0\xa3\xd0\xba\xd1\x80\xd0\xb0\xd1\x97\xd0\xbd\xd1\x96.html'

D:\pydevenv\win32>pip install --no-use-wheel -U CherryPy
DEPRECATION: --no-use-wheel is deprecated and will be removed in the future.  Please use --no-binary :all: instead.
Collecting CherryPy
  Using cached CherryPy-7.1.0.tar.gz
Exception:
Traceback (most recent call last):
  File "d:\devtool\py27x86\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "d:\devtool\py27x86\lib\site-packages\pip\commands\install.py", line 299, in run
    requirement_set.prepare_files(finder)
  File "d:\devtool\py27x86\lib\site-packages\pip\req\req_set.py", line 370, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "d:\devtool\py27x86\lib\site-packages\pip\req\req_set.py", line 587, in _prepare_file
    session=self.session, hashes=hashes)
  File "d:\devtool\py27x86\lib\site-packages\pip\download.py", line 810, in unpack_url
    hashes=hashes
  File "d:\devtool\py27x86\lib\site-packages\pip\download.py", line 653, in unpack_http_url
    unpack_file(from_path, location, content_type, link)
  File "d:\devtool\py27x86\lib\site-packages\pip\utils\__init__.py", line 605, in unpack_file
    untar_file(filename, location)
  File "d:\devtool\py27x86\lib\site-packages\pip\utils\__init__.py", line 577, in untar_file
    with open(path, 'wb') as destfp:
IOError: [Errno 22] invalid mode ('wb') or filename: 'd:\\users\\me\\appdata\\local\\temp\\pip-build-0gfrbn\\CherryPy\\cherrypy/test/static/\xd0\xa1\xd0\xbb\xd0\xb0\xd0\xb2\xd0\xb0 \xd0\xa3\xd0\xba\xd1\x80\xd0\xb0\xd1\x97\xd0\xbd\xd1\x96.html'