Error when installing a wheel on Windows

[lelit]

I'm using Python 3.6.1, zc.buildout 2.9.3 and buildout.wheel 0.2.0.

I activated the buildout.wheel extension and tried to build an application that needs binary distributions; for each downloaded wheel, buildout errors out the following traceback:

Getting distribution for 'reportlab==3.4.0'.
While:
  Installing sol.
  Getting distribution for 'reportlab==3.4.0'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\buildout.py", line 2123, in main
    getattr(buildout, command)(args)
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\buildout.py", line 796, in install
    installed_files = self[part]._call(recipe.install)
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\buildout.py", line 1553, in _call
    return f()
  File "f:\solista\eggs\zc.recipe.egg-2.0.3-py3.6.egg\zc\recipe\egg\egg.py", line 126, in install
    reqs, ws = self.working_set()
  File "f:\solista\eggs\zc.recipe.egg-2.0.3-py3.6.egg\zc\recipe\egg\egg.py", line 84, in working_set
    allow_hosts=self.allow_hosts)
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\easy_install.py", line 913, in install
    return installer.install(specs, working_set)
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\easy_install.py", line 714, in install
    for dist in self._get_dist(req, ws):
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\easy_install.py", line 567, in _get_dist
    shutil.rmtree(tmp)
  File "c:\Python36\lib\shutil.py", line 494, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\Python36\lib\shutil.py", line 389, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "c:\Python36\lib\shutil.py", line 387, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\IEUser\\AppData\\Local\\Temp\\tmprdfug7p3get_dist\\reportlab-3.4.0-cp36-cp36m-win32.whl'

Apparently the wheel got installed under eggs/reportlab-3.4.0-cp36-cp36m-win32.ovo, and indeed re-executing the buildout it goes on fetching the remaining packages, failing again on the next wheel.

[lelit]

A different failure:

Getting distribution for 'Pillow==4.1.0'.
While:
  Installing sol.
  Getting distribution for 'Pillow==4.1.0'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\buildout.py", line 2123, in main
    getattr(buildout, command)(args)
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\buildout.py", line 796, in install
    installed_files = self[part]._call(recipe.install)
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\buildout.py", line 1553, in _call
    return f()
  File "f:\solista\eggs\zc.recipe.egg-2.0.3-py3.6.egg\zc\recipe\egg\egg.py", line 126, in install
    reqs, ws = self.working_set()
  File "f:\solista\eggs\zc.recipe.egg-2.0.3-py3.6.egg\zc\recipe\egg\egg.py", line 84, in working_set
    allow_hosts=self.allow_hosts)
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\easy_install.py", line 913, in install
    return installer.install(specs, working_set)
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\easy_install.py", line 714, in install
    for dist in self._get_dist(req, ws):
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\easy_install.py", line 563, in _get_dist
    dists = [_move_to_eggs_dir_and_compile(dist, self._dest)]
  File "f:\solista\eggs\zc.buildout-2.9.3-py3.6.egg\zc\buildout\easy_install.py", line 1731, in _move_to_eggs_dir_and_compile
    assert newdist is not None, "TROUBLE: dist=%s newloc=%s"%(dist,newloc)  # newloc above is missing our dist?!
AssertionError: TROUBLE: dist=Pillow 4.1.0 newloc=f:\solista\eggs\Pillow-4.1.0-py3.6-win32.egg

NB: I changed the final assert to emit some more detail..

This time buildout actually installed an eggs/Pillow-4.1.0-py3.6-win32.egg.

[daybarr]

@lelit I have a similar problem to the second issue you report here. In my case with PyYAML. I think the root cause is the non-lowercase package name. I can reproduce a failure with the same stack trace:

While:
  Installing pyyaml.
  Getting distribution for 'pyyaml==3.10'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "c:\users\day.barr\dev\bms\src\buildout\src\zc\buildout\buildout.py", line 2123, in main
    getattr(buildout, command)(args)
  File "c:\users\day.barr\dev\bms\src\buildout\src\zc\buildout\buildout.py", line 796, in install
    installed_files = self[part]._call(recipe.install)
  File "c:\users\day.barr\dev\bms\src\buildout\src\zc\buildout\buildout.py", line 1553, in _call
    return f()
  File "c:\users\day.barr\dev\bms\eggs\zc.recipe.egg-2.0.3-py2.7.egg\zc\recipe\egg\egg.py", line 126, in install
    reqs, ws = self.working_set()
  File "c:\users\day.barr\dev\bms\eggs\zc.recipe.egg-2.0.3-py2.7.egg\zc\recipe\egg\egg.py", line 84, in working_set
    allow_hosts=self.allow_hosts)
  File "c:\users\day.barr\dev\bms\src\buildout\src\zc\buildout\easy_install.py", line 913, in install
    return installer.install(specs, working_set)
  File "c:\users\day.barr\dev\bms\src\buildout\src\zc\buildout\easy_install.py", line 665, in install
    for dist in self._get_dist(requirement, ws):
  File "c:\users\day.barr\dev\bms\src\buildout\src\zc\buildout\easy_install.py", line 563, in _get_dist
    dists = [_move_to_eggs_dir_and_compile(dist, self._dest)]
  File "c:\users\day.barr\dev\bms\src\buildout\src\zc\buildout\easy_install.py", line 1730, in _move_to_eggs_dir_and_compile
    assert newdist is not None  # newloc above is missing our dist?!
AssertionError

In my case this reproduces even when I comment out the

extensions = buildout.wheel

in my buildout.cfg (pyyaml gets installed before the thing I need wheel support for), so it looks like this is a problem in buildout itself. I am also using zc.buildout==2.9.3, but with python 2.7.13 on Windows.

I have a small change to buildout which fixes this, am going to try to make a PR for that too. Can you try this change to the zc.buildout.easy_install module and see if it works for you too?

Change this function (line 1648):

def _get_matching_dist_in_location(dist, location):
    """
    Check if `locations` contain only the one intended dist.
    Return the dist with metadata in the new location.
    """
    # Getting the dist from the environment causes the
    # distribution meta data to be read.  Cloning isn't
    # good enough.
    env = pkg_resources.Environment([location])
    dists = [ d for project_name in env for d in env[project_name] ]
    dist_infos = [ (d.project_name, d.version) for d in dists ]
    if dist_infos == [(dist.project_name, dist.version)]:
        return dists.pop()

to add a call to lower() to the project_name when comparing dist_infos like so:

    dist_infos = [ (d.project_name.lower(), d.version) for d in dists ]
    if dist_infos == [(dist.project_name.lower(), dist.version)]:

[daybarr]

That issue (discussed above) has now been reported independently to the buildout project. See buildout/buildout#385. There is some suggestion that this should perhaps be fixed in pkg_resources which is messing up the path case in the first place.

[lelit]

Thank you.

I'm sorry but it's unlikely I will be able to try your suggestion [soon enough] as I'm deeply focused on other project these days.

[grant-humphries]

@daybarr I can confirm that your changes resolve this issue for me. Are you still willing to put together a pull request?