Error building scie PEX for Python 2.7

[cerha]

Hello,

I am trying to build a PEX for Python 2.7. The reason is that I need to communicate with a service using RPyC, but RPyC is incompatible between Python 2 and 3. Some instances of the service still run on Python 2, so I need to build Python 2 and Python 3 versions of the client software. It would be very practical to build this client as a SCIE PEX to avoid dependency on two Python versions on target machines.

Building scie PEX for Python 3.11 works fine, but if I switch to a Python 2.7 virtual environment (using pyenv), I get the following error on macOS:

pex -r requirements-mac-py2.txt --scie eager -o p2go-py2-macos 
You selected `--scie eager`, but none of the selected targets have compatible interpreters that can be embedded to form a scie:
cp27-cp27m-macosx_14_0_arm64 interpreter at /Users/cerha/.pyenv/versions/2.7.18/envs/p2go-py2/bin/python

Similarly on Linux:

pex -r requirements-linux-py2.txt --scie eager -o p2go-py2-linux
You selected `--scie eager`, but none of the selected targets have compatible interpreters that can be embedded to form a scie:
cp27-cp27mu-manylinux_2_28_x86_64 interpreter at /home/cerha/.pyenv/versions/2.7.18/envs/p2go-py2/bin/python

Can someone, please, explain me this error message. Am I doing something wrong or is Python 2.7 not supported by scie PEX?

Thank you very much for any advice and for working on this great project.

Best regards

Tomas Cerha

[jsirois]

CPython 2.7 is not supported, although PyPy 2.7 is.

These PEX scies are built using the science tool and it uses https://github.com/indygreg/python-build-standalone for portable CPython distributions and https://pypy.org/ (for PyPy).

So the crux here is finding a portable CPython 2.7 distribution (a pre-built Python tarball that can be unpacked on any Linux or Mac machine and automatically work). There is none that I'm aware of.

[jsirois]

If your project can run under PyPy, then using a pyenv pypy2.7 venv to build the PEX scie from should work.

[jsirois]

For example:

# CPython 2.7 is not supported for `--scie` PEX builds:
:; pex --python python2.7 cowsay==5.0 -c cowsay --scie eager -o cowsay.pex
You selected `--scie eager`, but none of the selected targets have compatible interpreters that can be embedded to form a scie:
cp27-cp27mu-manylinux_2_40_x86_64 interpreter at /home/jsirois/.pyenv/versions/2.7.18/bin/python2.7

# But PyPy 2.7 is:
:; pex --python pypy2.7 cowsay==5.0 -c cowsay --scie eager -o cowsay.pex
:; file cowsay.pex cowsay
cowsay.pex: Zip archive data, made by v2.0 UNIX, extract using at least v2.0, last modified, last modified Sun, Jan 01 1980 00:00:00, uncompressed size 0, method=deflate
cowsay:     ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), static-pie linked, BuildID[sha1]=f1f01ca2ad165fed27f8304d4b2fad02dcacdffe, stripped
# The standard PEX requires PyPy 2.7 is on the PATH:
:; PATH= ./cowsay.pex Moo!
/usr/bin/env: ‘pypy2.7’: No such file or directory
# But the PEX scie does not since it contains a PyPy 2.7 distribution:
:; PATH= ./cowsay Moo!
  ____
| Moo! |
  ====
    \
     \
       ^__^
       (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||
:; SCIE=inspect ./cowsay
{
  "scie": {
    "lift": {
      "name": "cowsay",
      "files": [
        {
          "name": "pypy2.7-v7.3.17-linux64.tar.bz2",
          "key": "python-distribution",
          "size": 30988704,
          "hash": "9f3497f87b3372d17e447369e0016a4bec99a6b4d2a59aba774a25bfe4353474",
          "type": "tar.bz2"
        },
        {
          "name": "configure-binding.py",
          "size": 1828,
          "hash": "0d6b095aa231552b51fef8acc87b31a1a6da1dbd4b733ba341eb8a46fdf22255",
          "type": "blob"
        },
        {
          "name": "pex",
          "size": 741582,
          "hash": "767db5ffde5e0774024eac69f896e0db91f523a38462f9a5be54f533c56d1dd8",
          "type": "blob",
          "executable": true
        }
      ],
      "boot": {
        "commands": {
          "": {
            "exe": "{scie.bindings.configure:PYTHON}",
            "args": [
              "{scie.bindings.configure:PEX}"
            ],
            "env": {
              "=PEX_VENV": null
            }
          }
        },
        "bindings": {
          "configure": {
            "exe": "{python-distribution}/pypy2.7-v7.3.17-linux64/bin/pypy",
            "args": [
              "{pex}",
              "{configure-binding.py}",
              "--installed-pex-dir",
              "{scie.bindings}/pex_root/unzipped_pexes/0/ebb8157cf529705344e5faa90d3e3e049849c55c"
            ],
            "env": {
              "=PATH": null,
              "=PEX_EMIT_WARNINGS": "0",
              "=PEX_INTERPRETER": "1",
              "=PEX_ROOT": "{scie.bindings}/pex_root",
              "PEX_.*": null,
              "PYTHON.*": null
            }
          }
        }
      }
    },
    "jump": {
      "size": 1811752,
      "version": "0.14.0"
    }
  }
}

[cerha]

Thank you very much for your help. Creating a new pyenv virtualenv using pypy2.7-7.3.17 and running pex inside it helped in my case. This is awesome and will greatly simplify distribution of our client software for Linux and macOS. If only was that as simple for Windows... Nevertheless, thanks once again. Best regards, Tomas.

[jsirois]

You're welcome. Note that if the client app will be run more than one time, you should build with --venv if you're not already. You'll find the noop latency go down for runs 2+.

As for Windows, I will add support here at some point, but Windows is a foreign beast for me and it is a pain in the butt for Pex since it doesn't always support symlinks and it has tricky gymnastics to handle long path names, both of which are used extensively in the Pex cache to reduce disk usage.