Spacy on m1 pro chip doesn't work

[maroxtn]

When following the instructions of installing Spacy on the m1 pro chip, I got an error.
I tried to build it from source, install it in a new virtual environment but the same error persists.

How to reproduce the behaviour

pip install -U pip setuptools wheel
pip install -U 'spacy[apple]'
python -m spacy download en_core_web_sm

When running the third command, I get this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/lib/python3.9/site-packages/spacy/__init__.py", line 11, in <module>
    from thinc.api import prefer_gpu, require_gpu, require_cpu  # noqa: F401
  File "/opt/homebrew/lib/python3.9/site-packages/thinc/api.py", line 2, in <module>
    from .initializers import normal_init, uniform_init, glorot_uniform_init, zero_init
  File "/opt/homebrew/lib/python3.9/site-packages/thinc/initializers.py", line 4, in <module>
    from .backends import Ops
  File "/opt/homebrew/lib/python3.9/site-packages/thinc/backends/__init__.py", line 8, in <module>
    from .cupy_ops import CupyOps, has_cupy
  File "/opt/homebrew/lib/python3.9/site-packages/thinc/backends/cupy_ops.py", line 19, in <module>
    from .numpy_ops import NumpyOps
  File "thinc/backends/numpy_ops.pyx", line 1, in init thinc.backends.numpy_ops
ImportError: dlopen(/opt/homebrew/lib/python3.9/site-packages/cymem/cymem.cpython-39-darwin.so, 0x0002): tried: '/opt/homebrew/lib/python3.9/site-packages/cymem/cymem.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cymem.cpython-39-darwin.so' (no such file), '/usr/lib/cymem.cpython-39-darwin.so' (no such file)

Your Environment

• Operating System: MacOs 12.01
• Python Version Used: 3.9.10
• spaCy Version Used: latest
• Environment Information: m1 MacBook Pro 14 inch

[adrianeboyd]

It sounds like your python environment has accidentally ended up with a mix of x86_64 and arm64 packages. (Universal binaries can make it too easy to get things mixed up, although off the top of my head I would have expected the homebrew python and the packages in in /opt/homebrew/lib/python3.9/site-packages/ to be arm64-only.)

Double-check the platform with this command, it should be arm64:

/opt/homebrew/bin/python3.9 -c "import platform; print(platform.machine())"

Try starting from scratch in a new venv without using any previously cached wheels:

/opt/homebrew/bin/python3.9 -m venv .venv
source .venv/bin/activate
python -m pip install -U pip setuptools wheel
python -m pip install 'spacy[apple]' --no-cache-dir

Does that lead to a working python -m spacy info in this venv?

[vkm-ushur]

Hello, i tried the same set of statements and it does not work - I still get the same error "zsh: illegal hardware instruction python -m spacy info".

Can you please help?

[adrianeboyd]

Clear your pip cache with pip cache purge and try again in a fresh venv. If you're still running into errors, please open a new thread, copying and pasting text output as code blocks rather than using screenshots.

[maroxtn]

@adrianeboyd Thank you, that worked for me.

[adrianeboyd]

Thanks for reporting back! If you want, you can clear your pip cache, rerun this without --no-cache-dir, and then your installs will be faster in new environments, too, as long as you're consistently using arm64.

[jacobquintero]

Hi I think I have a similar error, when I run

python3 -m spacy download en_core_web_sm 

  File "/Users/jacobquintero/Desktop/586/NLP/Diversity/SPACY/test.py", line 1, in <module>
    import spacy
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/__init__.py", line 6, in <module>
    from .errors import setup_default_warnings
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/errors.py", line 2, in <module>
    from .compat import Literal
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/compat.py", line 38, in <module>
    from thinc.api import Optimizer  # noqa: F401
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/api.py", line 2, in <module>
    from .initializers import normal_init, uniform_init, glorot_uniform_init, zero_init
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/initializers.py", line 4, in <module>
    from .backends import Ops
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/backends/__init__.py", line 7, in <module>
    from .ops import Ops
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/backends/ops.py", line 16, in <module>
    from .cblas import CBlas
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/backends/cblas.cpython-39-darwin.so, 0x0002): tried: '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/backends/cblas.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/backends/cblas.cpython-39-darwin.so' (no such file), '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/backends/cblas.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

I have installed using pip3 install -U 'spacy[apple]' which is successful. Any idea what is going on?

[adrianeboyd]

This does sound like the same problem as the original question. Did you try pip cache purge and the steps described in #10683 (comment)?

[jacobquintero]

I installed the above properly which did not solve my issue. I tried uninstalling and reinstalling thinc using arch -arm64 pip install thinc and python3 -m spacy info works and I was able to run
python -m spacy download en_core_web_sm and it was downloaded successfully. Then I try spacy.load and I get

Traceback (most recent call last):
  File "/Users/jacobquintero/Desktop/586/NLP/Diversity/SPACY/test.py", line 10, in <module>
    nlp = spacy.load("en_core_web_sm")
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/__init__.py", line 54, in load
    return util.load_model(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/util.py", line 442, in load_model
    return load_model_from_package(name, **kwargs)  # type: ignore[arg-type]
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/util.py", line 478, in load_model_from_package
    return cls.load(vocab=vocab, disable=disable, enable=enable, exclude=exclude, config=config)  # type: ignore[attr-defined]
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/en_core_web_sm/__init__.py", line 10, in load
    return load_model_from_init_py(__file__, **overrides)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/util.py", line 659, in load_model_from_init_py
    return load_model_from_path(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/util.py", line 516, in load_model_from_path
    nlp = load_model_from_config(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/util.py", line 564, in load_model_from_config
    nlp = lang_cls.from_config(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/language.py", line 1773, in from_config
    nlp = lang_cls(vocab=vocab, create_tokenizer=create_tokenizer, meta=meta)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/spacy/language.py", line 176, in __init__
    vocab = create_vocab(self.lang, self.Defaults, vectors_name=vectors_name)
  File "spacy/vocab.pyx", line 38, in spacy.vocab.create_vocab
  File "spacy/vocab.pyx", line 81, in spacy.vocab.Vocab.__init__
  File "spacy/vectors.pyx", line 109, in spacy.vectors.Vectors.__init__
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/backends/__init__.py", line 134, in get_current_ops
    require_cpu()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/util.py", line 179, in require_cpu
    ops = get_ops("cpu")
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc/backends/__init__.py", line 95, in get_ops
    ops_by_name = {ops_cls.name: ops_cls for ops_cls in registry.ops.get_all().values()}  # type: ignore
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/catalogue/__init__.py", line 119, in get_all
    result.update(self.get_entry_points())
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/catalogue/__init__.py", line 134, in get_entry_points
    result[entry_point.name] = entry_point.load()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc_apple_ops/__init__.py", line 1, in <module>
    from .ops import AppleOps
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc_apple_ops/ops.cpython-39-darwin.so, 0x0002): tried: '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc_apple_ops/ops.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc_apple_ops/ops.cpython-39-darwin.so' (no such file), '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/thinc_apple_ops/ops.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

Is there a arm64 specific model that I have to use?

[jacobquintero]

All solved now, I had to reinstall thinc_apple_ops with arm64 arch too. Thank you for your help!

[adiboy-23]

i tried to install thinc_apple_ops using pip install thinc_apple_ops --no-binary :all: , but it gave the below error . Can someone please tell what can I do to use stacy
error.txt