BUG: -P nonexistentmodule should fail

[bsipocz]

python setup.py test -P nonexistent used to raise a Valuerror:

python setup.py test -P nonexistent/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/setuptools/distutils_patch.py:25: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
  warnings.warn(
running test
INFO: installing to temporary directory: /private/var/folders/9s/070g0pd502q70k3gffpxv8km0000gq/T/astroquery-test-_jo10zsr [astropy.tests.command]
no previously-included directories found matching 'build'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'docs/api'
no previously-included directories found matching 'astropy_helpers/build'
warning: no previously-included files matching '*.o' found anywhere in distribution
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 259, in test
    return runner.run_tests(**kwargs)
  File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 604, in run_tests
    return super().run_tests(**kwargs)
  File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 216, in run_tests
    args = self._generate_args(**kwargs)
  File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 145, in _generate_args
    result = func(keywords[keyword], keywords)
  File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 41, in keyword
    return f(*args, **kwargs)
  File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 351, in package
    self.package_path = self.packages_path(package, self.base_path,
  File "/Users/bsipocz/.pyenv/versions/3.9.1/lib/python3.9/site-packages/astropy/tests/runner.py", line 316, in packages_path
    raise ValueError(error.format(**info))
ValueError: package to test is not found: nonexistent (at path /private/var/folders/9s/070g0pd502q70k3gffpxv8km0000gq/T/astroquery-test-_jo10zsr/lib/python3.9/site-packages/astroquery/nonexistent).

but unfortunately it's not the case anymore with pytest:

pytest -P nonexistent
================================================= test session starts ==================================================
platform darwin -- Python 3.9.1, pytest-7.1.2, pluggy-0.13.1
WARNING: AstropyDeprecationWarning: The ignore_warnings class is deprecated and may be removed in a future version.
        Use pytest.mark.filterwarnings instead. [pytest_astropy_header.display]
WARNING: AstropyDeprecationWarning: The catch_warnings class is deprecated and may be removed in a future version.
        Use pytest.warns instead. [astropy.tests.helper]

Running tests with astroquery version 0.4.7.dev8203_testrun.
Running tests in astroquery docs.

Date: 2022-11-09T13:29:18

Platform: macOS-10.16-x86_64-i386-64bit

Executable: /Users/bsipocz/.pyenv/versions/3.9.1/bin/python3.9

Full Python Version: 
3.9.1 (default, Jan 11 2021, 15:50:22) 
[Clang 10.0.0 (clang-1000.11.45.5)]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Package versions: 
Numpy: 1.23.4
Scipy: 1.5.4
Matplotlib: 3.5.3
h5py: 3.1.0
Pandas: 1.4.4
Astropy: 5.1.1
regions: 0.6
pyVO: 1.5.dev12+gbacb8ab
mocpy: 0.10.1
astropy-healpix: 0.7
vamdclib: not available
astropy-helpers: 4.0.1

Using Astropy options: remote_data: none.

rootdir: /Users/bsipocz/munka/devel/worktrees/astroquery/testings, configfile: setup.cfg, testpaths: astroquery, docs
plugins: anyio-3.6.1, arraydiff-0.3, requests-mock-1.9.3, rerunfailures-10.2, mock-3.7.0, nbval-0.9.6, dependency-0.5.1, astropy-0.10.0, hypothesis-6.3.1, filter-subpackage-0.1.1, openfiles-0.5.0, doctestplus-0.11.1, astropy-header-0.1.2, cov-3.0.0, remotedata-0.3.4.dev11+g319a61b
collected 0 items                                                                                                      

================================================ no tests ran in 1.70s =================================================