Kokkos: cuda architectures allowed by Spack are not all supported by Kokkos

[mpokorny]

The list of CUDA architectures at

spack-packages/repos/spack_repo/builtin/packages/kokkos/package.py

Line 183 in 53d3158

spack_cuda_arch_map = {

does not include "90a", which is allowed by Spack (through CudaPackage, I guess). I noticed that there are other discrepancies, too (all others with a letter suffix, for example.)

[JBludau]

yeah, we should add a check/mapping for this

[tpadioleau]

@mpokorny thanks for letting us know, out of curiosity, how did you figure it out ? What kind of error did you encounter ?

[mpokorny]

@tpadioleau , I tried using cuda_arch=90a as an all: package requirement in a Spack environment, so that the variant would apply to all packages in the environment that use CUDA. I don't recall the details, but I believe environment concretization was successful, and the Kokkos build failed. I believe that it failed during Spack's processing of the Kokkos package file when I ran spack install. It might have been near line 410.

[rbberger]

Kokkos currently doesn't have logic for cuda_arch=90a or cuda_arch=100a in its CMake. While there are special implementation CMake flags for unified memory within Kokkos itself, those aren't exposed via the Spack package right now (and it's unclear if they should be). Your best bet at the moment is to do cuda_arch=90.

[mpokorny]

Yes, that's what I did. My point is simply that there's a mismatch between what Spack tells you and what Kokkos will actually allow for values of cuda_arch.
I suppose this does raise the question of what the Kokkos Spack package should do:

1. silently convert, say, 90a to 90
2. prune the unsupported architectures from what's provided by CudaPackage (is this even possible?)
3. fail when building (as happens now)

[rbberger]

2. is possible by overriding the cuda_arch variant and filtering allowed values.