Question about pointer arrays for grouped GEMM in cublasLtMatmul

[Shmilkky]

I am trying to understand how grouped GEMM is represented in cuBLASLt.

My confusion is the following:

For grouped GEMM, each operation in the group should have its own matrix pointers A[i], B[i], and C[i].
But from the cublasLtMatmul function signature, A/B/C seem to be single pointers rather than pointer-to-pointer arguments.

Because of this, I am unsure which of the following is correct:

1. The API expects pointer arrays for A/B/C.
   If yes, are these arrays located in host memory or device memory?
2. The API expects a single pointer, and grouped GEMM is expressed through offsets or strides.
3. There is another grouped GEMM-specific mechanism in cuBLASLt that I may be missing.

[rsdubtso]

Hi @Shmilkky,

It is (1) and the arrays must be located in device memory to avoid additional synchronization. We are working on additional samples to make it easier to understand how to use this API. I'll update this thread once they are posted.

[Shmilkky]

Hi @Shmilkky,

It is (1) and the arrays must be located in device memory to avoid additional synchronization. We are working on additional samples to make it easier to understand how to use this API. I'll update this thread once they are posted.

Thank you for the clarification. This is very helpful. I look forward to your update and the additional samples.