[Feature]:Support for non-NVIDIA backends (e.g., Iluvatar CoreX )

[SawanoHao]

🚀 The feature, motivation and pitch

Background

I am very interested in the Unified Cache Management (UCM) project. Currently, it provides an excellent way to persist and reuse KV Cache to speed up LLM inference. In the current landscape of diverse hardware, adapting such cache management logic to domestic GPUs (like Iluvatar CoreX / 天数智芯) is becoming increasingly important.

Question

I would like to know if there are any plans or architectural considerations for supporting non-NVIDIA backends. Specifically:

1. Hardware Abstraction: Does the current implementation of UCM heavily rely on NVIDIA-specific features (e.g., CUDA VMM API, specific NVLink behaviors)?
2. Framework Dependency: Does UCM require a specific version of vLLM or other engines that are strictly tied to CUDA?
3. Porting Effort: In your opinion, what are the most critical modules that need to be rewritten or abstracted to support the Iluvatar CoreX software stack (which uses the DeepLink/CoreX SDK)?

I have access to Iluvatar CoreX hardware and would love to hear your thoughts on the feasibility of this adaptation.

Alternatives

No response

Additional context

No response

[ygwpz]

Thank you for your attention.
Naturally, UCM is built on the philosophy of an open engine ecosystem and computing power ecosystem.
Therefore, we support a variety of different inference engines as well as diversified computing power.
Currently, we have integrated Huawei Ascend, NVIDIA, Muxi and Moore Threads. Any platform can connect to UCM simply by implementing the interfaces defined in idevice.h.

we will release developer documentation to provide guidance on how to integrate computing power ecosystems into UCM later.