Add vLLM EP launch script and SGLang scripts for Intel irdma NICs#823
Conversation
1eeb894 to
58884dc
Compare
|
@manojgop Thank you! Do you want to share the results or performance in this PR or README.md? |
- Add launch_vllm_intel_nic.sh for multi-node vLLM Expert Parallel with Intel irdma RoCE NICs (Qwen3-30B-A3B-FP8) - Add common_env_intel_nic.sh with shared Intel NIC env config - Add Qwen3-30B_uep_intel_nic.sh for SGLang EP on Intel NICs - Set LOCAL_WORLD_SIZE to fix cross-node CUDA IPC handle error Signed-off-by: Manoj Gopalakrishnan <[email protected]>
58884dc to
db9a0a5
Compare
I'll raise one more PR for P-D disaggregation. Shall I update the the README.md with that PR ? This script is similar to existing script except that environment variables are Intel NIC specific. Don't want to share performance results now since this is tested with a smaller setup with only 1 GPU per node. |
I see. sounds good. |
Description
Please include a summary of the changes and the related issue.
Fixes # (issue)
Type of Change
How Has This Been Tested?
Include any tests here.
Checklist
format.shto follow the style guidelines.build.shto verify compilation.