feat: add onerec in supported model docs and align rec utility style.#1055
feat: add onerec in supported model docs and align rec utility style.#1055DragonFive wants to merge 2 commits intojd-opensource:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates documentation for supported Rec models, enables a feature flag, and aligns some code with project conventions. The documentation and flag changes appear correct. However, replacing std::runtime_error with LOG(FATAL) in xllm/core/util/utils.cpp introduces a critical issue. While this aligns with some of the file's style, it breaks the error handling in xllm/core/distributed_runtime/rec_master.cpp, which uses a try-catch block to handle these errors gracefully. The new implementation will cause the process to terminate on invalid input, which is a significant regression for a server application. I've added comments with suggestions to revert these specific changes to use throw again.
a0daea5
fe21af2 to
a0daea5
Compare
a0daea5 to
4d71462
Compare
Summary
This PR fixes the Rec support documentation and aligns a few Rec-related code paths with the current project style. This PR will merge after #1051 is merged.
Changes
Qwen2.5andQwen3from theRecsection in:docs/zh/supported_models.mddocs/en/supported_models.mdOneRecas the only documented Rec model entryFLAGS_enable_rec_prefill_only = true;inxllm/c_api/internal/rec.cpptorch::Dtypewithtorch::ScalarTypeinxllm/core/util/utils.cppstd::runtime_errorwithLOG(FATAL)in the same Rec tensor conversion path to match existing code styleWhy
Qwen2.5andQwen3do not support ILU in the Rec path, so the previous documentation was incorrectrec.cppflag should be enabled instead of left commented oututils.cppshould follow the existing repository convention for PyTorch scalar type naming and fatal error handlingValidation
Recsection in both Chinese and English support docsxllm/c_api/internal/rec.cppnow enablesFLAGS_enable_rec_prefill_onlyxllm/core/util/utils.cppno longer usestorch::Dtypeorstd::runtime_errorin the updated pathNotes