Skip to content

Fetch from nvidia Megatron-LM #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
RaymondLi0 wants to merge 6,387 commits into
base: load-iter
Choose a base branch
from
Open

Fetch from nvidia Megatron-LM #5

RaymondLi0 wants to merge 6,387 commits into from

Conversation

@RaymondLi0
Copy link

@RaymondLi0 RaymondLi0 commented Aug 3, 2022

No description provided.

yobibyte and others added 30 commits December 8, 2025 19:13
@yobibyte
Move model configs to github. (#2587)
e2cf81c
@iuyo5678 @TencentXiaowei
fix: Assign tokenizer to Encoder.tokenizer in legacy mode (#2498)
8d18afd 
Co-authored-by: drakezhang <[email protected]>
@wplf @yaox12 @ko3n1g
Delete redundant import in yaml_arguments.py (#2139)
c21bf6e 
Co-authored-by: Xin Yao <[email protected]>
Co-authored-by: oliver könig <[email protected]>
@CodersAcademy006 @yaox12
Fix world size mismatch causing distributed init deadlock (Issue #2458)…
dfb78dc 
… (#2571)

Co-authored-by: Xin Yao <[email protected]>
@tdene
Improve performance of request_metadata logic (#2378)
4a4f23a
@JungHoyoun @yaox12
Fix broken Table of Contents links in README.md (#1954)
7b11553 
Co-authored-by: Xin Yao <[email protected]>
@gautham-kollu @yaox12
Add minor log update (#2080)
9fba363 
Signed-off-by: gkollu <[email protected]>
Co-authored-by: Xin Yao <[email protected]>
@janbernloehr @yaox12
Fix link to NeMo performance summary documentation (#2190)
bd32927 
Co-authored-by: Xin Yao <[email protected]>
@wdykas
Prep for refit (#2590)
ef12f16 
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
@pablo-garay
feat: API compat: ignore ParameterMovedBreakage for __init__ methods (#…
a2aafe3 
…2595)

Signed-off-by: Pablo Garay <[email protected]>
@ko3n1g
Revert "Prep for refit (#2590)"
5c54bb6 
This reverts commit ef12f16.
@vignesh1507 @yaox12
Fix NameError in pretrain_retro.py (add import_module), remove unused… (
be4baad 
#2084)

Co-authored-by: Xin Yao <[email protected]>
@Autumn1998
Use the latest Hybrid-EP (#2479)
d2b500f
@BoxiangW
QK logits clipping (non-split version) (#1929)
dd54609 
Signed-off-by: Boxiang Wang <[email protected]>
@dimapihtar
update checkpointing documentation (#2606)
93da800 
Signed-off-by: dimapihtar <[email protected]>
@maanug-nv @ericharper
[training migration] add training config dataclass and arg generation…
72230d2 
… utility (#2306)

Signed-off-by: Maanu Grover <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
@tdene
Check skip_prompt_log_probs in add_request (#2593)
d9c911a
@wdykas @ko3n1g
Refit prep 2 (#2608)
587a0ff 
Signed-off-by: oliver könig <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: William Dykas <[email protected]>
Co-authored-by: oliver könig <[email protected]>
@wdykas
Batch Invariance (#2308)
d2bd9fa
@dimapihtar
Remove flattened_range code paths for distributed optimizer checkpoin…
5ab481c 
…ting (#2126)

Signed-off-by: dimapihtar <[email protected]>
@dimapihtar
update commit (#2631)
5a24ff3 
Signed-off-by: dimapihtar <[email protected]>
@ko3n1g
tests: Disable grads test
f67b7bd 
Signed-off-by: oliver könig <[email protected]>
@AAnoosheh
Create separate teacher Layer Spec in KD mode (#2429)
44899aa 
Signed-off-by: Asha Anoosheh <[email protected]>
@lmcafee-nvidia
Dynamic context | Re-add max_requests arg. (#2488)
6b186c1
@lmcafee-nvidia
Inference | Fix entangled request generations. (#2584)
2ab9253
@dimapihtar
fix gpt3_mcore_reruns_resume_check_grads (#2646)
3a9f086 
Signed-off-by: dimapihtar <[email protected]>
@cuichenx @ericharper
Nemotron nano v2 vl changes for Megatron Bridge (#2078)
f5daa16 
Signed-off-by: Chen Cui <[email protected]>
Co-authored-by: Eric Harper <[email protected]>
@Phlip79
[docs] Migrate docs to new Sphinx (#2489)
4f700f7
@tdene
Add option to only log inference every N steps (#2637)
1c6f6eb
@Phlip79
[docs] Use autodoc2 and remove automodule (#2542)
0a59bea
ko3n1g and others added 30 commits January 16, 2026 15:53
@ko3n1g
ci(fix): CI_COMMIT_BRANCH on forks (#2982)
b96cadb 
Signed-off-by: oliver könig <[email protected]>
@yuzhongw-nvidia @ko3n1g
[main] feat(moe): Support gated delta net for Qwen3-Next (1/4) (#1989)
20d66d5 
Signed-off-by: oliver könig <[email protected]>
Co-authored-by: oliver könig <[email protected]>
@ko3n1g
ci: Add more gb200 nightly tests (#2981)
92f052b 
Signed-off-by: oliver könig <[email protected]>
@ko3n1g
ci(hotfix): gpt3_mcore_te_tp1_pp2_resume_torch_dist_rope_embeddings
4ebd1ad 
Signed-off-by: oliver könig <[email protected]>
@ko3n1g
ci(hotfix): move test to nightly
b50da25 
Signed-off-by: oliver könig <[email protected]>
@yuzhongw-nvidia @jstjohn @deepakn94
[main] feat(moe): Support apply wd to qk layernorm for Qwen3-Next (4/…
622a06a 
…4) (#2753)

Signed-off-by: John St. John <[email protected]>
Co-authored-by: John St. John <[email protected]>
Co-authored-by: Deepak Narayanan <[email protected]>
@asolergi-nv @ko3n1g
Set token_dtype_code init value in GPTDatasetConfig to fix CI (#2912
b305422 
)

Co-authored-by: oliver könig <[email protected]>
@hexinw-nvidia @rhewett-nv @chtruong814
Re-submit "Various fixes to in-job restarter and better time accounti…
c40b6ea 
…ng of startup operations" (#2954)

Co-authored-by: Russell Hewett <[email protected]>
Co-authored-by: Charlie Truong <[email protected]>
@Phlip79
Use slack-sdk in a different manner (#2950)
100e11e
@lmcafee-nvidia
Inference | Move assert active_request_count > 0. (#2958)
b705148
@parthmannan @xiaoyao0115 @kunlunl
Hybrid Context Parallel Feature (#2282)
98d8c56 
Signed-off-by: tailaim <[email protected]>
Signed-off-by: Parth Mannan <[email protected]>
Co-authored-by: Mcore Bot <[email protected]>
Co-authored-by: tailaim <[email protected]>
Co-authored-by: kunlunl <[email protected]>
Co-authored-by: Kunlun Li <[email protected]>
@yuzhongw-nvidia
[main] ci(moe): Add --apply-wd-to-qk-layernorm flag to the gdn test…
829e461 
… case (#2995)
@ko3n1g
ci: Disable step time on `gpt3_moe_mcore_te_tp2_pp2_ep4_etp1_no_mtp_n… (
a191791 
#2991)

Signed-off-by: oliver könig <[email protected]>
@ko3n1g
ci: Fix workflows on main (#2990)
b49d810 
Signed-off-by: oliver könig <[email protected]>
@shjwudp @cspades
Make Megatron-FSDP torch.compile compatible (#2425)
35129e7 
Co-authored-by: Cory Ye <[email protected]>
@cspades
[Megatron-FSDP] Test FP8 activations + parameter sharding with Megatr…
517dfd4 
…on-FSDP fully-shard. Update README. (#2894)

Signed-off-by: Cory Ye <[email protected]>
@ko3n1g
chore: Escape special chars (#3014)
2f3fa0f 
Signed-off-by: oliver könig <[email protected]>
@deepakn94
Improve memory logging (#2839)
82ea022 
Signed-off-by: Deepak Narayanan <[email protected]>
@github-actions
chore: rotate oncall schedule
bcdd405
@santhnm2
Add a wrapper function for FA3 _flash_attn_forward call (#2933)
a3615d7 
Signed-off-by: Keshav Santhanam <[email protected]>
@ko3n1g
ci(hotfix): Fix unit tests on main
a1ba844 
Signed-off-by: oliver könig <[email protected]>
@ko3n1g
ci(hotfix): Tests on main
e4b18f7 
Signed-off-by: oliver könig <[email protected]>
@ko3n1g
chore: Set umask 0002 (#3027)
ba876ef 
Signed-off-by: oliver könig <[email protected]>
@mathemakitten
Make attn mask inversion in-place instead of allocating it again (#3019)
28c7221
@cspades
[Megatron-FSDP] Fix incorrect gradient scaling target. (#3023)
ba456fd 
Signed-off-by: Cory Ye <[email protected]>
@santhnm2
Ensure that last prefill chunk is handled correctly by Mamba models (#…
096dbeb 
…2897)

Signed-off-by: Keshav Santhanam <[email protected]>
@nschank
Replaces ModuleSpec with Protocols for some of the inputs to SelfAtte…
90e685b 
…ntion/CrossAttention (#2761)
@Phlip79
Update oncall schedule (#3017)
dd72aee
@jiemingz @buptzyb
Various CUDA graph improvements on capture time, replay time, memory …
8baf014 
…footprint (#2572)

Signed-off-by: Jimmy Zhang <[email protected]>
Signed-off-by: Jieming Zhang <[email protected]>
Co-authored-by: Robin Zhang <[email protected]>
@github-actions
chore: rotate oncall schedule
bbbedbb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.