Skip to content

GRPO gradient accumulation steps update and DAPO support #308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Oct 29, 2025
Merged

GRPO gradient accumulation steps update and DAPO support #308

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
49b621e
Update rl_replacements.py, adjust losses for gradient accumulations a…
pluesclues Sep 29, 2025
2f4137d
Merge branch 'unslothai:main' into GRPO_gradient_accumulation_adjustm…
pluesclues Oct 1, 2025
7529e17
Update rl_replacements.py, current update
pluesclues Oct 1, 2025
e2a5e78
Update rl_replacements.py, update again
pluesclues Oct 1, 2025
df7e1af
Update rl_replacements.py, fixed naming
pluesclues Oct 1, 2025
e3fc325
Merge branch 'unslothai:main' into GRPO_gradient_accumulation_adjustm…
pluesclues Oct 1, 2025
bcfb418
Update rl_replacements.py, log importnace sampling logic
pluesclues Oct 1, 2025
6d190c5
Update rl_replacements.py, added metrics
pluesclues Oct 1, 2025
90d9a39
Update rl_replacements.py, log prob importance sampling
pluesclues Oct 1, 2025
5d2d3b2
Update rl_replacements.py handled non vllm case
pluesclues Oct 1, 2025
379e9c8
Update rl_replacements.py
pluesclues Oct 1, 2025
8370c4a
Merge branch 'unslothai:main' into GRPO_gradient_accumulation_adjustm…
pluesclues Oct 5, 2025
132bb28
Merge branch 'unslothai:main' into GRPO_gradient_accumulation_adjustm…
pluesclues Oct 20, 2025
1bdad0c
Merge branch 'unslothai:main' into GRPO_gradient_accumulation_adjustm…
pluesclues Oct 20, 2025
dba3c83
Add conditional check for sampling_per_token_logps
pluesclues Oct 20, 2025
26af751
Add textconfig check for vision model creation, for gemma 1b
pluesclues Oct 22, 2025
10afdfc
Refactor vision model condition check
pluesclues Oct 22, 2025
f140660
Merge branch 'unslothai:main' into GRPO_gradient_accumulation_adjustm…
pluesclues Oct 22, 2025
cd97d33
Merge branch 'unslothai:main' into GRPO_gradient_accumulation_adjustm…
pluesclues Oct 28, 2025
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'unslothai:main' into GRPO_gradient_accumulation_adjustm… 
…ents
  • Loading branch information
@pluesclues
pluesclues authored Oct 1, 2025
commit e3fc325a0fb8910b61b3f2c25511adb2ac08e25c

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.