Skip to content

[Tracking issue] Wrong loss scaling when accumulating gradient #2617

New issue
New issue
Open
16 of 18 issues completed
Open
[Tracking issue] Wrong loss scaling when accumulating gradient#2617
16 of 18 issues completed
Labels
🏋 DPORelated to DPO🏋 DPPORelated to DDPO🏋 GKDRelated to GKD🏋 GRPORelated to GRPO🏋 Iterative SFTRelated to Iterative SFT🏋 KTORelated to KTO🏋 ORPORelated to ORPO🏋 Online DPORelated to Online DPO🏋 PPORelated to PPO🏋 PRMRelated to PRM🏋 RLOORelated to RLOO🏋 RewardRelated to Reward modelling🏋 SFTRelated to SFT🏋 XPORelated to XPO🐛 bugSomething isn't working
@qgallouedec

Description

@qgallouedec
qgallouedec
opened on Jan 23, 2025

Caused by huggingface/transformers#35651 that adds a new condition for scaling the loss.
Spotted huggingface/transformers#35856.

For each one, check if there is the same issue, if so fix it.

Sub-issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    🏋 DPORelated to DPO🏋 DPPORelated to DDPO🏋 GKDRelated to GKD🏋 GRPORelated to GRPO🏋 Iterative SFTRelated to Iterative SFT🏋 KTORelated to KTO🏋 ORPORelated to ORPO🏋 Online DPORelated to Online DPO🏋 PPORelated to PPO🏋 PRMRelated to PRM🏋 RLOORelated to RLOO🏋 RewardRelated to Reward modelling🏋 SFTRelated to SFT🏋 XPORelated to XPO🐛 bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions