Skip to content

Conversation

@hhaAndroid
Copy link
Collaborator

@hhaAndroid hhaAndroid commented Dec 30, 2025

         dataset_config = []
        _data_cfg = {"dataset": DatasetConfig(name='apach',
                                              anno_path=ALPACA_PATH),
                     "tokenize_fn": OpenaiTokenizeFunctionConfig(
                         chat_template='qwen3',
                         max_length=32768
                     )
                     }
        dataset_config.append(_data_cfg)

        sft_dataloader_cfg = DataloaderConfig(
            dataset_config_list=dataset_config,
            pack_max_length=32768,
            pack_to_max_length=True,
            num_workers=0,
        )
        sft_global_batch_size = 8
        loss_reduction = "square"
        sft_loss_cfg = CELossConfig(mode="chunk", chunk_size=1024, loss_reduction=loss_reduction)

        worker_cfg: WorkerConfig = WorkerConfig(
            sft_dataloader_cfg=sft_dataloader_cfg,
            sft_global_batch_size=sft_global_batch_size,
            sft_loss_cfg=sft_loss_cfg,
            seed=42,
            ...
        )
@hhaAndroid hhaAndroid changed the title add sft trainging with RL Dec 31, 2025
@hhaAndroid hhaAndroid merged commit 8dae899 into InternLM:main Dec 31, 2025
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants