(unsloth-gpu) ~ ➜ /home/bl/miniconda3/envs/unsloth-gpu/bin/python /home/bl/qwenvl_dataset/t.py
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
🦥 Unsloth Zoo will now patch everything to make training faster!
==((====))== Unsloth 2025.11.1: Fast Qwen3_Vl patching. Transformers: 4.57.1.
\ /| NVIDIA GeForce RTX 4080 Laptop GPU. Num GPUs = 1. Max memory: 11.571 GB. Platform: Linux.
O^O/ _/ \ Torch: 2.8.0+cu128. CUDA: 8.9. CUDA Toolkit: 12.8. Triton: 3.4.0
\ / Bfloat16 = TRUE. FA [Xformers = 0.0.32.post2. FA2 = False]
"-____-" Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
Traceback (most recent call last):
File "/home/bl/qwenvl_dataset/t.py", line 31, in
data_collator=UnslothVisionDataCollator(model, tokenizer),
File "/home/bl/miniconda3/envs/unsloth-gpu/lib/python3.10/site-packages/unsloth_zoo/vision_utils.py", line 659, in init
raise TypeError("Unsloth: UnslothVisionDataCollator is only for image models!")
TypeError: Unsloth: UnslothVisionDataCollator is only for image models!
(unsloth-gpu) ~ ➜

2️⃣ 启用视觉微调

FastVisionModel.for_training(model) # 官方 patch
model._is_vision_model = True # ✅ 手动标记视觉模型

3️⃣ 加载数据集

dataset = load_dataset("json", data_files=DATA_PATH, split="train")

4️⃣ Trainer 直接传 Collator

trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
train_dataset=dataset,
data_collator=UnslothVisionDataCollator(model, tokenizer),
args=SFTConfig(
per_device_train_batch_size=1,
gradient_accumulation_steps=4,
learning_rate=2e-4,
logging_steps=5,
max_length=120,
remove_unused_columns=False,
dataset_text_field="",
dataset_kwargs={"skip_prepare_dataset": True},
output_dir=OUTPUT_DIR,
),
)