Skip to content

[BUG] nb/Kaggle-Llama3.2_(1B_and_3B)-Conversational.ipynb the tokenizer.chat_template is 3.1 #140

@wenbindu

Description

@wenbindu

I use the nb/Kaggle-Llama3.2_(1B_and_3B)-Conversational.ipynb with custome dataset.

Then I got error:
['<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 26 July 2024\n\n\n你是一个助理,对用户的问题就行准确,和精确性回答\n<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n你知道中国上海吗?介绍一下那里的甜品<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n上海的甜品有很多,包括小笼包、汤圆、芝麻糕、芝士蛋糕等等。其中小笼包和汤圆最为著名。小笼包是一种由面粉和水制成的面饼,包裹着糖浆和肉馅。汤圆是一种由面粉、水、糖和发酵酵母制成的面饼,包裹着糖浆和肉馅。芝麻糕是一种由芝麻、面粉和水制成的糕点,包裹着糖浆和肉馅。芝士蛋糕是一种由芝士、面粉和水制成的糕点,包裹着糖浆和肉馅。这些甜品都是中国上海的特色美食,非常受欢迎。<|reserved_special_token_166|><|reserved_special_token_198|>system<|reserved_special_token_10|> \n你知道中国上海吗?介绍一下那里的甜品。 \n上海的甜品有很多,包括小笼包、汤圆、芝麻糕、芝士蛋糕等等。其中小笼包和汤圆最为著']

I think the chat_template is confused.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions