Skip to content

Conversation

@JarbasAl
Copy link
Contributor

was failing to distill models that didn't include a model card in huggingface

fdemelo/xlm-roberta-ovos-intent-classifier
Some weights of XLMRobertaModel were not initialized from the model checkpoint at fdemelo/xlm-roberta-ovos-intent-classifier and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Encoding tokens: 100%|██████████| 249999/249999 [03:34<00:00, 1165.90 tokens/s]
Traceback (most recent call last):
  File "/home/miro/PycharmProjects/NLP/distilintent/distill.py", line 124, in <module>
    m2v_model = distill(model_name=m, pca_dims=256)
  File "/home/miro/PycharmProjects/model2vec/model2vec/distill/distillation.py", line 239, in distill
    return distill_from_model(
        model=model,
    ...<8 lines>...
        use_subword=use_subword,
    )
  File "/home/miro/PycharmProjects/model2vec/model2vec/distill/distillation.py", line 139, in distill_from_model
    language = info.cardData.get("language", None)
               ^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'
@stephantul
Copy link
Contributor

@JarbasAl Hey, thanks for the PR. I think this was fixed on main already. See the except block right below your fix, it catches any exception because there were just too many ways in which this simple call could fail, very annoying.

@stephantul stephantul merged commit 86d5378 into MinishLab:main May 25, 2025
@stephantul
Copy link
Contributor

@JarbasAl thanks again! It's a bit cleaner to not throw an exception.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants