Skip to content
This repository was archived by the owner on Oct 25, 2024. It is now read-only.

A beginner friendly quantize and text embeddings tutorial for XPUs #1663

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

A beginner friendly quantize and text embeddings tutorial for XPUs #1663

wants to merge 2 commits into from

Conversation

sleepingcat4
Copy link

I have added a beginner friendly tutorial to illustrate how HF text embedding models can be quantised and loaded using Intel XPUs and then use to generate embeddings. (through jupyter notebook)

I have used "BAAI/bge-m3" model from HF and used Intel extension for transformers and transformers library to quantise it on XPU.
@sleepingcat4
embedding on xpu
0cb80be 
in this notebook, I have showed how text embeddings can be generated using HF models and quantise them at the same moment on Intel XPUs

Signed-off-by: tawsif <sleeping4cat@outlook.com>
@sleepingcat4
added beginner friendly embed tutorial
9aece09 
Signed-off-by: tawsif <sleeping4cat@outlook.com>
@Zhenzhong1
Copy link
Contributor

@sleepingcat4 Looks good! Could you please provide some screenshots of jupyter notebook results in this PR?
@sleepingcat4
Copy link
Author

@Zhenzhong1 sure!

image
@sleepingcat4
Copy link
Author

Here's another screenshot (although this is from another script where I distributed 8 files on 8 Intel XPUs)

image
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
2 participants
@sleepingcat4 @Zhenzhong1