Skip to content

Commit 61254cb

Browse files
sharvil10sharvil10
authored
Merge pull request #8 from sharvils/odh_2024.0
Updated notebooks for 2024.0 releases.
2 parents 13fbb71 + 132ed9c commit 61254cb

File tree

6 files changed

+277
-140
lines changed

6 files changed

+277
-140
lines changed

‎Intel_Extension_for_PyTorch_GettingStarted_and_AutoMixedPrecision_Sample/Intel_Extension_for_PyTorch_Getting_Started_and_AutoMixedPrecision.ipynb‎

Lines changed: 127 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -8,35 +8,44 @@
88
"outputs": [],
99
"source": [
1010
"# =============================================================\n",
11-
"# Copyright © 2020 Intel Corporation\n",
11+
"# Copyright © 2024 Intel Corporation\n",
1212
"# \n",
1313
"# SPDX-License-Identifier: MIT\n",
1414
"# ============================================================="
1515
]
1616
},
17+
{
18+
"cell_type": "markdown",
19+
"id": "b35a82c9-42b5-4d51-9c3f-ccffe6f8fdcd",
20+
"metadata": {},
21+
"source": [
22+
"### NOTE: Before starting this notebook, please pick a jupyter kernel from dropdown from `pytorch-cpu` or `pytorch-gpu`. Choose the kernel depending on the hardware available to you. The `pytorch-gpu` kernel will only work on Intel® Flex/Max GPUs."
23+
]
24+
},
1725
{
1826
"cell_type": "markdown",
1927
"id": "d26d6b97-e1f5-4dc8-926d-b3ccb7a141e6",
2028
"metadata": {},
2129
"source": [
22-
"# `Intel® Extension for PyTorch* Getting Started and AutoMixedPrecision Feature` Sample\n",
30+
"# `Intel® Extension for PyTorch* Getting Started and Features` Sample\n",
2331
"\n",
2432
"\n",
2533
"\n",
26-
"## About Intel Extension for PyTorch\n",
34+
"## About Intel® Extension for PyTorch\n",
2735
"\n",
28-
"PyTorch* is a very popular framework for deep learning, while also compute-heavy package demanding perfromance optimizations. Intel and Facebook* having been collaborating to boost PyTorch* CPU Performance. The official PyTorch has been optimized using oneAPI Deep Neural Network Library (oneDNN) primitives by default.\n",
36+
"PyTorch* is a very popular framework for deep learning, while also compute-heavy package demanding perfromance optimizations. Intel and Facebook* having been collaborating to boost PyTorch* Performance for Intel hardware which includes [Intel GPUs](https://www.intel.com/content/www/us/en/products/details/discrete-gpus/data-center-gpu.html) and [Intel CPUs](https://www.intel.com/content/www/us/en/products/details/processors.html). The official PyTorch has been optimized using oneAPI Deep Neural Network Library (oneDNN) primitives by default.\n",
2937
"\n",
30-
"To deliver the latest and greatest optimizations, Intel Optimizations for Pytorch offers accelerations beyond the stock Pytorch via **Intel® Extension for PyTorch***(IPEX). IPEX is a Python package to extend the official PyTorch with optimizations for extra performance boost on Intel hardware. Most of the optimizations will be included in stock PyTorch releases eventually, and the intention of the extension is to deliver up-to-date features and optimizations for PyTorch on Intel hardware, examples include AVX-512 Vector Neural Network Instructions (AVX512 VNNI) and Intel® Advanced Matrix Extensions (Intel® AMX).\n",
38+
"To deliver the latest and greatest optimizations, Intel® Optimizations for Pytorch offers accelerations beyond the stock Pytorch via **Intel® Extension for PyTorch***(IPEX). IPEX is a Python package to extend the official PyTorch with optimizations for extra performance boost on Intel hardware. Most of the optimizations will be included in stock PyTorch releases eventually, and the intention of the extension is to deliver up-to-date features and optimizations for PyTorch on Intel hardware, examples include AVX-512 Vector Neural Network Instructions (AVX512 VNNI) and Intel® Advanced Matrix Extensions (Intel® AMX).\n",
3139
"\n",
3240
"More detailed tutorials are available at [Intel® Extension for PyTorch* online document website](https://intel.github.io/intel-extension-for-pytorch/).\n",
3341
"\n",
3442
"## Purpose\n",
35-
"This sample code shows how to get started with Intel Extension for PyTorch as well as how to use AutoMixedPrecision with Intel Extension for PyTorch.\n",
43+
"This sample code shows how to get started with Intel® Extension for PyTorch as well as how to use AutoMixedPrecision with Intel® Extension for PyTorch.\n",
3644
"\n",
3745
"## Sample Table of Contents\n",
3846
"1. [Intel® Extension for PyTorch* Getting Started Sample](#sec-gs)\n",
39-
"2. [Intel® Extension for PyTorch* Auto Mixed Precision Sample](#sec-amp)"
47+
"2. [Intel® Extension for PyTorch* Auto Mixed Precision Sample](#sec-amp)\n",
48+
"3. [Intel® Neural Compressor for quantization](#sec-inc)"
4049
]
4150
},
4251
{
@@ -53,18 +62,37 @@
5362
"id": "a3d948ec-4a90-4b1a-9f9c-93fc430eafff",
5463
"metadata": {},
5564
"source": [
56-
"If you want to explore Intel Extension for PyTorch, you just need to convert the model and input tensors to the extension device, then the extension will be enabled automatically. Take an example, the code as follows is a model without the extension.\n",
65+
"If you want to explore Intel® Extension for PyTorch, you just need to convert the model and input tensors to the extension device, then the extension will be enabled automatically. Take an example, the code as follows is a model without the extension.\n",
66+
"\n",
67+
"\n",
68+
"**Please run this sample in the Intel® PyTorch & Quantization Jupyter Kernel environment.**"
69+
]
70+
},
71+
{
72+
"cell_type": "code",
73+
"execution_count": null,
74+
"id": "46a7ffdd-e9b4-4bf3-95a9-ab0bb6e5d84d",
75+
"metadata": {
76+
"tags": []
77+
},
78+
"outputs": [],
79+
"source": [
80+
"DEVICE=\"\"\n",
5781
"\n",
82+
"import intel_extension_for_pytorch as ipex\n",
5883
"\n",
59-
"**Please run this sample in the Intel PyTorch & Quantization Jupyter Kernel environment.**"
84+
"if ipex.xpu.is_available() == True:\n",
85+
" DEVICE=\"xpu\"\n",
86+
"else:\n",
87+
" DEVICE=\"cpu\""
6088
]
6189
},
6290
{
6391
"cell_type": "markdown",
6492
"id": "7ee02134-6598-4170-89d2-6a1211caf5b9",
6593
"metadata": {},
6694
"source": [
67-
"### PyTorch Model without Intel Extension for PyTorch"
95+
"### PyTorch Model without Intel® Extension for PyTorch"
6896
]
6997
},
7098
{
@@ -75,27 +103,26 @@
75103
"outputs": [],
76104
"source": [
77105
"import torch\n",
78-
"import torch.nn as nn\n",
106+
"import torchvision.models as models\n",
79107
"\n",
80-
"class Model(nn.Module):\n",
81-
" def __init__(self):\n",
82-
" super(Model, self).__init__()\n",
83-
" self.linear = nn.Linear(4, 5)\n",
108+
"model = models.resnet50(weights='ResNet50_Weights.DEFAULT')\n",
109+
"model.eval()\n",
110+
"data = torch.rand(128, 3, 224, 224)\n",
84111
"\n",
85-
" def forward(self, input):\n",
86-
" return self.linear(input)\n",
112+
"if DEVICE=='xpu':\n",
113+
" model = model.to(DEVICE)\n",
114+
" data = data.to(DEVICE)\n",
87115
"\n",
88-
"input = torch.randn(2, 4)\n",
89-
"model = Model()\n",
90-
"res = model(input)"
116+
"with torch.no_grad():\n",
117+
" model(data)"
91118
]
92119
},
93120
{
94121
"cell_type": "markdown",
95122
"id": "63e39ecb-2a24-4923-84e1-8e913cd6078f",
96123
"metadata": {},
97124
"source": [
98-
"### PyTorch Model with Intel Extension for PyTorch\n",
125+
"### PyTorch Model with Intel® Extension for PyTorch\n",
99126
"\n",
100127
"You just need to transform the above python script with **a couple lines of code** as follows and then the extension will be enabled and accelerate the computation automatically:"
101128
]
@@ -108,27 +135,27 @@
108135
"outputs": [],
109136
"source": [
110137
"import torch\n",
111-
"import torch.nn as nn\n",
112-
"\n",
113-
"# Import Extension\n",
114-
"import intel_pytorch_extension as ipex\n",
138+
"import torchvision.models as models\n",
115139
"\n",
116-
"class Model(nn.Module):\n",
117-
" def __init__(self):\n",
118-
" super(Model, self).__init__()\n",
119-
" self.linear = nn.Linear(4, 5)\n",
140+
"import intel_extension_for_pytorch as ipex\n",
120141
"\n",
121-
" def forward(self, input):\n",
122-
" return self.linear(input)\n",
142+
"model = models.resnet50(weights='ResNet50_Weights.DEFAULT')\n",
143+
"model.eval()\n",
144+
"data = torch.rand(128, 3, 224, 224)\n",
123145
"\n",
124146
"####################IPEX code changes#############################\n",
125-
"# Convert the input tensor to the Extension device\n",
126-
"input = torch.randn(2, 4).to(ipex.DEVICE)\n",
127-
"# Convert the model to the Extension device\n",
128-
"model = Model().to(ipex.DEVICE)\n",
147+
"if DEVICE=='xpu':\n",
148+
" model = model.to(DEVICE, memory_format=torch.channels_last)\n",
149+
" data = data.to(DEVICE, memory_format=torch.channels_last)\n",
150+
"else:\n",
151+
" model = model.to(memory_format=torch.channels_last)\n",
152+
" data = data.to(memory_format=torch.channels_last)\n",
153+
"\n",
154+
"model = ipex.optimize(model, dtype=torch.float32)\n",
129155
"##################################################################\n",
130156
"\n",
131-
"res = model(input)"
157+
"with torch.no_grad():\n",
158+
" model(data)"
132159
]
133160
},
134161
{
@@ -141,7 +168,7 @@
141168
"<a id=\"sec-amp\"></a>\n",
142169
"## Automatically Mix Precision\n",
143170
"\n",
144-
"In addition, Intel Extension for PyTorch supports the mixed precision. It means that some operators of a model may run with Float32 and some other operators may run with BFloat16 or INT8 to accelerate inference workload.\n",
171+
"In addition, Intel® Extension for PyTorch supports the mixed precision. It means that some operators of a model may run with Float32 and some other operators may run with BFloat16 or INT8 to accelerate inference workload.\n",
145172
"\n",
146173
"Traditionally if you want to run a model with a low precision type, you need to convert the parameters and the input tensors to the low precision type manually. And if the model contains some operators that do not support the low precision type, then you have to convert back to Float32. Round after round until the model can run normally.\n",
147174
"\n",
@@ -158,50 +185,92 @@
158185
"outputs": [],
159186
"source": [
160187
"import torch\n",
161-
"import torch.nn as nn\n",
188+
"import torchvision.models as models\n",
162189
"\n",
163-
"import intel_pytorch_extension as ipex\n",
164-
"# Automatically mix precision\n",
165-
"ipex.enable_auto_mixed_precision(mixed_dtype = torch.bfloat16)\n",
190+
"import intel_extension_for_pytorch as ipex\n",
166191
"\n",
167-
"class Model(nn.Module):\n",
168-
" def __init__(self):\n",
169-
" super(Model, self).__init__()\n",
170-
" self.linear = nn.Linear(4, 5)\n",
192+
"model = models.resnet50(weights='ResNet50_Weights.DEFAULT')\n",
193+
"model.eval()\n",
194+
"data = torch.rand(128, 3, 224, 224, dtype=torch.bfloat16)\n",
171195
"\n",
172-
" def forward(self, input):\n",
173-
" return self.linear(input)\n",
196+
"if DEVICE=='xpu':\n",
197+
" model = model.to(DEVICE, memory_format=torch.channels_last)\n",
198+
" data = data.to(DEVICE, memory_format=torch.channels_last)\n",
199+
"else:\n",
200+
" model = model.to(memory_format=torch.channels_last)\n",
201+
" data = data.to(memory_format=torch.channels_last)\n",
202+
"\n",
203+
"####################IPEX code changes#############################\n",
204+
"model = ipex.optimize(model, dtype=torch.bfloat16)\n",
205+
"##################################################################\n",
174206
"\n",
175-
"input = torch.randn(2, 4).to(ipex.DEVICE)\n",
176-
"model = Model().to(ipex.DEVICE)\n",
207+
"with torch.no_grad(), torch.cpu.amp.autocast():\n",
208+
" model(data)"
209+
]
210+
},
211+
{
212+
"cell_type": "markdown",
213+
"id": "c43af1ee-dff9-4704-be45-020b06123708",
214+
"metadata": {},
215+
"source": [
216+
"<a id=\"sec-inc\"></a>\n",
217+
"## Intel® Neural Compressor\n",
218+
"In addition to Intel® Extension for PyTorch, this container also provides Intel® Neural Compressor(INC) package. You can use INC to perform model optimization like compressing the model size and optimizing the performance for CPUs and GPUs. INC provides quantization, pruning, and knowledge distillation to compress and optimize the model. More details about the Intel® Neural Compressor package can be found [in the documentation](https://intel.github.io/neural-compressor/latest/docs/source/get_started.html).\n",
177219
"\n",
178-
"res = model(input)"
220+
"##### NOTE: Intel Neural Compressor does not yet suppot XPU with torchvision models."
179221
]
180222
},
181223
{
182224
"cell_type": "code",
183225
"execution_count": null,
184-
"id": "11dbfb50-cad2-43aa-8e7a-ade307cc7537",
185-
"metadata": {},
226+
"id": "e86f85b9-dd79-4789-b0fb-3bd222d6b57d",
227+
"metadata": {
228+
"tags": []
229+
},
186230
"outputs": [],
187231
"source": [
188-
"print('[CODE_SAMPLE_COMPLETED_SUCCESFULLY]')"
232+
"from neural_compressor.data import DataLoader, Datasets\n",
233+
"from neural_compressor.config import PostTrainingQuantConfig\n",
234+
"from neural_compressor.quantization import fit\n",
235+
"\n",
236+
"import torch\n",
237+
"import torchvision.models as models\n",
238+
"import intel_extension_for_pytorch as ipex\n",
239+
"\n",
240+
"DEVICE=\"cpu\" #INC does not work with GPU as of now.\n",
241+
"\n",
242+
"dataset = Datasets('pytorch')['dummy'](shape=(1, 3, 224, 224))\n",
243+
"# Built-in calibration dataloader and evaluation dataloader for Quantization.\n",
244+
"dataloader = DataLoader(framework='pytorch', dataset=dataset)\n",
245+
"# Post Training Quantization Config\n",
246+
"config = PostTrainingQuantConfig(backend='ipex', device=DEVICE)\n",
247+
"model = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)\n",
248+
"\n",
249+
"if DEVICE=='xpu':\n",
250+
" model = model.to(DEVICE, memory_format=torch.channels_last)\n",
251+
" \n",
252+
"# Just call fit to do quantization.\n",
253+
"q_model = fit(model=model,\n",
254+
" conf=config,\n",
255+
" calib_dataloader=dataloader)"
189256
]
190257
},
191258
{
192259
"cell_type": "code",
193260
"execution_count": null,
194-
"id": "9436f8ae-494a-4ee2-8b88-e1b34e11cbb6",
261+
"id": "11dbfb50-cad2-43aa-8e7a-ade307cc7537",
195262
"metadata": {},
196263
"outputs": [],
197-
"source": []
264+
"source": [
265+
"print('[CODE_SAMPLE_COMPLETED_SUCCESFULLY]')"
266+
]
198267
}
199268
],
200269
"metadata": {
201270
"kernelspec": {
202-
"display_name": "Intel PyTorch & Quantization",
271+
"display_name": "pytorch-gpu",
203272
"language": "python",
204-
"name": "oneapi-aikit-dlpackage-with-pytorch"
273+
"name": "python3"
205274
},
206275
"language_info": {
207276
"codemirror_mode": {
@@ -213,7 +282,7 @@
213282
"name": "python",
214283
"nbconvert_exporter": "python",
215284
"pygments_lexer": "ipython3",
216-
"version": "3.9.7"
285+
"version": "3.9.18"
217286
}
218287
},
219288
"nbformat": 4,

‎Intel_Modin_and_Intel_Extension_for_Scikit-Learn_E2E_Sample/Expected_output.jpg‎

51 KB
Loading

0 commit comments

Comments
 (0)