Skip to content

Commit 59a7ca0

Browse files
shashwat1198shashwat1198
committed
package installations added
1 parent e2e0737 commit 59a7ca0

File tree

3 files changed

+518
-43
lines changed

3 files changed

+518
-43
lines changed

‎notebooks/4_quant_lstm.ipynb

Lines changed: 38 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,19 @@
22
"cells": [
33
{
44
"cell_type": "markdown",
5-
"id": "5ef5f772-f48a-4bb1-bb68-4e8e9236fd2e",
65
"metadata": {},
76
"source": [
87
"# QuantLSTM - ONNX (QCDQ) representation"
98
]
109
},
1110
{
1211
"cell_type": "markdown",
13-
"id": "e5a747f9-fd74-4ebc-8d74-17bf06ff2d48",
1412
"metadata": {},
1513
"source": [
16-
"This notebook is divided into `five` parts:\n",
14+
"This notebook is divided into `six` parts:\n",
1715
"\n",
16+
"<br><b>Part 0</b> : Package Installations.\n",
17+
"<br>\n",
1818
"<br><b>Part 1</b> : Introduction to LSTMs.\n",
1919
"<br>\n",
2020
"<br><b>Part 2</b> : Model creation with brevitas QuantLSTM layer. \n",
@@ -28,16 +28,45 @@
2828
},
2929
{
3030
"cell_type": "markdown",
31-
"id": "69ae7154-8cf3-4ee7-88c3-3bec0550008a",
31+
"metadata": {},
32+
"source": [
33+
"# Package Installations"
34+
]
35+
},
36+
{
37+
"cell_type": "code",
38+
"execution_count": null,
39+
"metadata": {},
40+
"outputs": [],
41+
"source": [
42+
"#Required package installations, This cell only needs to be executed once at the start\n",
43+
"!pip install torch==1.13.1\n",
44+
"!pip install brevitas==0.9.1\n",
45+
"!pip install onnx==1.13.0\n",
46+
"!pip install onnxoptimizer==0.3.13\n",
47+
"!pip install onnxruntime==1.11.1\n",
48+
"!pip install netron==7.2.5\n",
49+
"!pip install qonnx==0.2.0\n",
50+
"!pip install IPython\n",
51+
"!pip install ipykernel\n",
52+
"!ipython kernel install --user --name=venv\n",
53+
"\n",
54+
"#The below location can change depending on your installation of the 'venv' virtual environment\n",
55+
"!cp ./4_quant_lstm_helper/function.py ../venv/lib/python3.8/site-packages/brevitas/export/onnx/standard/\n",
56+
"!cp ./4_quant_lstm_helper/handler.py ../venv/lib/python3.8/site-packages/brevitas/export/onnx/standard/qcdq/\n",
57+
"\n",
58+
"#NOTE : Make sure to chnage the kernel to from \"Python 3\" to \"venv\" before running the below commands"
59+
]
60+
},
61+
{
62+
"cell_type": "markdown",
3263
"metadata": {},
3364
"source": [
3465
"# Introduction to LSTM's "
3566
]
3667
},
3768
{
38-
"attachments": {},
3969
"cell_type": "markdown",
40-
"id": "e7a903ef-1680-4a20-8c61-267884b76c96",
4170
"metadata": {},
4271
"source": [
4372
"`LSTM’s (Long Short-Term Memory)` are sequential neural networks that are capable of learning long term dependencies especially in sequence prediction problems. They are deployed in machine translation, speech recognition, image captioning and especially used for time-series analysis applications.\n",
@@ -73,15 +102,13 @@
73102
},
74103
{
75104
"cell_type": "markdown",
76-
"id": "70d052c8-e5cd-4eb1-89e5-f8ae956cb853",
77105
"metadata": {},
78106
"source": [
79107
"# QuantLSTM model creation"
80108
]
81109
},
82110
{
83111
"cell_type": "markdown",
84-
"id": "6a64be7c",
85112
"metadata": {},
86113
"source": [
87114
"In the 2nd part of the notebook, we will create a single layer `QuantLSTM` model in brevitas. We will evaluate with a given set of inputs. We then export this model to `QONNX` so that the same parameters (weights/biases/scales) can be extracted and used in the `QCDQ-LSTM` implementation."
@@ -90,7 +117,6 @@
90117
{
91118
"cell_type": "code",
92119
"execution_count": null,
93-
"id": "84d66548-365d-46a5-9eaa-bb767085f9aa",
94120
"metadata": {},
95121
"outputs": [],
96122
"source": [
@@ -119,7 +145,6 @@
119145
{
120146
"cell_type": "code",
121147
"execution_count": null,
122-
"id": "23a7682c",
123148
"metadata": {},
124149
"outputs": [],
125150
"source": [
@@ -153,7 +178,6 @@
153178
},
154179
{
155180
"cell_type": "markdown",
156-
"id": "347ef1f5-36e8-4103-9b13-efa7fe93eb5e",
157181
"metadata": {},
158182
"source": [
159183
"`Abbreviations` : Short-forms defined in the next code block can be referenced here for definitions.\n",
@@ -166,7 +190,6 @@
166190
{
167191
"cell_type": "code",
168192
"execution_count": null,
169-
"id": "0bfbf5a3-8556-4190-a28f-4fe9859c55a9",
170193
"metadata": {},
171194
"outputs": [],
172195
"source": [
@@ -210,15 +233,13 @@
210233
},
211234
{
212235
"cell_type": "markdown",
213-
"id": "10237589-f84e-423a-829e-3e2c2e806ed7",
214236
"metadata": {},
215237
"source": [
216238
"# LSTM ONNX model"
217239
]
218240
},
219241
{
220242
"cell_type": "markdown",
221-
"id": "367547b8",
222243
"metadata": {},
223244
"source": [
224245
"In the 3rd part of the notebook, we will construct the `QCDQ-LSTM` model with standard ONNX operators. After loading all the parameters in the above block we can now start building our ONNX model with QCDQ quantization to represent the LSTM computations described in part-1.\n"
@@ -227,7 +248,6 @@
227248
{
228249
"cell_type": "code",
229250
"execution_count": null,
230-
"id": "02fe4d94-af24-4d5e-a809-7d8c49e7fd90",
231251
"metadata": {},
232252
"outputs": [],
233253
"source": [
@@ -249,7 +269,6 @@
249269
},
250270
{
251271
"cell_type": "markdown",
252-
"id": "15098a9e-4187-4987-82cc-275eba650923",
253272
"metadata": {},
254273
"source": [
255274
"`Abbreviations` : These describe different short-forms used in the next two blocks.\n",
@@ -265,7 +284,6 @@
265284
},
266285
{
267286
"cell_type": "markdown",
268-
"id": "f2edc0cc",
269287
"metadata": {},
270288
"source": [
271289
"We start defining the model by defining the `inputs` and `outputs` defined as value_info tensors in ONNX.\n",
@@ -276,7 +294,6 @@
276294
{
277295
"cell_type": "code",
278296
"execution_count": null,
279-
"id": "02761646-4c6d-440f-8e90-4935beebab56",
280297
"metadata": {},
281298
"outputs": [],
282299
"source": [
@@ -294,7 +311,6 @@
294311
{
295312
"cell_type": "code",
296313
"execution_count": null,
297-
"id": "c08e5a23-ef2e-4bca-9293-c800350c2c62",
298314
"metadata": {},
299315
"outputs": [],
300316
"source": [
@@ -412,7 +428,6 @@
412428
},
413429
{
414430
"cell_type": "markdown",
415-
"id": "3d10867f",
416431
"metadata": {},
417432
"source": [
418433
"After defining the above operations we now connect them and create a graph with the help of onnx.helper `make_graph` utility function"
@@ -421,7 +436,6 @@
421436
{
422437
"cell_type": "code",
423438
"execution_count": null,
424-
"id": "79839558-8752-4fc8-9b0e-8fed47c91701",
425439
"metadata": {},
426440
"outputs": [],
427441
"source": [
@@ -632,7 +646,6 @@
632646
},
633647
{
634648
"cell_type": "markdown",
635-
"id": "b1b16751",
636649
"metadata": {},
637650
"source": [
638651
"The above created graph can now be converted into a qonnx model with the `qonnx_make_model` utility. We save the model with `onnx.save` utility and then view it in Netron with the help of `showInNetron` utility. \n"
@@ -641,7 +654,6 @@
641654
{
642655
"cell_type": "code",
643656
"execution_count": null,
644-
"id": "c6ec7b2a-456d-4452-97ec-df9a471d5391",
645657
"metadata": {},
646658
"outputs": [],
647659
"source": [
@@ -652,7 +664,6 @@
652664
},
653665
{
654666
"cell_type": "markdown",
655-
"id": "40b49257",
656667
"metadata": {},
657668
"source": [
658669
"In this block of code we execute the onnx graph to check that it can execute without any errors. We perform it's functional verification in the later part of the notebook."
@@ -661,7 +672,6 @@
661672
{
662673
"cell_type": "code",
663674
"execution_count": null,
664-
"id": "db5892bc-ac8d-4972-afcf-20bf880f5e86",
665675
"metadata": {},
666676
"outputs": [],
667677
"source": [
@@ -691,15 +701,13 @@
691701
},
692702
{
693703
"cell_type": "markdown",
694-
"id": "5d2b5a1e-654e-46a5-9d4f-8708611a6d1e",
695704
"metadata": {},
696705
"source": [
697706
"# SCAN Operation Integration"
698707
]
699708
},
700709
{
701710
"cell_type": "markdown",
702-
"id": "7365329a-f3d2-4f74-8e2f-9076771e07a7",
703711
"metadata": {},
704712
"source": [
705713
"### Introduction to ONNX Scan operation\n",
@@ -721,7 +729,6 @@
721729
},
722730
{
723731
"cell_type": "markdown",
724-
"id": "17f247f7",
725732
"metadata": {},
726733
"source": [
727734
"The `Scan` operation is essentially a container operator which will consume the LSTM graph that we created above in it's body.\n",
@@ -733,7 +740,6 @@
733740
{
734741
"cell_type": "code",
735742
"execution_count": null,
736-
"id": "700a93a8-f757-4fa1-88dd-47a3f2a7f171",
737743
"metadata": {},
738744
"outputs": [],
739745
"source": [
@@ -750,7 +756,6 @@
750756
},
751757
{
752758
"cell_type": "markdown",
753-
"id": "572f191e",
754759
"metadata": {},
755760
"source": [
756761
"We will now create the scan operator here now utilizing the `make_node` utility from ONNX.\n",
@@ -760,7 +765,6 @@
760765
{
761766
"cell_type": "code",
762767
"execution_count": null,
763-
"id": "111fdce4-464f-40c1-ac4d-3022b05f153e",
764768
"metadata": {},
765769
"outputs": [],
766770
"source": [
@@ -775,7 +779,6 @@
775779
},
776780
{
777781
"cell_type": "markdown",
778-
"id": "ea8a05d9",
779782
"metadata": {},
780783
"source": [
781784
"We can now define the graph for the scan operator utilizing the `make_graph` utility."
@@ -784,7 +787,6 @@
784787
{
785788
"cell_type": "code",
786789
"execution_count": null,
787-
"id": "4668cf2b-524e-4768-8dc8-9d619f6273da",
788790
"metadata": {},
789791
"outputs": [],
790792
"source": [
@@ -810,7 +812,6 @@
810812
},
811813
{
812814
"cell_type": "markdown",
813-
"id": "0673e335",
814815
"metadata": {},
815816
"source": [
816817
"Now that we have the SCAN based quantized LSTM model ready, we can now go forward and test it with the same sets of inputs we used for the testing of the brevitas model.\n"
@@ -819,7 +820,6 @@
819820
{
820821
"cell_type": "code",
821822
"execution_count": null,
822-
"id": "818d2a81-686f-4a4a-8e78-17dbf75d8451",
823823
"metadata": {},
824824
"outputs": [],
825825
"source": [
@@ -854,15 +854,13 @@
854854
},
855855
{
856856
"cell_type": "markdown",
857-
"id": "907d2ff9-f605-4aec-891e-0c77a1a92346",
858857
"metadata": {},
859858
"source": [
860859
"# Functional Verification"
861860
]
862861
},
863862
{
864863
"cell_type": "markdown",
865-
"id": "b6bb6c60",
866864
"metadata": {},
867865
"source": [
868866
"In the final part of the notebook, we compare the output of the 8-bit quantized `(QCDQ)-LSTM` implementation with the `QuantLSTM` brevitas model.\n"
@@ -871,7 +869,6 @@
871869
{
872870
"cell_type": "code",
873871
"execution_count": null,
874-
"id": "2fe07395-6cf9-4c99-a0d3-a27aa6a326b5",
875872
"metadata": {},
876873
"outputs": [],
877874
"source": [
@@ -900,7 +897,6 @@
900897
},
901898
{
902899
"cell_type": "markdown",
903-
"id": "7bcca933",
904900
"metadata": {},
905901
"source": [
906902
"Note the difference in outputs increases as we progress with processing the inputs. The first two outputs are very close to one another, but as we get the outputs for more inputs we see for some values differ from the brevitas output by a considerable amount.\n",
@@ -909,16 +905,15 @@
909905
},
910906
{
911907
"cell_type": "markdown",
912-
"id": "81c6d531",
913908
"metadata": {},
914909
"source": []
915910
}
916911
],
917912
"metadata": {
918913
"kernelspec": {
919-
"display_name": "Python 3 (ipykernel)",
914+
"display_name": "venv",
920915
"language": "python",
921-
"name": "python3"
916+
"name": "venv"
922917
},
923918
"language_info": {
924919
"codemirror_mode": {
@@ -930,7 +925,7 @@
930925
"name": "python",
931926
"nbconvert_exporter": "python",
932927
"pygments_lexer": "ipython3",
933-
"version": "3.8.10"
928+
"version": "3.8.0"
934929
}
935930
},
936931
"nbformat": 4,

0 commit comments

Comments
 (0)