Docs enhancement (#1032)

Signed-off-by: chensuyue <suyue.chen@intel.com>

Author: chensuyue

.azure-pipelines/code-scan-neural-insights.yaml

@@ -11,7 +11,6 @@ pr:
       - neural_insights
       - setup.py
       - .azure-pipelines/code-scan-neural-insights.yml
-      - .azure-pipelines/scripts/codeScan
 
 pool:
   vmImage: "ubuntu-latest"

.azure-pipelines/code-scan-neural-solution.yaml

@@ -11,7 +11,6 @@ pr:
       - neural_solution
       - setup.py
       - .azure-pipelines/code-scan-neural-solution.yml
-      - .azure-pipelines/scripts/codeScan
 
 pool:
   vmImage: "ubuntu-latest"

.azure-pipelines/scripts/codeScan/pyspelling/inc_dict.txt

@@ -2490,6 +2490,7 @@ Thalaiyasingam
 Torr
 QOperator
 MixedPrecisionConfig
+mixedprecision
 contrib
 ONNXConfig
 Arial

docker/README.md

@@ -1,6 +1,6 @@
-## build `Neural Compressor(INC)` Containers:
+## Build Intel Neural Compressor Containers:
 
-### To build the the `Pip` based deployment container:
+### To build the `Pip` based deployment container:
 Please note that `INC_VER` must be set to a valid version published here:
 https://pypi.org/project/neural-compressor/#history
 
@@ -12,7 +12,7 @@ $ IMAGE_TAG=${INC_VER}
 $ docker build --build-arg PYTHON=${PYTHON} --build-arg INC_VER=${INC_VER} -f Dockerfile -t ${IMAGE_NAME}:${IMAGE_TAG} .
 ```
 
-###  To build the the `Pip` based development container:
+###  To build the `Pip` based development container:
 Please note that `INC_BRANCH` must be a set to a valid branch name otherwise, Docker build fails.
 If `${INC_BRANCH}-devel` does not meet Docker tagging requirements described here:
 https://docs.docker.com/engine/reference/commandline/tag/

docs/source/FX.md

@@ -2,7 +2,7 @@ FX
 ====
 1. [Introduction](#introduction)
 2. [FX Mode Support Matrix in Neural Compressor](#fx-mode-support-matrix-in-neural-compressor)
-3. [Get Start](#get-start)
+3. [Get Started](#get-started)
 
     3.1. [Post Training Static Quantization](#post-training-static-quantization)
 
@@ -34,7 +34,7 @@ For detailed description, please refer to [PyTorch FX](https://pytorch.org/docs/
 |Quantization-Aware Training         |✔     |
 
 
-## Get Start
+## Get Started
 
 **Note:** "backend" field indicates the backend used by the user in configure. And the "default" value means it will quantization model with fx backend for PyTorch model.
 

docs/source/adaptor.md

@@ -3,7 +3,7 @@ Adaptor
 1. [Introduction](#introduction)
 2. [Adaptor Support Matrix](#adaptor-support-matrix)
 3. [Working Flow](#working-flow)
-4. [Get Start with Adaptor API](#get-start-with-adaptor-api)
+4. [Get Started with Adaptor API](#get-start-with-adaptor-api)
 
     4.1 [Query API](#query-api)
 
@@ -33,7 +33,7 @@ tuning strategy and vanilla framework quantization APIs.
 ## Working Flow
 Adaptor only provide framework API for tuning strategy. So we can find complete working flow in [tuning strategy working flow](./tuning_strategies.md).
 
-## Get Start with Adaptor API
+## Get Started with Adaptor API
 
 Neural Compressor supports a new adaptor extension by
 implementing a subclass `Adaptor` class in the neural_compressor.adaptor package

docs/source/dataloader.md

@@ -5,7 +5,7 @@ DataLoader
 
 2. [Supported Framework Dataloader Matrix](#supported-framework-dataloader-matrix)
 
-3. [Get Start with Dataloader](#get-start-with-dataloader)
+3. [Get Started with Dataloader](#get-start-with-dataloader)
 
     3.1 [Use Intel® Neural Compressor DataLoader API](#use-intel®-neural-compressor-dataloader-api)
 
@@ -37,7 +37,7 @@ Of cause, users can also use frameworks own dataloader in Neural Compressor.
 | PyTorch       |  ✔  |
 | ONNX Runtime   |  ✔  |
 
-## Get Start with DataLoader
+## Get Started with DataLoader
 
 ### Use Intel® Neural Compressor DataLoader API
 

docs/source/get_started.md

@@ -2,10 +2,6 @@
 
 1. [Quick Samples](#quick-samples)
 
-    1.1 [Quantization with Python API](#quantization-with-python-api)
-
-    1.2 [Quantization with JupyterLab Extension](#quantization-with-jupyterlab-extension)
-
 2. [Validated Models](#validated-models)
 
 ## Quick Samples
@@ -35,14 +31,6 @@ q_model = fit(
     eval_dataloader=dataloader)
 ```
 
-### Quantization with [JupyterLab Extension](/neural_coder/extensions/neural_compressor_ext_lab/README.md)
-
-Search for ```jupyter-lab-neural-compressor``` in the Extension Manager in JupyterLab and install with one click:
-
-<a target="_blank" href="/neural_coder/extensions/screenshots/extmanager.png">
-  <img src="/neural_coder/extensions/screenshots/extmanager.png" alt="Extension" width="35%" height="35%">
-</a>
-
 ## Validated Models
 Intel® Neural Compressor validated the quantization for 10K+ models from popular model hubs (e.g., HuggingFace Transformers, Torchvision, TensorFlow Model Hub, ONNX Model Zoo). 
 Over 30 pruning, knowledge distillation and model export samples are also available. 

docs/source/installation_guide.md

@@ -30,7 +30,7 @@ You can install Neural Compressor using one of three options: Install single com
 
 The following prerequisites and requirements must be satisfied for a successful installation:
 
-- Python version: 3.7 or 3.8 or 3.9 or 3.10
+- Python version: 3.7 or 3.8 or 3.9 or 3.10 or 3.11
 
 > Notes:
 > - If you get some build issues, please check [frequently asked questions](faq.md) at first.
@@ -82,7 +82,7 @@ The AI Kit is distributed through many common channels, including from Intel's w
 
 The following prerequisites and requirements must be satisfied for a successful installation:
 
-- Python version: 3.7 or 3.8 or 3.9 or 3.10
+- Python version: 3.7 or 3.8 or 3.9 or 3.10 or 3.11
 
 ### Install from Binary
 
@@ -127,7 +127,7 @@ The following prerequisites and requirements must be satisfied for a successful
 ### Validated Software Environment
 
 * OS version: CentOS 8.4, Ubuntu 22.04
-* Python version: 3.7, 3.8, 3.9, 3.10
+* Python version: 3.7, 3.8, 3.9, 3.10, 3.11
 
 <table class="docutils">
 <thead>
@@ -148,20 +148,20 @@ The following prerequisites and requirements must be satisfied for a successful
     <td class="tg-7zrl"><a href=https://github.com/tensorflow/tensorflow/tree/v2.12.0>2.12.0</a><br>
     <a href=https://github.com/tensorflow/tensorflow/tree/v2.11.0>2.11.0</a><br>
     <a href=https://github.com/tensorflow/tensorflow/tree/v2.10.1>2.10.1</a><br></td>
-    <td class="tg-7zrl"><a href=https://github.com/Intel-tensorflow/tensorflow/tree/v2.11.0>2.11.0</a><br>
-    <a href=https://github.com/Intel-tensorflow/tensorflow/tree/v2.10.0>2.10.0</a><br>
-    <a href=https://github.com/Intel-tensorflow/tensorflow/tree/v2.9.1>2.9.1</a><br></td>
-    <td class="tg-7zrl"><a href=https://github.com/intel/intel-extension-for-tensorflow/tree/v1.1.0>1.1.0</a><br>
-    <a href=https://github.com/intel/intel-extension-for-tensorflow/tree/v1.0.0>1.0.0</a></td>
-    <td class="tg-7zrl"><a href=https://download.pytorch.org/whl/torch_stable.html>2.0.0+cpu</a><br>
-    <a href=https://download.pytorch.org/whl/torch_stable.html>1.13.0+cpu</a><br>
+    <td class="tg-7zrl"><a href=https://github.com/Intel-tensorflow/tensorflow/tree/v2.12.0>2.12.0</a><br>
+    <a href=https://github.com/Intel-tensorflow/tensorflow/tree/v2.11.0>2.11.0</a><br>
+    <a href=https://github.com/Intel-tensorflow/tensorflow/tree/v2.10.0>2.10.0</a><br></td>
+    <td class="tg-7zrl"><a href=https://github.com/intel/intel-extension-for-tensorflow/tree/v1.2.0>1.2.0</a><br>
+    <a href=https://github.com/intel/intel-extension-for-tensorflow/tree/v1.1.0>1.1.0</a></td>
+    <td class="tg-7zrl"><a href=https://download.pytorch.org/whl/torch_stable.html>2.0.1+cpu</a><br>
+    <a href=https://download.pytorch.org/whl/torch_stable.html>1.13.1+cpu</a><br>
     <a href=https://download.pytorch.org/whl/torch_stable.html>1.12.1+cpu</a><br></td>
-    <td class="tg-7zrl"><a href=https://github.com/intel/intel-extension-for-pytorch/tree/v2.0.0+cpu>2.0.0+cpu</a><br>
-    <a href=https://github.com/intel/intel-extension-for-pytorch/tree/v1.13.0+cpu>1.13.0+cpu</a><br>
+    <td class="tg-7zrl"><a href=https://github.com/intel/intel-extension-for-pytorch/tree/v2.0.100+cpu>2.0.1+cpu</a><br>
+    <a href=https://github.com/intel/intel-extension-for-pytorch/tree/v1.13.100+cpu>1.13.1+cpu</a><br>
     <a href=https://github.com/intel/intel-extension-for-pytorch/tree/v1.12.100>1.12.1+cpu</a><br></td>
-    <td class="tg-7zrl"><a href=https://github.com/microsoft/onnxruntime/tree/v1.14.1>1.14.1</a><br>
-    <a href=https://github.com/microsoft/onnxruntime/tree/v1.13.1>1.13.1</a><br>
-    <a href=https://github.com/microsoft/onnxruntime/tree/v1.12.1>1.12.1</a><br></td>
+    <td class="tg-7zrl"><a href=https://github.com/microsoft/onnxruntime/tree/v1.15.0>1.15.0</a><br>
+    <a href=https://github.com/microsoft/onnxruntime/tree/v1.14.1>1.14.1</a><br>
+    <a href=https://github.com/microsoft/onnxruntime/tree/v1.13.1>1.13.1</a><br></td>
     <td class="tg-7zrl"><a href=https://github.com/apache/incubator-mxnet/tree/1.9.1>1.9.1</a><br></td>
   </tr>
 </tbody>

docs/source/metric.md

@@ -11,7 +11,7 @@ Metrics
 
     2.4. [ONNXRT](#onnxrt)
 
-3. [Get Start with Metric](#get-start-with-metric)
+3. [Get Started with Metric](#get-start-with-metric)
 
     3.1. [Use Intel® Neural Compressor Metric API](#use-intel®-neural-compressor-metric-api)
 
@@ -88,11 +88,11 @@ Neural Compressor supports some built-in metrics that are popularly used in indu
 
 
 
-## Get Start with Metric
+## Get Started with Metric
 
 ### Use Intel® Neural Compressor Metric API
 
-Users can specify an Neural Compressor built-in metric such as shown below:
+Users can specify a Neural Compressor built-in metric such as shown below:
 
 ```python
 from neural_compressor import Metric

docs/source/mixed_precision.md

@@ -12,30 +12,126 @@ The recent growth of Deep Learning has driven the development of more complex mo
 
 The recently launched 3rd Gen Intel® Xeon® Scalable processor (codenamed Cooper Lake), featuring Intel® Deep Learning Boost, is the first general-purpose x86 CPU to support the bfloat16 format. Specifically, three new bfloat16 instructions are added as a part of the AVX512_BF16 extension within Intel Deep Learning Boost: VCVTNE2PS2BF16, VCVTNEPS2BF16, and VDPBF16PS. The first two instructions allow converting to and from bfloat16 data type, while the last one performs a dot product of bfloat16 pairs. Further details can be found in the [hardware numerics document](https://software.intel.com/content/www/us/en/develop/download/bfloat16-hardware-numerics-definition.html) published by Intel.
 
-<a target="_blank" href="./imgs/data_format.png" text-align:center>
-    <center> 
-        <img src="./imgs/data_format.png" alt="Architecture" height=200> 
-    </center>
-</a>
+<p align="center" width="100%">
+    <img src="./imgs/data_format.png" alt="Architecture" height=230>
+</p>
 
 ## Mixed Precision Support Matrix
-
-|Framework     |BF16         |FP16         |
-|--------------|:-----------:|:-----------:|
-|TensorFlow    |&#10004;     |:x:     |
-|PyTorch       |&#10004;     |:x:     |
-|ONNX Runtime  |&#10004;     |&#10004;     |
-|MXNet         |&#10004;     |:x:     |
-
-> **During quantization, BF16 conversion is default enabled, FP16 can be executed if 'device' of config is 'gpu'. Please refer to this [document](./quantization_mixed_precision.md) for its workflow.**
+<table class="center">
+    <thead>
+        <tr>
+            <th>Framework</th>
+            <th>Backend</th>
+            <th>Backend Library</th>
+            <th>Backend Value</th>
+            <th>Support Device(cpu as default)</th> 
+            <th>Support BF16</th>
+            <th>Support FP16</th>
+        </tr>
+    </thead>
+    <tbody>
+        <tr>
+            <td rowspan="2" align="left">PyTorch</td>
+            <td align="left">FX</td>
+            <td align="left">FBGEMM</td>
+            <td align="left">"default"</td>
+            <td align="left">cpu</td>
+            <td align="left">&#10004;</td>
+            <td align="left">:x:</td>
+        </tr>
+        <tr>
+            <td align="left">IPEX</td>
+            <td align="left">OneDNN</td>
+            <td align="left">"ipex"</td>
+            <td align="left">cpu</td>
+            <td align="left">&#10004;</td>
+            <td align="left">:x:</td>
+        </tr>
+        <tr>
+            <td rowspan="3" align="left">ONNX Runtime</td>
+            <td align="left">CPUExecutionProvider</td>
+            <td align="left">MLAS</td>
+            <td align="left">"default"</td>
+            <td align="left">cpu</td>
+            <td align="left">:x:</td>
+            <td align="left">:x:</td>
+        </tr>
+        <tr>
+            <td align="left">TensorrtExecutionProvider</td>
+            <td align="left">TensorRT</td>
+            <td align="left">"onnxrt_trt_ep"</td>
+            <td align="left">gpu</td>
+            <td align="left">:x:</td>
+            <td align="left">:x:</td>
+        </tr>
+        <tr>
+            <td align="left">CUDAExecutionProvider</td>
+            <td align="left">CUDA</td>
+            <td align="left">"onnxrt_cuda_ep"</td>
+            <td align="left">gpu</td>
+            <td align="left">&#10004;</td>
+            <td align="left">&#10004;</td>
+        </tr>
+        <tr>
+            <td rowspan="2" align="left">Tensorflow</td>
+            <td align="left">Tensorflow</td>
+            <td align="left">OneDNN</td>
+            <td align="left">"default"</td>
+            <td align="left">cpu</td>
+            <td align="left">&#10004;</td>
+            <td align="left">:x:</td>
+        </tr>
+        <tr>
+            <td align="left">ITEX</td>
+            <td align="left">OneDNN</td>
+            <td align="left">"itex"</td>
+            <td align="left">cpu | gpu</td>
+            <td align="left">&#10004;</td>
+            <td align="left">:x:</td>
+        </tr>  
+        <tr>
+            <td align="left">MXNet</td>
+            <td align="left">OneDNN</td>
+            <td align="left">OneDNN</td>
+            <td align="left">"default"</td>
+            <td align="left">cpu</td>
+            <td align="left">&#10004;</td>
+            <td align="left">:x:</td>
+        </tr>
+    </tbody>
+</table>
+
+
+### Hardware and Software requests for **BF16**
+- TensorFlow
+  1. Hardware: CPU supports `avx512_bf16` instruction set.
+  2. Software: intel-tensorflow >= [2.3.0](https://pypi.org/project/intel-tensorflow/2.3.0/).
+- PyTorch
+  1. Hardware: CPU supports `avx512_bf16` instruction set.
+  2. Software: torch >= [1.11.0](https://download.pytorch.org/whl/torch_stable.html).
+- ONNX Runtime
+  1. Hardware: GPU, set 'device' of config to 'gpu' and 'backend' to 'onnxrt_cuda_ep'.
+  2. Software: onnxruntime-gpu.
+
+### Hardware and Software requests for **FP16**
+- ONNX Runtime
+  1. Hardware: GPU, set 'device' of config to 'gpu' and 'backend' to 'onnxrt_cuda_ep'.
+  2. Software: onnxruntime-gpu.
+
+### During quantization mixed precision
+During quantization, if the hardware support BF16, the conversion is default enabled. So you may get an INT8/BF16/FP32 mixed precision model on those hardware. FP16 can be executed if 'device' of config is 'gpu'.
+Please refer to this [document](https://github.com/intel/neural-compressor/blob/master/docs/source/quantization_mixed_precision.md) for its workflow.
+
+### Accuracy-driven mixed precision
+BF16/FP16 conversion may lead to accuracy drop. Intel® Neural Compressor provides an accuracy-driven tuning function to reduce accuracy loss, 
+which will fallback converted ops to FP32 automatically to get better accuracy. To enable this function, users only to provide 
+`evaluation function` or (`evaluation dataloader` plus `evaluation metric`) for [mixed precision inputs](https://github.com/intel/neural-compressor/blob/master/neural_compressor/mix_precision.py).   
+To be noticed, IPEX backend doesn't support accuracy-driven mixed precision.  
 
 ## Get Started with Mixed Precision API
 
 To get a bf16/fp16 model, users can use the Mixed Precision API as follows.
 
-
-Supported precisions for mix precision include bf16 and fp16. If users want to get a pure fp16 or bf16 model, they should add another precision into excluded_precisions.
-
 - BF16:
 
 ```python
@@ -60,34 +156,10 @@ conf = MixedPrecisionConfig(
 converted_model = mix_precision.fit(model, conf=conf)
 converted_model.save('./path/to/save/')
 ```
-
-> **BF16/FP16 conversion may lead to accuracy drop. Intel® Neural Compressor provides an accuracy-aware tuning function to reduce accuracy loss, which will fallback converted ops to FP32 automatically to get better accuracy. To enable this function, users only need to provide an evaluation function (or dataloader + metric).**
-
 
 ## Examples
 
-There are some pre-requirements to run mixed precision examples for each framework. If the hardware requirements cannot be met, the program would exit consequently.
-
-- **BF16:** 
-
-    #### TensorFlow
-
-    1. Hardware: CPU supports `avx512_bf16` instruction set.
-    2. Software: intel-tensorflow >= [2.3.0](https://pypi.org/project/intel-tensorflow/2.3.0/).
-
-    #### PyTorch
-
-    1. Hardware: CPU supports `avx512_bf16` instruction set.
-    2. Software: torch >= [1.11.0](https://download.pytorch.org/whl/torch_stable.html).
-
-    #### ONNX Runtime
-
-    1. Hardware: GPU, set 'device' of config to 'gpu' and 'backend' to 'onnxrt_cuda_ep'.
-    2. Software: onnxruntime-gpu.
-
-- **FP16:**
-
-    #### ONNX Runtime
-
-    1. Hardware: GPU, set 'device' of config to 'gpu' and 'backend' to 'onnxrt_cuda_ep'.
-    2. Software: onnxruntime-gpu.
+- Quick started with [helloworld example](/examples/helloworld/tf_example3)
+- PyTorch [ResNet18](/examples/pytorch/image_recognition/torchvision_models/mixed_precision/resnet18)
+- IPEX [DistilBERT base](/examples/pytorch/nlp/huggingface_models/question-answering/mixed_precision/ipex)
+- Tensorflow [ResNet50](/examples/tensorflow/image_recognition/tensorflow_models/resnet50_v1/mixed_precision) 

docs/source/releases_info.md

@@ -28,3 +28,5 @@ The diagnosis function does not work with ONNX Runtime 1.13.1 for QDQ format qua
 [Neural Compressor v1.7](https://github.com/intel/neural-compressor/tree/v1.7) renames the pip/conda package name from lpot to neural_compressor. To run old examples on latest software, please replace package name for compatibility with `sed -i "s|lpot|neural_compressor|g" your_script.py` .
 
 [Neural Compressor v2.0](https://github.com/intel/neural-compressor/tree/v2.0) renames the `DATASETS` class as `Datasets`, please notice use cases like `from neural_compressor.data import Datasets`. Details please check the [PR](https://github.com/intel/neural-compressor/pull/244/files).
+
+[Neural Compressor v2.2](https://github.com/intel/neural-compressor/tree/v2.2) from this release, binary `neural-compressor-full` is deprecated, we deliver 3 binaries named `neural-compressor`, `neural-solution` and `neural-insights`.