Skip to content

Commit 9d2613d

Browse files
spadacescoFrancesco Spadafora
andauthored
Doc/3839/update sample int8 readme (#3845)
* Replaced non existing call setINit8Mode INT8 ANN quantization is started by setting the builder flag kINT8 over the setFlags of the builder option Signed-off-by: Francesco Spadafora <sf-fs@windowslive.com> * Fix wring naming in example Replaced name with tensor_name in example Signed-off-by: Francesco Spadafora <sf-fs@windowslive.com> * setStrictTypeConstraints updated to BuilderFlag::kINT8 The documentation regarding debugging of the INT8 mode is updated to match the builder flags Signed-off-by: Francesco Spadafora <sf-fs@windowslive.com> * typo fix Signed-off-by: Francesco Spadafora <sf-fs@windowslive.com> --------- Signed-off-by: Francesco Spadafora <sf-fs@windowslive.com> Co-authored-by: Francesco Spadafora <francesco.spadafora@advertima.com>
1 parent d2f4ef7 commit 9d2613d

File tree

1 file changed

+5
-4
lines changed

1 file changed

+5
-4
lines changed

samples/sampleINT8API/README.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ Specifically, this sample performs the following steps:
4747
`if (!builder->platformHasFastInt8()) return false;`
4848

4949
2. Enable INT8 mode by setting the builder flag:
50-
`builder->setInt8Mode(true);`
50+
`builder->setFlag(BuilderFlag::kINT8);`
5151

5252
You can choose not to provide the INT8 calibrator.
5353
`builder->setInt8Calibrator(nullptr);`
@@ -56,10 +56,10 @@ Specifically, this sample performs the following steps:
5656

5757
3. Optionally and for debugging purposes, the following flag configures the builder to choose type conforming layer implementation, if one exists.
5858

59-
For example, in the case of `DataType::kINT8`, types are requested by `setInt8Mode(true)`. Setting this flag ensures that only the conformant layer implementation (with `kINT8` input and output types), are chosen even if a high performance non-conformat implementation is available. If no conformant layer exists, TensorRT will choose a non-conformant layer if available regardless of the setting for this flag.
60-
6159
`builder->setStrictTypeConstraints(true);`
6260

61+
Setting `setStrictTypeConstraints(true)` together with the builder flag `setFlag(BuilderFlag::kINT8)` ensures that only the conformant layer implementation (with `kINT8` input and output types) is chosen even if a high performance non-conformant implementation is available. If no conformant layer exists, TensorRT will choose a non-conformant layer if available regardless of the setting for this flag.
62+
6363
### Configuring the network to use custom dynamic ranges and set per-layer precision
6464

6565
1. Iterate through the network to set the per activation tensor dynamic range.
@@ -75,7 +75,8 @@ Specifically, this sample performs the following steps:
7575
7676
3. Set the dynamic range for per layer tensors:
7777
```
78-
string tensor_name = network->getLayer(i)->getOutput(j)->getName(); network->getLayer(i)->getOutput(j)->setDynamicRange(-tensorMap.at(name), tensorMap.at(name));
78+
string tensor_name = network->getLayer(i)->getOutput(j)->getName();
79+
network->getLayer(i)->getOutput(j)->setDynamicRange(-tensorMap.at(tensor_name), tensorMap.at(tensor_name));
7980
```
8081
8182
4. Optional: This sample also showcases using layer precision APIs. Using these APIs, you can selectively choose to run the layer with user configurable precision and type constraints. It may not result in optimal inference performance, but can be helpful while debugging mixed precision inference.

0 commit comments

Comments
 (0)