You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Replaced non existing call setINit8Mode
INT8 ANN quantization is started by setting the builder flag kINT8 over the setFlags of the builder option
Signed-off-by: Francesco Spadafora <sf-fs@windowslive.com>
* Fix wring naming in example
Replaced name with tensor_name in example
Signed-off-by: Francesco Spadafora <sf-fs@windowslive.com>
* setStrictTypeConstraints updated to BuilderFlag::kINT8
The documentation regarding debugging of the INT8 mode is updated to match the builder flags
Signed-off-by: Francesco Spadafora <sf-fs@windowslive.com>
* typo fix
Signed-off-by: Francesco Spadafora <sf-fs@windowslive.com>
---------
Signed-off-by: Francesco Spadafora <sf-fs@windowslive.com>
Co-authored-by: Francesco Spadafora <francesco.spadafora@advertima.com>
You can choose not to provide the INT8 calibrator.
53
53
`builder->setInt8Calibrator(nullptr);`
@@ -56,10 +56,10 @@ Specifically, this sample performs the following steps:
56
56
57
57
3. Optionally and for debugging purposes, the following flag configures the builder to choose type conforming layer implementation, if one exists.
58
58
59
-
For example, in the case of `DataType::kINT8`, types are requested by `setInt8Mode(true)`. Setting this flag ensures that only the conformant layer implementation (with `kINT8` input and output types), are chosen even if a high performance non-conformat implementation is available. If no conformant layer exists, TensorRT will choose a non-conformant layer if available regardless of the setting for this flag.
60
-
61
59
`builder->setStrictTypeConstraints(true);`
62
60
61
+
Setting `setStrictTypeConstraints(true)` together with the builder flag `setFlag(BuilderFlag::kINT8)` ensures that only the conformant layer implementation (with `kINT8` input and output types) is chosen even if a high performance non-conformant implementation is available. If no conformant layer exists, TensorRT will choose a non-conformant layer if available regardless of the setting for this flag.
62
+
63
63
### Configuring the network to use custom dynamic ranges and set per-layer precision
64
64
65
65
1. Iterate through the network to set the per activation tensor dynamic range.
@@ -75,7 +75,8 @@ Specifically, this sample performs the following steps:
4. Optional: This sample also showcases using layer precision APIs. Using these APIs, you can selectively choose to run the layer with user configurable precision and type constraints. It may not result in optimal inference performance, but can be helpful while debugging mixed precision inference.
0 commit comments