BenchmarkOptions Struct
Options for Model::benchmark() inputs and measurement. More...
Declaration
Included Headers
Public Member Attributes Index
| int | num_samples = 100 |
|
Number of measured inputs; must be greater than zero. More... | |
| std::optional< int > | original_width |
|
Source-image width, in pixels. More... | |
| std::optional< int > | original_height |
|
Source-image height, in pixels. More... | |
| std::optional< ResizeMode > | resize_mode |
|
Resize policy used to map the source geometry to the model input. More... | |
| bool | include_plugin_latency = false |
|
Deprecated: plugin latency will be removed. More... | |
| int | min_duration_ms = 10000 |
|
Minimum length of each pass, latency and throughput, in milliseconds; must be zero or greater. More... | |
| bool | include_mla_layer_stats = false |
|
Advanced: measure each compiled MLA layer's hardware latency with mla-rt after the benchmark. More... | |
| bool | include_mla_stage_stats = false |
|
Advanced: measure each MLA stage's uninterrupted hardware latency with mla-rt after the benchmark. More... | |
| TensorList | inputs |
|
Inputs pushed for every sample: one tensor per model input, in input_specs() order, as run() takes them. More... | |
Description
Options for Model::benchmark() inputs and measurement.
Set inputs to measure with real data, such as an image read with OpenCV. Without it, the benchmark generates deterministic synthetic inputs from Model::input_specs().
BoxDecode routes need source-image geometry to map detections from model coordinates back to source coordinates. Set original_width and original_height together to benchmark a specific source geometry. When they are omitted, the benchmark infers geometry from the resolved preprocess and model ingress contracts, preserving the behavior of benchmark(num_samples). Image preprocessing routes allocate the synthetic image at that source geometry; a provided image supplies its own geometry, so the original size must be omitted or match it. A provided tensor that already carries Tensor::semantic.preprocess keeps it, as run() does, and the original size and resize_mode do not apply to it. Tensor routes (including cast/quantize) retain the model input shape; the source geometry is used only for BoxDecode metadata. Per-run benchmark geometry takes precedence over deprecated model-construction BoxDecode hints.
Definition at line 164 of file Model.h.
Public Member Attributes
include_mla_layer_stats
|
Advanced: measure each compiled MLA layer's hardware latency with mla-rt after the benchmark.
Default off. Requires mla-rt with the MIP-to-kernel handoff fix in PATH and an idle devkit with no other active MLA workloads. Only MLA stages in the selected inference route are measured. A failure leaves the measurements intact and names the reason in BenchmarkReport::mla_layer_stats.status.
Definition at line 182 of file Model.h.
include_mla_stage_stats
|
Advanced: measure each MLA stage's uninterrupted hardware latency with mla-rt after the benchmark.
Default off. Requires mla-rt in PATH and an idle devkit with no other active MLA workloads. A failure leaves the measurements intact and names the reason in BenchmarkReport::mla_stage_stats.status.
Definition at line 187 of file Model.h.
include_plugin_latency
|
Deprecated: plugin latency will be removed.
Definition at line 171 of file Model.h.
inputs
|
Inputs pushed for every sample: one tensor per model input, in input_specs() order, as run() takes them.
Empty selects synthetic inputs. Inputs whose count, shape or data type does not match the model throw NeatError with error code misconfig.input_shape before the benchmark starts; a color format the route cannot convert fails when the pipeline is built.
Definition at line 192 of file Model.h.
min_duration_ms
|
Minimum length of each pass, latency and throughput, in milliseconds; must be zero or greater.
Each pass continues past num_samples inputs until this much time has elapsed, so both passes follow the same rule and the throughput pass's power and temperature readings cover several telemetry samples. Set it to 0 to stop each pass after num_samples inputs.
Definition at line 176 of file Model.h.
num_samples
|
Number of measured inputs; must be greater than zero.
Definition at line 165 of file Model.h.
original_height
|
Source-image height, in pixels.
Definition at line 167 of file Model.h.
original_width
|
Source-image width, in pixels.
Definition at line 166 of file Model.h.
resize_mode
|
Resize policy used to map the source geometry to the model input.
When omitted, use the resolved preprocess policy, or stretch when the model route has no resize stage.
Definition at line 170 of file Model.h.
The documentation for this struct was generated from the following file:
Generated via doxygen2docusaurus 2.0.0 by Doxygen 1.15.0.