Model
Model loads a compiled model archive and exposes the route Neat can run.
Use Model when you want model-aware execution without hand-wiring every preprocess, inference, and postprocess stage. Inspect the contract first. Then run it. No mystery tensors.
What Model gives you
input_specs()andoutput_specs()show the tensor contracts the model expects and produces.metadata(),info(), and Pythonsummary()help you inspect the loaded model.run(...)runs one direct inference.build(...)creates a reusable model runner for push/pull execution.graph()returns the model route as a reusableGraphfragment.preprocess(),inference(), andpostprocess()expose route stages for advanced composition.
Reference:
Load a model archive
The examples on this page assume model_path points to a compiled model archive from the Model Compiler, usually a .tar.gz copied to the machine that runs Neat. Use default options first; add ModelOptions only when the contract or input source requires them.
const std::string model_path = "resnet_50_model.tar.gz";
simaai::neat::Model model(model_path);
const auto info = model.info();
const auto inputs = model.input_specs();
const auto outputs = model.output_specs();
If this inspection step surprises you, stop there. Fix the model path, artifact, or contract before writing a bigger app around it.
Choose a model execution path
Pick the path that matches the job. Keep smoke tests small; save graph-level machinery for graph-level problems.
| Need | Use | Returns |
|---|---|---|
| Run one model input once | model.run(...) | TensorList for tensor input, or Sample for sample input |
| Reuse the model route across many inputs | model.build(...) and a model runner | Push/pull control around the model route |
| Measure only model execution | model.benchmark(...) or a measured model runner | BenchmarkReport or MeasureReport |
| Put the model in an app flow | graph.add(model) | A Graph stage |
| Expose route boundaries for composition | model.graph(route_options) | A reusable Graph fragment |
| Debug one model stage | model.fragment(ModelStage::...) | A stage-specific Graph fragment |
Use direct model calls for confidence. Use Graph when the model becomes part of an application: named inputs, source nodes, branches, joins, render, video output, metadata output, or multiple models.
Choose model options
Start with default ModelOptions. Add options only when the model contract or input source requires them.
| Goal | Options to use | Notes |
|---|---|---|
| Send decoded image input | preprocess.kind, preprocess.preset, and specific preprocess fields such as resize, color, layout, normalize, quantize, or tessellate | Use this when your app sends pixels and Neat should adapt them to the model contract. |
| Supply tensors matching the model's public inputs | Leave preprocessing options unset | Preserves required Cast/Quant/Tess adapters, including for resized, normalized FP32 inputs. |
| Decode detection output | decode_type, decode_type_option, score_threshold, nms_iou_threshold, top_k, num_classes | Detection models need explicit decode intent. |
| Keep extracted model files for inspection | cleanup_extracted_model_data = false | Useful while debugging. Leave the default for normal runs. |
| Run several model routes in one process | name_suffix and graph element-name prefix/suffix | Keeps generated names and diagnostics readable. |
| Stop the model route early | inference_terminal | Advanced route-debugging path. Do not use it for first-run code. |
| Tune execution placement or internal queues | processcvu, processmla, advanced_execution | Advanced. Measure the default path before tuning. |
Image input with detection decode
Use this pattern when the model expects image preprocessing and emits YOLO-style detections.
simaai::neat::Model::Options options;
options.preprocess.kind = simaai::neat::InputKind::Image;
options.preprocess.preset = simaai::neat::NormalizePreset::COCO_YOLO;
options.decode_type = simaai::neat::BoxDecodeType::YoloV8;
options.score_threshold = 0.25f;
options.nms_iou_threshold = 0.45f;
options.top_k = 100;
simaai::neat::Model model(model_path, options);
Keep the setup honest: do not set default or deprecated fields unless they change the contract you need. If tensor or image metadata already carries the source format, avoid repeating it unless the page explains why.
Detection decode field guide
Set detection decode options when the route should return decoded boxes, pose results, or segmentation results instead of raw inference tensors. Leave fields unset when you want the raw model outputs.
| Field | Use it when | Default meaning |
|---|---|---|
decode_type | You need Neat to attach a BoxDecode stage for a detection model. | Unspecified; no detection decode intent. |
decode_type_option | The detection head tensor ordering needs a specific variant. | Auto; let the model contract and route planner decide. |
score_threshold | You want to drop low-confidence candidates during decode. | 0; keep candidates before later filtering. |
nms_iou_threshold | The decode path should apply non-max suppression. | 0; keep the model's packaged value. When none is packaged, YOLO26 skips NMS and other decoders suppress every overlapping box. |
top_k | You want a hard cap on detections per output. | 0; no top-K cap. |
num_classes | The class-head depth cannot be inferred reliably, such as a single-class YOLO split head. | 0; use MPK metadata or legacy inference. |
For most YOLO-style models, start with decode_type, score_threshold, nms_iou_threshold, and top_k. Add num_classes only when the model contract needs help. Keep decode_type_option on Auto unless the tensor ordering is known to require a specific selector.
simaai::neat::Model::Options options;
options.decode_type = simaai::neat::BoxDecodeType::YoloV8;
options.score_threshold = 0.25f;
options.nms_iou_threshold = 0.45f;
options.top_k = 100;
options.num_classes = 1; // Set only when the model contract needs the class count.
simaai::neat::Model model(model_path, options);
Do not use boxdecode_original_width or boxdecode_original_height in new examples. Coordinate inversion uses preprocess metadata; preserve that metadata instead of hard-coding image size.
Raw tensor input
Use defaults when supplying tensors matching model.input_specs(). Neat retains the MPK's required tensor adapters.
simaai::neat::Model model(model_path);
The legacy preprocess.enable = Off setting does not select MLA-only execution or remove MPK-required Cast/Quant/Tess adapters. A non-empty preprocess.transforms list takes precedence over Off.
Preprocess field guide
ModelOptions.preprocess states what kind of input you will provide and what Neat may adapt before inference.
| Need | Field |
|---|---|
| Choose image or tensor input | preprocess.kind |
| Legacy preprocessing control (required MPK adapters remain) | preprocess.enable |
| Bound dynamic image input | preprocess.input_max_width, preprocess.input_max_height, preprocess.input_max_depth |
| Resize, crop, or letterbox | preprocess.resize |
| Convert RGB, BGR, NV12, I420, or grayscale | preprocess.color_convert |
| Convert HWC, CHW, or another axis order | preprocess.layout_convert |
| Apply mean/std normalization | preprocess.normalize or preprocess.preset |
| Quantize before inference | preprocess.quantize |
| Tessellate before inference | preprocess.tessellate |
| Override with an ordered transform list | preprocess.transforms |
Set the smallest option set that describes the input you actually send. The specs are the contract; the resolved preprocess plan is the receipt.
Inspect specs and run directly
Before composing a Graph, inspect the model contract. The specs tell you what to allocate, send, and decode.
simaai::neat::Model model(model_path, options);
const auto inputs = model.input_specs();
const auto outputs = model.output_specs();
const auto metadata = model.metadata();
const auto info = model.info();
for (const auto& input : inputs) {
const auto& shape = input.shape;
}
for (const auto& output : outputs) {
const auto& shape = output.shape;
}
simaai::neat::Tensor input = simaai::neat::Tensor::from_cv_mat(
frame,
simaai::neat::ImageSpec::PixelFormat::BGR,
simaai::neat::TensorMemory::CPU);
simaai::neat::TensorList result = model.run(
simaai::neat::TensorList{input},
/*timeout_ms=*/2000);
In Python, pass a list or tuple of inputs. model.run([tensor]) means “one model input.” It does not add a batch dimension.
model.run(...) is one-shot-style, not rebuild-every-call. The first call lazily builds and caches an internal runner; later calls reuse that runner and push/pull through it. Use explicit model.build(...) when you need route options, runtime options, measurement, close/drain control, or a producer/consumer loop.
C++ builds with OpenCV support also expose model.run(std::vector<cv::Mat>{frame}, timeout_ms). Use the explicit Tensor::from_cv_mat(...) path when the example must state pixel format or memory ownership.
Work with multiple inputs and batches
A model can have multiple input ports, a compiled batch size, or both. Do not mix those concepts:
- Multiple inputs means one tensor per model ingress. Inspect
model.input_specs()for the expected order and contracts. - Batch means the model was compiled to process several logical samples per inference. Inspect
model.compiled_batch_size(). - Python input lists choose the ingress list, not the batch dimension.
model.run([tensor])is one ingress. It does not turntensorinto batch size 1.
const int batch = model.compiled_batch_size();
const auto specs = model.input_specs();
// Two-input model: pass one tensor per ingress.
simaai::neat::TensorList inputs{left_tensor, right_tensor};
simaai::neat::TensorList outputs = model.run(inputs, /*timeout_ms=*/2000);
If a batched model expects shape [N, ...], put N in the tensor shape according to the input spec. Do not pass N separate Python list items unless the model has N input ports.
Inspect the route plan
When preprocessing, output topology, or route selection surprises you, inspect the route instead of guessing.
| Question | Inspect |
|---|---|
| What tensor shapes and dtypes does the model expose? | input_specs() and output_specs() |
| What did the compiled artifact declare? | info() and metadata() |
| What input does preprocessing expect? | preprocess_requirements() |
| What preprocess path did Neat compile? | resolved_preprocess_plan() / Python preprocess_plan() |
| Which batch size was compiled? | compiled_batch_size() |
| Which output form should I decode? | output specs plus info().output_topology |
const auto requirements = model.preprocess_requirements();
const auto plan = model.resolved_preprocess_plan();
std::cout << "preprocess: " << plan.to_debug_string() << "\n";
The resolved plan is the audit trail: requested options, effective options, graph family, ingress contracts, MLA contract, and warnings.
Read the model route snapshot
Use info() when you need a compact route snapshot without reading the model archive by hand. Python also provides summary() for a quick text view; C++ uses the structured info() result.
| Question | info() field to inspect |
|---|---|
| Which artifact did I load? | model_name, mpk_json_path |
| Which adapter stages are needed? | needs |
| Which adapter stages are available? | capabilities |
| Did the route include preprocess or postprocess? | selection.include_preprocess_stage, selection.include_postprocess_stage |
| Is this an inference-only route? | selection.infer_only |
| Which preprocess graph or postprocess kind was selected? | selection.preprocess_graph, selection.selected_post_kind |
| How many outputs are physical versus logical? | output_topology.physical_outputs, logical_outputs, packed_outputs |
| What warnings did the route planner produce? | warnings |
const auto info = model.info();
std::cout << "model=" << info.model_name << "\n";
std::cout << "physical_outputs=" << info.output_topology.physical_outputs
<< " logical_outputs=" << info.output_topology.logical_outputs << "\n";
for (const auto& warning : info.warnings) {
std::cerr << warning << "\n";
}
Treat route warnings as first-class evidence. If the route planner is telling you something odd, do not tune around it. Fix the contract or the options first.
See what the model executes
Use execution_plan() to see the nodes a model runs, where each node runs, and the tensors that flow between them. Model::execution_plan(...) takes the same arguments as build(...), builds the model without running a frame, and closes it after reading the plan, so it needs the device.
const simaai::neat::ExecutionPlan plan = model.execution_plan();
std::cout << plan.to_json() << "\n";
The JSON lists the model's graph nodes in pipeline order and one edge between each pair of consecutive nodes.
| Field | Meaning |
|---|---|
model | Model name, batch (frames per MLA run), and memory (device memory per frame in flight) |
nodes[].kind | Node kind, such as Input, Preproc, Quant, ModelFragment, Dequant, SimaBoxDecode, or Output |
nodes[].runs_on | Where the node runs: A65 (Arm cores), EV74 (vision processor), or MLA |
nodes[].kernel | The kernel an EV74 or A65 node executes |
nodes[].settings | Resolved Preproc and SimaBoxDecode settings, named as in PreprocOptions and BoxDecodeOptions |
edges[].tensors | Tensors from one node to the next: batch-first shape, dtype, and quant (scale, zero_point) for quantized tensors |
SimaBoxDecode settings show thresholds and top_k only when the model or your options set them; otherwise the decoder default applies. The schema is schemas/execution_plan_v1.schema.json.
Use a model in a Graph
For the common path, add the model directly between a public input and output.
simaai::neat::Graph graph("classifier");
graph.add(simaai::neat::nodes::Input("image"));
graph.add(model);
graph.add(simaai::neat::nodes::Output("classes"));
A default public Input directly connected to one model image ingress inherits
that ingress's image interpretation and format. You can still set the source's
format explicitly; the graph validates it against the model contract. Source
memory, dimensions, timestamps, and queue options remain unchanged. For multiple
or ambiguous consumers, provide explicit input options.
This interpretation applies at the Graph input boundary. Direct model calls still require inputs that satisfy the model's declared image or tensor contract.
Use model.graph(...) when you need a reusable fragment or explicit route boundaries.
Route options control how that fragment exposes its contract:
| Need | Route option |
|---|---|
| Add a public input node to the returned graph | include_input |
| Add a public output node to the returned graph | include_output |
| Expose separate physical outputs | expose_all_outputs |
| Disambiguate generated element names | name_suffix |
| Connect from a specific upstream element name | upstream_name |
| Override advanced execution for this route | advanced_execution |
simaai::neat::Model::RouteOptions route;
route.include_input = true;
route.include_output = true;
route.name_suffix = "_detector";
simaai::neat::Graph model_fragment = model.graph(route);
Route boundaries are a contract. Add them when you want the fragment to expose public inputs or outputs. Leave them out when the model is just one stage inside a larger graph.
Use stage fragments for advanced composition
Most apps should add the whole model route. Use stage fragments only when you are composing or debugging a route deliberately.
simaai::neat::Graph preprocess =
model.fragment(simaai::neat::Model::Stage::Preprocess);
simaai::neat::Graph inference =
model.fragment(simaai::neat::Model::Stage::Inference);
simaai::neat::Graph postprocess =
model.fragment(simaai::neat::Model::Stage::Postprocess);
Advanced helpers are for diagnostics and deliberate composition. Keep them out of first-run code.
| Helper | Language | Use it when |
|---|---|---|
backend_fragment(stage) | C++ and Python | You need to inspect the backend fragment for one model stage while debugging a route. |
input_appsrc_options(tensor_mode) | C++ and Python | A single-ingress model needs the exact InputOptions Neat derived for an input boundary. |
input_appsrc_options_list(tensor_mode) | C++ | A multi-input model needs per-ingress InputOptions. |
find_config_path_by_plugin(plugin_id) | C++ and Python | You need the extracted config file for a specific plugin while collecting evidence. |
find_config_path_by_processor(processor) | C++ and Python | You need a config file for a processor such as MLA, CVU, or APU. |
infer_output_name() | C++ and Python | You need the canonical inference-stage output element name for diagnostics. |
If a helper exposes backend names or extracted files, treat the result as evidence, not as application control flow. Public Model, Graph, Run, Input, and Output APIs should carry the normal app path.
Tune model execution only after a baseline
Most models should run with default execution settings first. Once you have a correct baseline, use the advanced execution fields to test one change at a time. Change one setting, measure, and keep the default route as your control.
advanced_execution is the preferred option surface for new docs and examples because each field states the intent directly. Unset fields are no-ops. Route-level options override model-level options for that route.
| Need | Field | Notes |
|---|---|---|
| Choose where model-managed preprocessing runs | advanced_execution.preprocess_target | Use target tokens such as "AUTO", "A65", or "EV74" only when you are proving placement. |
| Choose where model-managed postprocessing runs | advanced_execution.postprocess_target | Use this for postprocess stages that can run on more than one target. |
| Allow preprocessing to run asynchronously | advanced_execution.preprocess_async | Measure latency and throughput before and after enabling it. |
| Allow MLA inference to run asynchronously | advanced_execution.inference_async | Useful for streaming workloads that keep work in flight. |
| Adjust inference output buffering | advanced_execution.inference_output_buffers | Increase only when measurement shows output buffering is the bottleneck. |
| Defer output cache synchronization | advanced_execution.defer_output_cache_sync | Advanced. Use only when the consumer and memory lifetime are understood. |
| Adjust internal concurrency | advanced_execution.internal_queue_depth | Bounds internally inserted queues and numerical Runner execution capacity; does not override MLA output buffering. |
For bounded overlap, input lifetime, and CPU-per-frame measurement, see
Overlap frames without growing queues indefinitely.
The retained prepared_runner field is for source compatibility, not a selectable
execution backend. Leave it at its default; retired modes fail at graph construction.
Set advanced execution fields at the narrowest scope that matches the experiment:
simaai::neat::Model::RouteOptions route;
route.name_suffix = "_lane0";
route.advanced_execution.inference_async = true;
auto runner = model.build(route);
Keep the default route as your control. Change one field, measure, export evidence, then decide whether the change earns a place in the app.
C++ also exposes lower-level processcvu, processmla, prepared_runner, and async_queue_depth fields on Model::Options, Model::RouteOptions, and GraphOptions. Prefer advanced_execution in customer examples unless you are preserving existing C++ code or debugging a specific lower-level field.
Reuse a model runner
Use build(...) when you want to avoid rebuilding the route for each input.
auto runner = model.build();
// Convenience path: push one input and wait for its output.
simaai::neat::TensorList result = runner.run(
simaai::neat::TensorList{input},
/*timeout_ms=*/2000);
// Async path: push now, pull later.
runner.push(simaai::neat::TensorList{input});
simaai::neat::Sample sample = runner.pull(/*timeout_ms=*/2000);
runner.close_input();
runner.close();
Use runner.run(...) when each call should push one input and wait for its output. Use push(...) / pull(...) when your app wants to keep work in flight. Use close_input() when no more inputs are coming and you want remaining work to drain. Use close() when the runner is done.
push waits for input-queue space. To submit without that wait, use C++ runner.try_push with a Sample or TensorList, or Python runner.try_push_samples / runner.try_push_tensors. Set RunOptions.queue_depth and overflow_policy before building the runner. When a bounded queue is full, KeepLatest replaces the oldest pending sample and returns true; Block and DropIncoming return false. Closed input also returns false. Keep a frame's paired tensors together in one TensorSet sample. Replacement does not cancel active inference, and the forwarding methods add no payload copy.
Build a route-specific runner
Use a route-specific runner when the reusable runner should use route-level execution settings or route naming that differ from the default route. Model::build(...) adds the runner input and output boundaries for you.
simaai::neat::Model::RouteOptions route;
route.name_suffix = "_detector";
auto runner = model.build(route);
Seed the build when the actual input should prove shape, format, or payload compatibility before the first production push:
auto runner = model.build(simaai::neat::TensorList{input}, route);
Use seeded build for evidence and early failure, not as a habit. If the default build already proves the contract, keep the example smaller.
For image input, the seed supplies the current shape and pixel format; it does not
replace preprocess.input_max_width, input_max_height, or input_max_depth.
For example, a 640×640 seed with a declared 1920×1920 capacity accepts later
800×800 and 1920×1080 frames on the same runner. Frames above the capacity are
rejected. Omitting the limits keeps the same resolved capacity as an unseeded build.
Measure model execution
Use model.benchmark(...) for a quick synthetic model benchmark. It generates inputs from input_specs() and returns headline latency, throughput, power, and board telemetry. Use it to compare model/runtime changes, not to claim production application throughput.
simaai::neat::BenchmarkReport report = model.benchmark();
std::cout << "latency_ms=" << report.latency_ms
<< " fps=" << report.fps << "\n";
For a BoxDecode route, describe the synthetic input's source-image geometry so the benchmark can map detections back from model coordinates:
simaai::neat::BenchmarkOptions options;
options.num_samples = 100;
options.original_width = 1920;
options.original_height = 1080;
options.resize_mode = simaai::neat::ResizeMode::Letterbox;
simaai::neat::BenchmarkReport report = model.benchmark(options);
Set original_width and original_height together. The benchmark still generates a model-shaped
synthetic tensor; these fields supply the source geometry and resize transform required by
BoxDecode metadata. If you omit them, the benchmark infers geometry from the resolved preprocess
and model input contracts. These per-run values take precedence over deprecated BoxDecode geometry
hints in ModelOptions. Existing benchmark(num_samples) calls keep the same behavior.
To also time each MLA layer or stage, set include_mla_layer_stats or include_mla_stage_stats.
To keep the results, report.save(path) writes the report as YAML, or as JSON for a .json
path. Benchmark Your Model shows both.
Measure with your own inputs
Synthetic inputs can misrepresent a model whose cost depends on its data, such as a detection or
segmentation decoder. To measure with real data, set inputs to the tensors you would pass to
run(), one per model input. The benchmark uses them for both the latency and throughput
measurements.
cv::Mat image;
cv::cvtColor(cv::imread("bus.jpg"), image, cv::COLOR_BGR2RGB); // the format input_specs() reports
simaai::neat::BenchmarkOptions options;
options.inputs = simaai::neat::TensorList{
simaai::neat::Tensor::from_cv_mat(image, simaai::neat::ImageSpec::PixelFormat::RGB,
simaai::neat::TensorMemory::EV74)};
simaai::neat::BenchmarkReport report = model.benchmark(options);
std::cout << "input=" << report.input << " fps=" << report.fps << "\n"; // input=provided
If the count, shape or data type of the inputs does not match the model, benchmark() throws
NeatError with error code misconfig.input_shape before it starts. A color format the route
cannot convert fails when the pipeline is built, as it does in run(). On a route that
preprocesses images, the image can have any size. Pass it in the color format that input_specs()
reports.
Run length
The benchmark measures single-flight latency, then runs a separate throughput pass. Each pass
continues until it has measured at least num_samples inputs and min_duration_ms has elapsed
(10 s by default), so both passes stop by the same rule and the throughput pass's power and
temperature readings cover several telemetry samples rather than a short burst. latency_ms is the
mean latency of the latency pass, and latency_p50_ms, latency_p95_ms, and latency_p99_ms are
its percentiles. throughput_samples and throughput_duration_ms report what the throughput pass
measured. For a quick smoke check, set min_duration_ms to 0 to stop each pass after num_samples
inputs:
simaai::neat::BenchmarkOptions options;
options.min_duration_ms = 0;
simaai::neat::BenchmarkReport report = model.benchmark(options);
Power and board telemetry
Power and board telemetry come from Sentinel, the DevKit telemetry daemon that
the Neat installer sets up on the board. The benchmark records them with a
TelemetryMonitor around the throughput pass. It selects samples
after measurement starts and before measurement stops, after all required outputs complete.
Preparation, joining the monitor worker, and the final Sentinel read do not extend the timed
workload. Invalid sample timestamps or detected clock discontinuities make benchmark telemetry
unavailable rather than selecting an unrelated window. report.telemetry
summarizes them:
| Field | Meaning |
|---|---|
avg_power_watts, peak_power_watts | Estimated mean and highest SoM power during the throughput pass. |
energy_joules | Energy to process num_samples inputs: average power multiplied by the time the measured throughput needs for num_samples inputs. A longer pass does not increase it. |
telemetry.power_mla_watts, telemetry.power_rails_watts | MLA rail power, and every rail the board reports. Boards without an MLA rail report it with samples == 0. |
telemetry.die_temperatures_c, telemetry.die_temperature_max_c | On-die sensors at the MLA, APU, CVU, and TOP sites, and the hottest of them in each sample. |
telemetry.board_temperatures_c | SoM top-side and bottom-side sensors. |
telemetry.cpu_usage_pct | CPU utilization. Each sample averages the preceding interval, so maximum is the busiest interval. |
telemetry.cpu_cores_usage_pct | Utilization of each CPU core, in core order. A single busy core shows here even when cpu_usage_pct, the average over all cores, stays low. |
telemetry.mla_memory_mib | Memory the MLA holds, in MiB: buffers from the MLA memory pool, plus shared buffers only the MLA uses. The model allocates it when it loads, so with nothing else running this is the model's MLA memory. |
telemetry.ev74_memory_mib | Memory EV74 holds, in MiB, such as the input frames it preprocesses for the MLA. A shared buffer EV74 and the MLA both use counts here, not in mla_memory_mib, so the two add up to the model's accelerator memory. Frames a video decoder or camera ISP produced count in neither. |
telemetry.linux_memory_used_mib | Memory the whole board uses (Linux MemTotal - MemAvailable), in MiB. It includes every process and EV74's buffers, so do not add it to the two above. |
Each metric reports samples, mean, minimum, maximum, and last, and samples == 0 means
the board did not report it. A reported zero still counts as a sample: with no model loaded,
mla_memory_mib and ev74_memory_mib read 0 MiB with samples > 0. Sentinel samples every 2 s,
so a 10 s pass yields about 5 samples. Every value covers the whole board, including other running
processes.
When Sentinel is not running or the pass is too short to collect a sample, the benchmark still
returns latency and FPS. The power fields stay zero, telemetry.status names the reason (it is
"ok" otherwise), and the printed summary shows Power: unavailable (<reason>).
Measure your own loop
Use runner measurement when you own the input loop and want to measure that loop:
auto runner = model.build();
auto scope = runner.start_measurement();
for (const auto& input : inputs) {
(void)runner.run(simaai::neat::TensorList{input}, /*timeout_ms=*/2000);
}
simaai::neat::MeasureReport report = scope.stop();
runner.close();
Do not compare a one-shot smoke test against a warmed reusable runner. Those answer different questions.
Record board telemetry
Use TelemetryMonitor to record power, temperatures, CPU usage, and memory over any window you
choose, such as your own inference loop. It reads them from Sentinel and
returns the same Telemetry summary as report.telemetry:
simaai::neat::TelemetryMonitor monitor;
monitor.start();
for (const auto& input : inputs) {
(void)runner.run(simaai::neat::TensorList{input}, /*timeout_ms=*/2000);
}
simaai::neat::Telemetry telemetry = monitor.stop();
std::cout << telemetry; // YAML
telemetry.save("telemetry.json"); // .json writes JSON; other names write YAML
By default, the monitor reads /run/simaai-sentinel/api.sock. To use a Sentinel Unix socket
mounted at another path, pass its path to TelemetryMonitor(socket_path) in C++ or
pyneat.TelemetryMonitor(socket_path="...") in Python. This selects the endpoint for that monitor
only; it does not change Model::benchmark() or start Sentinel. Construction does not contact
the socket.
start()begins a new window. Only samples Sentinel takes after it count, and Sentinel samples every 2 s, so a window shorter than that can have none;statusthen says so.read()returns the summary so far without stopping. Each measurement'slastis its newest value, andmean,minimum, andmaximumcover the window so far.stop()returns the final summary. Calling it again returns the same result.- The summary keeps one entry per measurement, so memory use stays the same however long the window runs.
- Several monitors, in one process or many, can run at the same time as each other and as
Model::benchmark(). - Every value covers the whole board, including Sentinel, the monitor, and other processes. Each power value comes from one 250 ms reading per sample, so short spikes between readings do not show.
- When Sentinel is not running, the monitor prints one warning,
statusexplains how to install or start Sentinel, and the monitor counts samples once Sentinel answers. It never throws for Sentinel problems. - In Python,
start(),read(), andstop()release the GIL while they wait.
Decode output
Start by reading the output specs. Decode only when the model output contract says the payload is a detection, pose, or segmentation format.
| Output contract | Use |
|---|---|
| Raw tensor | Use the returned tensor directly, or convert it with to_numpy(...) / to_torch(...) in Python. |
| Packed boxes as tensors | C++ simaai::neat::decode_bbox(...) / Python pyneat.decode_bbox(...) |
| Packed boxes as typed records | C++ simaai::neat::decode_bbox_tensor(...) / Python pyneat.detections.decode_bbox_tensor(...) |
| Packed pose | C++ simaai::neat::decode_pose(...) / Python pyneat.decode_pose(...) |
| Packed segmentation | C++ simaai::neat::decode_segmentation(...) / Python pyneat.decode_segmentation(...) |
Use tensor decode when the next stage wants tensors. Use typed decode when application code wants records with x1, y1, x2, y2, score, and class_id.
const auto decoded = simaai::neat::decode_bbox_tensor(
outputs[0],
image_width,
image_height,
/*expected_topk=*/100,
/*strict=*/true);
for (const auto& box : decoded.boxes) {
handle_box(box.x1, box.y1, box.x2, box.y2, box.score, box.class_id);
}
Use decode helpers only on the output formats they expect.
Failure handling
Model-driven graphs use the same diagnostics contract as raw Graph:
NeatError.report().error_codefor terminal failures.NeatError.report().repro_notefor actionable context and hints.NeatError.report().busfor plugin or runtime detail.
try {
auto result = model.run(simaai::neat::TensorList{input}, /*timeout_ms=*/2000);
} catch (const simaai::neat::NeatError& error) {
const auto& report = error.report();
std::cerr << report.error_code << "\n";
std::cerr << report.repro_note << "\n";
}
Start with error_code values such as misconfig.*, build.*, runtime.*, or io.*. Read bus logs only after the structured error points you there.