Models & Inference
Run models, configure preprocessing and postprocessing, exchange tensors, and measure inference. Use these tutorials in order.
Load a compiled ResNet-50 archive, feed it an image, and read the top-1 class — the shortest path from "I have a model archiv...
Feed a model from a producer thread while consuming predictions from another, decoupling input and output for real throughput...
Run a compiled model with deterministic synthetic tensors and print the headline latency, throughput, power, and energy numbe...
`ModelOptions` is the one struct that declares the contract between your input data, the model's pipeline stages, and its out...
Configure the preprocessing stage — format, dimensions, and per-channel normalization — so raw image input becomes the exact...
Decode raw model output into usable bounding boxes using `SimaBoxDecode` — thresholding, NMS, and coordinate mapping built in...
Move data between Neat tensors and the structures you already have — NumPy arrays, PyTorch tensors, or `cv::Mat` — controllin...
Bundle several named tensors into one `Sample` and push it as a single inference event — the pattern for models that take mor...
Read back from `run.pull()` or `model.run()` safely. Every run returns a `Sample` — a small sum type that may be a tensor, a...