Run Your First Model over PCIe

Field	Value
Category	PCIe Co-Processing
Difficulty	Beginner
Estimated Read Time	15 minutes
Labels	PCIe, inference, tensor, image, detection

Run three independent programs with the same YOLOv8s archive and 640x480 street scene. Each program demonstrates one mode, uses queue 0 synchronously, and closes one model. This keeps every example short enough to copy on its own.

Walkthrough

Run a model-ready tensor

The host letterboxes the image to the model's reported [640, 640, 3] input, converts BGR to RGB, and scales pixels to [0, 1]. Model.run() sends that FP32 tensor without card-side image preprocessing and prints all six raw YOLO output routes.

pcie_host/tutorials/024_run_your_first_model_over_pcie/run_tensor_mode.cpp

pcie::ConnectionOptions connection;
connection.card_id = card_id;
pcie::Model model(kModelPath, {}, connection);
const auto info = model.info();
if (info.inputs.size() != 1) {
  throw std::runtime_error("YOLOv8s must expose one input tensor");
}
const pcie::Tensor input = make_yolo_tensor(image, info.inputs[0]);
model.build(kBuildTimeoutMs);
const auto outputs = model.run(input, kRunTimeoutMs);
if (outputs.empty()) {
  throw std::runtime_error("tensor mode returned no outputs");
}
std::cout << "Tensor mode raw outputs:\n";
for (const auto& output : outputs) {
  std::cout << "  " << output.route.name << " " << dtype_name(output.dtype) << " "
            << shape_string(output.shape) << '\n';
}
model.close();

Move preprocessing to the card

Set preprocess.kind to Image, identify the incoming pixels as BGR, and select the COCO_YOLO preset. The host now sends decoded pixels while the card performs letterbox resize, BGR-to-RGB conversion, and normalization. The program prints the six raw output route names and shapes so you can compare them with tensor mode.

pcie_host/tutorials/024_run_your_first_model_over_pcie/run_image_mode.cpp

pcie::ConnectionOptions connection;
connection.card_id = card_id;
pcie::ModelOptions options;
options.preprocess.kind = pcie::InputKind::Image;
options.preprocess.color_convert.input_format = pcie::ColorFormat::BGR;
options.preprocess.color_convert.output_format = pcie::ColorFormat::RGB;
options.preprocess.resize.enable = pcie::AutoFlag::On;
options.preprocess.resize.mode = pcie::ResizeMode::Letterbox;
options.preprocess.normalize.preset = pcie::NormalizePreset::COCO_YOLO;
pcie::Model model(kModelPath, options, connection);
model.build(kBuildTimeoutMs);
const auto outputs = model.run(image, kRunTimeoutMs);
if (outputs.empty()) {
  throw std::runtime_error("image mode returned no outputs");
}
std::cout << "Image mode raw outputs:\n";
for (const auto& output : outputs) {
  std::cout << "  " << output.route.name << " " << dtype_name(output.dtype) << " "
            << shape_string(output.shape) << '\n';
}
model.close();

Decode detections on the card

Add BoxDecodeType.YoloV8, a score threshold, NMS threshold, and output limit. The returned BBOX tensor begins with a detection count followed by fixed-size records containing (x, y, width, height, score, class_id). The example parses and prints the first ten records in source-image coordinates.

pcie_host/tutorials/024_run_your_first_model_over_pcie/run_image_boxdecode.cpp

pcie::ConnectionOptions connection;
connection.card_id = card_id;
pcie::ModelOptions options;
options.preprocess.kind = pcie::InputKind::Image;
options.preprocess.color_convert.input_format = pcie::ColorFormat::BGR;
options.preprocess.color_convert.output_format = pcie::ColorFormat::RGB;
options.preprocess.resize.enable = pcie::AutoFlag::On;
options.preprocess.resize.mode = pcie::ResizeMode::Letterbox;
options.preprocess.normalize.preset = pcie::NormalizePreset::COCO_YOLO;
options.decode_type = pcie::BoxDecodeType::YoloV8;
options.score_threshold = 0.25F;
options.nms_iou_threshold = 0.45F;
options.top_k = 100;
pcie::Model model(kModelPath, options, connection);
model.build(kBuildTimeoutMs);
const auto boxes = parse_boxes(model.run(image, kRunTimeoutMs));
std::cout << "Image + boxdecode detections=" << boxes.size() << '\n';
for (std::size_t index = 0; index < std::min<std::size_t>(boxes.size(), 10); ++index) {
  const auto& box = boxes[index];
  std::cout << "  " << class_name(box.class_id) << " score=" << std::fixed
            << std::setprecision(3) << box.score << " box=(" << box.x << ", " << box.y << ", "
            << box.width << ", " << box.height << ")\n";
}
if (boxes.empty()) {
  throw std::runtime_error("no detections passed the score threshold");
}
model.close();

Parse the BBOX tensor

Validate that box decode returned one populated tensor, read its leading count, and reject a count that exceeds the payload. Each remaining 24-byte record is then converted to one detection for printing.

pcie_host/tutorials/024_run_your_first_model_over_pcie/run_image_boxdecode.cpp

std::vector<Box> parse_boxes(const pcie::TensorList& outputs) {
  if (outputs.size() != 1 || outputs[0].data == nullptr || outputs[0].byte_offset < 0) {
    throw std::runtime_error("boxdecode must return one populated BBOX tensor");
  }
  const auto& tensor = outputs[0];
  const auto offset = static_cast<std::size_t>(tensor.byte_offset);
  if (offset > tensor.size_bytes || tensor.size_bytes - offset < sizeof(std::uint32_t)) {
    throw std::runtime_error("BBOX tensor is too small");
  }
  const auto* bytes = static_cast<const std::uint8_t*>(tensor.data) + offset;
  const std::size_t available = tensor.size_bytes - offset;
  const std::uint32_t count = read_value<std::uint32_t>(bytes);
  constexpr std::size_t record_size = 24;
  if (count > (available - 4) / record_size) {
    throw std::runtime_error("BBOX detection count exceeds its payload");
  }
  std::vector<Box> boxes;
  boxes.reserve(count);
  for (std::uint32_t index = 0; index < count; ++index) {
    const auto* record = bytes + 4 + index * record_size;
    boxes.push_back({read_value<std::int32_t>(record), read_value<std::int32_t>(record + 4),
                     read_value<std::int32_t>(record + 8), read_value<std::int32_t>(record + 12),
                     read_value<float>(record + 16), read_value<std::int32_t>(record + 20)});
  }
  return boxes;
}

Run

Install the PCIe host package and download the tutorial bundle as described in Tutorial Setup. Run the following commands from the extracted PCIe extras root:

sima-cli modelzoo get yolo_v8s

The programs require yolo_v8s_mpk.tar.gz in this directory. Model Zoo output names and locations can vary. If the command did not create that exact path, copy the downloaded archive into place and verify it:

cp /absolute/path/to/downloaded-yolov8s-archive.tar.gz yolo_v8s_mpk.tar.gz
test -f yolo_v8s_mpk.tar.gz

C++ (prebuilt):

./lib/sima-pcie-host/tutorials/tutorial_024_run_tensor_mode
./lib/sima-pcie-host/tutorials/tutorial_024_run_image_mode
./lib/sima-pcie-host/tutorials/tutorial_024_run_image_boxdecode

C++ (build from source):

./build.sh --target tutorial_024_run_tensor_mode
./build.sh --target tutorial_024_run_image_mode
./build.sh --target tutorial_024_run_image_boxdecode

./build/tutorials-standalone/tutorial_024_run_tensor_mode
./build/tutorials-standalone/tutorial_024_run_image_mode
./build/tutorials-standalone/tutorial_024_run_image_boxdecode

The matching C++ and Python programs print the same six raw output contracts for tensor and image mode, followed by decoded people, cars, or other visible objects:

Tensor mode raw outputs:
  bbox_0 FP32 [80, 80, 64]
  ...
[OK] 024_run_tensor_mode
Image mode raw outputs:
  bbox_0 FP32 [80, 80, 64]
  ...
[OK] 024_run_image_mode
Image + boxdecode detections=...
  person score=... box=(...)
[OK] 024_run_image_boxdecode

The default is card 0 and queue 0. Pass --card N only when using another card; its management address is derived automatically.

In Practice

Use tensor mode when your application already produces exactly the dtype, shape, layout, color order, and numeric range reported by model.info(). Use image mode when the application naturally owns decoded pixels and you want the card to apply repeatable model preprocessing. Enable box decode when the application needs detections rather than raw feature maps.

Every mode uses the same pcie::Model/pyneatpcie.Model lifecycle. Only ModelOptions and the submitted payload change. Continue with Run PCIe Inference Async to overlap submission and completion with push() and pull().

Full source

Show the complete source programs

Tensor Mode

pcie_host/tutorials/024_run_your_first_model_over_pcie/run_tensor_mode.cpp

// Run YOLOv8s tensor-mode inference over PCIe.
//
// Usage:
//   tutorial_024_run_tensor_mode [--card 0]

#include <simaai/neat/pcie/Model.h>

#include <opencv2/imgcodecs.hpp>
#include <opencv2/imgproc.hpp>

#include <algorithm>
#include <cmath>
#include <cstdint>
#include <cstdlib>
#include <filesystem>
#include <iostream>
#include <stdexcept>
#include <string>
#include <utility>
#include <vector>

namespace pcie = simaai::neat::pcie;

namespace {

constexpr int kBuildTimeoutMs = 180000;
constexpr int kRunTimeoutMs = 30000;
constexpr char kModelPath[] = "yolo_v8s_mpk.tar.gz";
constexpr char kImagePath[] = "share/sima-pcie-host/tutorials/assets/street-scene.png";

int parse_card(const int argc, char** argv) {
  int card_id = 0;
  for (int index = 1; index < argc; ++index) {
    const std::string arg = argv[index];
    if (arg == "--card" && index + 1 < argc) {
      card_id = std::stoi(argv[++index]);
    } else if (arg == "-h" || arg == "--help") {
      std::cout << "Usage: " << argv[0] << " [--card 0]\n";
      std::exit(0);
    } else {
      throw std::runtime_error("unknown or incomplete argument: " + arg);
    }
  }
  return card_id;
}

std::string shape_string(const std::vector<std::int64_t>& shape) {
  std::string text = "[";
  for (std::size_t index = 0; index < shape.size(); ++index) {
    text += (index == 0 ? "" : ", ") + std::to_string(shape[index]);
  }
  return text + "]";
}

const char* dtype_name(const pcie::TensorDType dtype) {
  switch (dtype) {
  case pcie::TensorDType::UInt8:
    return "UINT8";
  case pcie::TensorDType::Int8:
    return "INT8";
  case pcie::TensorDType::UInt16:
    return "UINT16";
  case pcie::TensorDType::Int16:
    return "INT16";
  case pcie::TensorDType::Int32:
    return "INT32";
  case pcie::TensorDType::BFloat16:
    return "BF16";
  case pcie::TensorDType::Float32:
    return "FP32";
  case pcie::TensorDType::Float64:
    return "FP64";
  }
  return "UNKNOWN";
}

pcie::Tensor make_yolo_tensor(const cv::Mat& bgr, const pcie::TensorInfo& input) {
  if (input.shape.size() != 3 || input.shape[2] != 3) {
    throw std::runtime_error("expected a three-channel HWC YOLO input");
  }
  const int target_height = static_cast<int>(input.shape[0]);
  const int target_width = static_cast<int>(input.shape[1]);
  const double scale = std::min(static_cast<double>(target_width) / bgr.cols,
                                static_cast<double>(target_height) / bgr.rows);
  const int resized_width = std::max(1, static_cast<int>(std::round(bgr.cols * scale)));
  const int resized_height = std::max(1, static_cast<int>(std::round(bgr.rows * scale)));

  cv::Mat resized;
  cv::resize(bgr, resized, cv::Size(resized_width, resized_height));
  cv::Mat letterboxed(target_height, target_width, CV_8UC3, cv::Scalar(114, 114, 114));
  const int left = (target_width - resized_width) / 2;
  const int top = (target_height - resized_height) / 2;
  resized.copyTo(letterboxed(cv::Rect(left, top, resized_width, resized_height)));

  cv::Mat rgb;
  cv::cvtColor(letterboxed, rgb, cv::COLOR_BGR2RGB);
  cv::Mat normalized;
  rgb.convertTo(normalized, CV_32FC3, 1.0 / 255.0);
  if (!normalized.isContinuous()) {
    normalized = normalized.clone();
  }
  const auto* begin = normalized.ptr<float>();
  std::vector<float> values(begin, begin + normalized.total() * normalized.channels());
  return pcie::Tensor::from_vector(std::move(values), input.shape, input.name);
}

} // namespace

int main(int argc, char** argv) {
  try {
    const int card_id = parse_card(argc, argv);
    if (!std::filesystem::is_regular_file(kModelPath)) {
      throw std::runtime_error(std::string("model does not exist: ") + kModelPath);
    }
    const cv::Mat image = cv::imread(kImagePath, cv::IMREAD_COLOR);
    if (image.empty()) {
      throw std::runtime_error(std::string("OpenCV could not decode: ") + kImagePath);
    }

    // CORE LOGIC
    pcie::ConnectionOptions connection;
    connection.card_id = card_id;
    pcie::Model model(kModelPath, {}, connection);
    const auto info = model.info();
    if (info.inputs.size() != 1) {
      throw std::runtime_error("YOLOv8s must expose one input tensor");
    }
    const pcie::Tensor input = make_yolo_tensor(image, info.inputs[0]);
    model.build(kBuildTimeoutMs);
    const auto outputs = model.run(input, kRunTimeoutMs);
    if (outputs.empty()) {
      throw std::runtime_error("tensor mode returned no outputs");
    }
    std::cout << "Tensor mode raw outputs:\n";
    for (const auto& output : outputs) {
      std::cout << "  " << output.route.name << " " << dtype_name(output.dtype) << " "
                << shape_string(output.shape) << '\n';
    }
    model.close();

    std::cout << "[OK] 024_run_tensor_mode\n";
    return 0;
  } catch (const std::exception& error) {
    std::cerr << "[FAIL] " << error.what() << '\n';
    return 1;
  }
}

Image Mode

pcie_host/tutorials/024_run_your_first_model_over_pcie/run_image_mode.cpp

// Run YOLOv8s image-mode inference over PCIe.
//
// Usage:
//   tutorial_024_run_image_mode [--card 0]

#include <simaai/neat/pcie/Model.h>

#include <opencv2/imgcodecs.hpp>

#include <cstdint>
#include <cstdlib>
#include <filesystem>
#include <iostream>
#include <stdexcept>
#include <string>
#include <vector>

namespace pcie = simaai::neat::pcie;

namespace {

constexpr int kBuildTimeoutMs = 180000;
constexpr int kRunTimeoutMs = 30000;
constexpr char kModelPath[] = "yolo_v8s_mpk.tar.gz";
constexpr char kImagePath[] = "share/sima-pcie-host/tutorials/assets/street-scene.png";

int parse_card(const int argc, char** argv) {
  int card_id = 0;
  for (int index = 1; index < argc; ++index) {
    const std::string arg = argv[index];
    if (arg == "--card" && index + 1 < argc) {
      card_id = std::stoi(argv[++index]);
    } else if (arg == "-h" || arg == "--help") {
      std::cout << "Usage: " << argv[0] << " [--card 0]\n";
      std::exit(0);
    } else {
      throw std::runtime_error("unknown or incomplete argument: " + arg);
    }
  }
  return card_id;
}

std::string shape_string(const std::vector<std::int64_t>& shape) {
  std::string text = "[";
  for (std::size_t index = 0; index < shape.size(); ++index) {
    text += (index == 0 ? "" : ", ") + std::to_string(shape[index]);
  }
  return text + "]";
}

const char* dtype_name(const pcie::TensorDType dtype) {
  switch (dtype) {
  case pcie::TensorDType::UInt8:
    return "UINT8";
  case pcie::TensorDType::Int8:
    return "INT8";
  case pcie::TensorDType::UInt16:
    return "UINT16";
  case pcie::TensorDType::Int16:
    return "INT16";
  case pcie::TensorDType::Int32:
    return "INT32";
  case pcie::TensorDType::BFloat16:
    return "BF16";
  case pcie::TensorDType::Float32:
    return "FP32";
  case pcie::TensorDType::Float64:
    return "FP64";
  }
  return "UNKNOWN";
}

} // namespace

int main(int argc, char** argv) {
  try {
    const int card_id = parse_card(argc, argv);
    if (!std::filesystem::is_regular_file(kModelPath)) {
      throw std::runtime_error(std::string("model does not exist: ") + kModelPath);
    }
    const cv::Mat image = cv::imread(kImagePath, cv::IMREAD_COLOR);
    if (image.empty()) {
      throw std::runtime_error(std::string("OpenCV could not decode: ") + kImagePath);
    }

    // CORE LOGIC
    pcie::ConnectionOptions connection;
    connection.card_id = card_id;
    pcie::ModelOptions options;
    options.preprocess.kind = pcie::InputKind::Image;
    options.preprocess.color_convert.input_format = pcie::ColorFormat::BGR;
    options.preprocess.color_convert.output_format = pcie::ColorFormat::RGB;
    options.preprocess.resize.enable = pcie::AutoFlag::On;
    options.preprocess.resize.mode = pcie::ResizeMode::Letterbox;
    options.preprocess.normalize.preset = pcie::NormalizePreset::COCO_YOLO;
    pcie::Model model(kModelPath, options, connection);
    model.build(kBuildTimeoutMs);
    const auto outputs = model.run(image, kRunTimeoutMs);
    if (outputs.empty()) {
      throw std::runtime_error("image mode returned no outputs");
    }
    std::cout << "Image mode raw outputs:\n";
    for (const auto& output : outputs) {
      std::cout << "  " << output.route.name << " " << dtype_name(output.dtype) << " "
                << shape_string(output.shape) << '\n';
    }
    model.close();

    std::cout << "[OK] 024_run_image_mode\n";
    return 0;
  } catch (const std::exception& error) {
    std::cerr << "[FAIL] " << error.what() << '\n';
    return 1;
  }
}

Image Boxdecode

pcie_host/tutorials/024_run_your_first_model_over_pcie/run_image_boxdecode.cpp

// Run YOLOv8s image inference with card-side box decode over PCIe.
//
// Usage:
//   tutorial_024_run_image_boxdecode [--card 0]

#include <simaai/neat/pcie/Model.h>

#include <opencv2/imgcodecs.hpp>

#include <algorithm>
#include <cstdint>
#include <cstring>
#include <cstdlib>
#include <filesystem>
#include <iomanip>
#include <iostream>
#include <stdexcept>
#include <string>
#include <vector>

namespace pcie = simaai::neat::pcie;

namespace {

constexpr int kBuildTimeoutMs = 180000;
constexpr int kRunTimeoutMs = 30000;
constexpr char kModelPath[] = "yolo_v8s_mpk.tar.gz";
constexpr char kImagePath[] = "share/sima-pcie-host/tutorials/assets/street-scene.png";

int parse_card(const int argc, char** argv) {
  int card_id = 0;
  for (int index = 1; index < argc; ++index) {
    const std::string arg = argv[index];
    if (arg == "--card" && index + 1 < argc) {
      card_id = std::stoi(argv[++index]);
    } else if (arg == "-h" || arg == "--help") {
      std::cout << "Usage: " << argv[0] << " [--card 0]\n";
      std::exit(0);
    } else {
      throw std::runtime_error("unknown or incomplete argument: " + arg);
    }
  }
  return card_id;
}

struct Box {
  int x;
  int y;
  int width;
  int height;
  float score;
  int class_id;
};

template <typename T> T read_value(const std::uint8_t* data) {
  T value{};
  std::memcpy(&value, data, sizeof(value));
  return value;
}

std::vector<Box> parse_boxes(const pcie::TensorList& outputs) {
  if (outputs.size() != 1 || outputs[0].data == nullptr || outputs[0].byte_offset < 0) {
    throw std::runtime_error("boxdecode must return one populated BBOX tensor");
  }
  const auto& tensor = outputs[0];
  const auto offset = static_cast<std::size_t>(tensor.byte_offset);
  if (offset > tensor.size_bytes || tensor.size_bytes - offset < sizeof(std::uint32_t)) {
    throw std::runtime_error("BBOX tensor is too small");
  }
  const auto* bytes = static_cast<const std::uint8_t*>(tensor.data) + offset;
  const std::size_t available = tensor.size_bytes - offset;
  const std::uint32_t count = read_value<std::uint32_t>(bytes);
  constexpr std::size_t record_size = 24;
  if (count > (available - 4) / record_size) {
    throw std::runtime_error("BBOX detection count exceeds its payload");
  }
  std::vector<Box> boxes;
  boxes.reserve(count);
  for (std::uint32_t index = 0; index < count; ++index) {
    const auto* record = bytes + 4 + index * record_size;
    boxes.push_back({read_value<std::int32_t>(record), read_value<std::int32_t>(record + 4),
                     read_value<std::int32_t>(record + 8), read_value<std::int32_t>(record + 12),
                     read_value<float>(record + 16), read_value<std::int32_t>(record + 20)});
  }
  return boxes;
}

std::string class_name(const int class_id) {
  switch (class_id) {
  case 0:
    return "person";
  case 1:
    return "bicycle";
  case 2:
    return "car";
  case 3:
    return "motorcycle";
  case 5:
    return "bus";
  case 7:
    return "truck";
  default:
    return "class_" + std::to_string(class_id);
  }
}

} // namespace

int main(int argc, char** argv) {
  try {
    const int card_id = parse_card(argc, argv);
    if (!std::filesystem::is_regular_file(kModelPath)) {
      throw std::runtime_error(std::string("model does not exist: ") + kModelPath);
    }
    const cv::Mat image = cv::imread(kImagePath, cv::IMREAD_COLOR);
    if (image.empty()) {
      throw std::runtime_error(std::string("OpenCV could not decode: ") + kImagePath);
    }

    // CORE LOGIC
    pcie::ConnectionOptions connection;
    connection.card_id = card_id;
    pcie::ModelOptions options;
    options.preprocess.kind = pcie::InputKind::Image;
    options.preprocess.color_convert.input_format = pcie::ColorFormat::BGR;
    options.preprocess.color_convert.output_format = pcie::ColorFormat::RGB;
    options.preprocess.resize.enable = pcie::AutoFlag::On;
    options.preprocess.resize.mode = pcie::ResizeMode::Letterbox;
    options.preprocess.normalize.preset = pcie::NormalizePreset::COCO_YOLO;
    options.decode_type = pcie::BoxDecodeType::YoloV8;
    options.score_threshold = 0.25F;
    options.nms_iou_threshold = 0.45F;
    options.top_k = 100;
    pcie::Model model(kModelPath, options, connection);
    model.build(kBuildTimeoutMs);
    const auto boxes = parse_boxes(model.run(image, kRunTimeoutMs));
    std::cout << "Image + boxdecode detections=" << boxes.size() << '\n';
    for (std::size_t index = 0; index < std::min<std::size_t>(boxes.size(), 10); ++index) {
      const auto& box = boxes[index];
      std::cout << "  " << class_name(box.class_id) << " score=" << std::fixed
                << std::setprecision(3) << box.score << " box=(" << box.x << ", " << box.y << ", "
                << box.width << ", " << box.height << ")\n";
    }
    if (boxes.empty()) {
      throw std::runtime_error("no detections passed the score threshold");
    }
    model.close();

    std::cout << "[OK] 024_run_image_boxdecode\n";
    return 0;
  } catch (const std::exception& error) {
    std::cerr << "[FAIL] " << error.what() << '\n';
    return 1;
  }
}

Source

• C++ — Tensor Mode
• C++ — Image Mode
• C++ — Image Boxdecode
• Python — Tensor Mode
• Python — Image Mode
• Python — Image Boxdecode
• README