PCIe Co-Processing

PCIe co-processing keeps application control and input preparation on the host machine while a connected Modalix PCIe Card executes the model. In this example, the host reads, resizes, and normalizes an image, sends the prepared tensor to the card for ResNet-50 inference, and prints the returned classification output.

Before you begin

You need:

• the Neat PCIe host package installed on the host machine;
• a compatible Neat Library release installed on the card; and
• the card management interface reachable from the host. Card 0 uses 10.0.0.2 by default.

Get the model and image

Create a working directory on the host machine and download ResNet-50 from the Model Zoo:

pcie-host$mkdir -p pcie-host-quickstart/assetspcie-host$cd pcie-host-quickstartpcie-host$sima-cli modelzoo get resnet_50

Place the downloaded archive in this directory as resnet_50.tar.gz.

Download the sample Labrador image and save it as assets/sample.png.

The image is based on YellowLabradorLooking new.jpg, photographed by Elf and modified by Djmirko and FT2, and is licensed under CC BY-SA 3.0.

Your working directory should now contain:

pcie-host-quickstart/
├── assets/
│   └── sample.png
└── resnet_50.tar.gz

Create the application

Choose Python or C++ and create the application in the working directory.

Create pcie_host.cpp:

pcie_host.cpp

#include <simaai/neat/pcie/Model.h>

#include <opencv2/imgcodecs.hpp>
#include <opencv2/imgproc.hpp>

#include <algorithm>
#include <cstdint>
#include <iostream>
#include <iterator>
#include <stdexcept>
#include <string>
#include <utility>
#include <vector>

namespace pcie = simaai::neat::pcie;

pcie::Tensor load_input(const std::string& path,
                        const std::string& route_name) {
  cv::Mat bgr = cv::imread(path, cv::IMREAD_COLOR);
  if (bgr.empty())
    throw std::runtime_error("failed to read image: " + path);

  cv::resize(bgr, bgr, cv::Size(224, 224), 0, 0, cv::INTER_AREA);
  cv::Mat rgb;
  cv::cvtColor(bgr, rgb, cv::COLOR_BGR2RGB);
  if (!rgb.isContinuous())
    rgb = rgb.clone();

  constexpr float mean[] = {0.485F, 0.456F, 0.406F};
  constexpr float stddev[] = {0.229F, 0.224F, 0.225F};
  std::vector<float> input(rgb.total() * rgb.channels());
  for (int row = 0; row < rgb.rows; ++row) {
    const auto* pixels = rgb.ptr<std::uint8_t>(row);
    for (int col = 0; col < rgb.cols; ++col) {
      for (int channel = 0; channel < 3; ++channel) {
        const std::size_t index =
            (static_cast<std::size_t>(row) * rgb.cols + col) * 3 + channel;
        const float value = pixels[col * 3 + channel] / 255.0F;
        input[index] = (value - mean[channel]) / stddev[channel];
      }
    }
  }
  return pcie::Tensor::from_vector(
      std::move(input), {rgb.rows, rgb.cols, rgb.channels()}, route_name);
}

int main() {
  pcie::ConnectionOptions connection;
  connection.card_host = "10.0.0.2";
  connection.card_id = 0;
  connection.queue = 0;

  pcie::Model model("resnet_50.tar.gz", {}, connection);
  const pcie::ModelInfo info = model.info();
  if (info.inputs.empty())
    throw std::runtime_error("model reports no inputs");

  model.build(/*readiness_timeout_ms=*/180000);
  pcie::TensorList outputs = model.run(
      load_input("assets/sample.png", info.inputs[0].name),
      /*timeout_ms=*/30000);

  if (outputs.empty() || outputs[0].dtype != pcie::TensorDType::Float32 ||
      outputs[0].data == nullptr || outputs[0].size_bytes == 0)
    throw std::runtime_error("expected one FP32 classification output");

  const auto* scores = static_cast<const float*>(outputs[0].data);
  const std::size_t count = outputs[0].size_bytes / sizeof(float);
  const auto best = std::max_element(scores, scores + count);

  std::cout << "output: " << outputs[0].route.name << " [";
  for (std::size_t index = 0; index < outputs[0].shape.size(); ++index) {
    if (index != 0)
      std::cout << ", ";
    std::cout << outputs[0].shape[index];
  }
  std::cout << "]\n";
  std::cout << "top1: " << std::distance(scores, best) << '\n';
  model.close();
}

Create CMakeLists.txt:

CMakeLists.txt

cmake_minimum_required(VERSION 3.16)
project(pcie_host LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(SimaPCIeHost REQUIRED CONFIG)
find_package(OpenCV REQUIRED COMPONENTS core imgcodecs imgproc)

add_executable(pcie_host pcie_host.cpp)
target_include_directories(pcie_host PRIVATE ${OpenCV_INCLUDE_DIRS})
target_link_libraries(
  pcie_host
  PRIVATE SimaPCIeHost::sima_neat_pcie_host ${OpenCV_LIBS}
)

Run the application

Build and run the C++ application:

pcie-host$cmake -S . -B buildpcie-host$cmake --build buildpcie-host$./build/pcie_host

A successful run prints the returned output tensor and its top-scoring ImageNet class index. With the documented ResNet-50 artifact and Labrador image, the expected top class is index 208, Labrador retriever:

output: resnetv17_dense0_fwd [1, 1000]
top1: 208

The Python context manager calls close() automatically. The C++ example calls it explicitly. Both versions release queue 0 before exiting.

Next step

Continue with PCIe Co-processing to learn the complete model API, pipeline requests with push() and pull(), and image preprocessing options.