Skip to main content

Diagnostics and debugging

GraphReport

GraphReport captures structured diagnostics:

  • pipeline string (for reproduction)
  • canonical error_code (machine triage)
  • repro_note (human summary + hint)
  • node reports and owned element names
  • bus messages and error details
  • optional flow/timing counters

When an error occurs, NeatError carries a GraphReport you can log or serialize.

Error taxonomy

Framework errors use stable code families:

Error codeMeaningTypical fix
misconfig.pipeline_shapeNode order/shape contract violationEnsure Input() first for push pipelines and Output() last for pull pipelines
misconfig.capsFramework caps-override or adjacent Node contract mismatchAlign caps_override and the declared Node contracts
misconfig.input_shapeInput tensor/frame/sample shape or data type does not match the model contractProvide the expected shape and data type, or configure model preprocessing
misconfig.runtime_abi_mismatchNeat and a runtime plugin use incompatible ABIsInstall a version-matched Neat Library and runtime
misconfig.graph_element_nameA custom element cannot be assigned a stable Node nameGive custom elements stable, unique names
misconfig.input_capacitySource image exceeds preprocessing input capacityIncrease input_max_width / input_max_height, or scale before the model stage
misconfig.media_capsAdjacent GStreamer stages require incompatible media capsAlign format, resolution, and frame rate or insert conversion
misconfig.media_formatA stage received an unsupported media formatConfigure a supported format or insert format conversion
misconfig.tensor_dtype_missingTensor contract has no dtype/formatDeclare a supported tensor dtype in the upstream contract
misconfig.option_out_of_rangeA stage option is invalid for the current tensorChoose a value in the range shown by the diagnostic
build.parse_launchA gst_parse_launch failure has no more specific classificationInspect the attached report for the parser context
build.pipeline_syntaxCustom GStreamer fragment syntax is invalidCorrect and validate the fragment with gst-launch-1.0
build.plugin_missingA required GStreamer element or codec plugin is not installedInstall/replace it and check with gst-inspect-1.0
build.property_invalidAn element property is unknown or invalidCheck the property name and value with gst-inspect-1.0
runtime.pullA pull failed without a more specific root causeInspect the attached report and first upstream error
runtime.element_failedA stage failed without a more specific mappingCorrect the reported stage and its upstream input
runtime.output_timeoutNo output arrived before the configured timeoutVerify source flow or increase an expected timeout
runtime.unexpected_eosThe pipeline reached EOS before a required outputCheck for premature source EOS and supply enough input
io.parseJSON or stage-configuration parse/schema failureValidate configuration syntax and required fields
io.openGraph save/load file open/read/write failureCheck path existence, permissions, and storage health
io.file_not_foundInput file does not existCorrect the path and confirm the file exists on the DevKit
io.permission_deniedFile or device is not readableCorrect ownership/permissions
io.rtsp_connection_failedRTSP source cannot be contactedVerify URL, reachability, server, and credentials
io.camera_not_foundRequested camera is unavailableSelect a reported camera or use the default
io.model_not_foundRequested model archive does not existCorrect the model path and confirm it is installed
io.source_endedInput source reached its normal endStop consuming it or provide more input
codec.invalid_h264_streamInput has no valid H.264 framesSupply a complete H.264 stream or correct the codec
codec.decode_failedDecoder failed after accepting the streamVerify the codec and input integrity
codec.encode_failedEncoder could not encode the supplied framesVerify input format, resolution, and encoder settings
resource.memory_allocation_failedA required memory allocation failedReduce workload memory use and free memory used by other applications or pipelines
resource.device_memory_exhaustedDevice DMA/CMA allocation failedReduce concurrent streams, resolution, or buffering
resource.output_pool_exhaustedAll output buffers remain in useRelease zero-copy outputs or use owned copies
resource.buffer_too_smallA buffer is smaller than its declared payloadCorrect dimensions/stride or allocate the required bytes
resource.disk_fullA write failed because storage is fullFree space or choose another destination
infra.dispatcher_unavailableAccelerator runtime cannot be acquiredStop competing workloads and verify DevKit compatibility
infra.accelerator_execution_failedAccelerator could not execute a model stageRestart the pipeline and reduce concurrent accelerator work
DispatcherUnavailableLegacy spelling of infra.dispatcher_unavailableMigrate handlers to the canonical infrastructure code
internal.plugin_failureA plugin failed without a user-actionable classificationCapture the report and contact support

PullError.code uses the same taxonomy (not only exception paths). See the Error code catalog for the C++ and Python constant names and migration guidance for applications that matched the previous coarse codes.

Production messages intentionally omit GStreamer internals. Plugin debug verbosity adds the raw GError domain/code, element factory, message, and structured plugin details. Recognized credentials and URL secret parameters—including URI userinfo, auth, playback-token, hdnts, stream-key, and tkn—are redacted before either form is stored. Report-facing pipeline strings, Node fragments, reproducer commands, and serialized JSON are redacted without changing the executable pipeline held internally.

Programmatic handling

#include "pipeline/ErrorCodes.h"
#include "pipeline/NeatError.h"

try {
auto run = graph.build(input);
simaai::neat::Sample out;
simaai::neat::PullError perr;
const auto st = run.pull(500, out, &perr);
if (st == simaai::neat::PullStatus::Error) {
if (perr.code == simaai::neat::error_codes::kMediaCaps) {
// Fix the incompatible upstream/downstream media contract.
} else {
// Handle another specific code, including future codes, or report it.
}
}
} catch (const simaai::neat::NeatError& e) {
if (e.report().error_code == simaai::neat::error_codes::kPluginMissing) {
// Install or replace the missing GStreamer component.
}
}

Debug knobs (environment)

Key environment variables (see Architecture for detail):

  • SIMA_GST_DOT_DIR: write DOT graphs for failures
  • SIMA_GST_BOUNDARY_PROBES: boundary flow counters
  • SIMA_GST_ELEMENT_TIMINGS: per-element timings
  • SIMA_GST_FLOW_DEBUG: per-element flow counters
  • SIMA_GST_ENFORCE_NAMES: enforce naming contract

To append redacted raw GStreamer context to NeatError::what() and GraphReport.repro_note, set both variables for the failing command:

SIMA_NEAT_VERBOSE_LEVEL=2 \
SIMA_NEAT_VERBOSE_TOPICS=gstreamer \
./your-neat-application

NEAT_LOG_LEVEL=debug is not a Neat Library setting. Keep verbose output disabled in normal operation; it is intended for short diagnostic runs and may contain deployment-specific paths or media addresses even though recognized credential fields are redacted.

Debug workflow

  1. Capture GraphReport.error_code and bucket the failure by taxonomy first.
  2. Capture GraphReport.repro_note for concrete context and built-in hint.
  3. Capture pipeline text: Graph::describe_backend() or last_pipeline().
  4. Capture structured diagnostics: MeasureReport::to_text() or NeatError::report().
  5. Inspect GraphReport.bus for first terminal ERROR source + detail.
  6. If runtime stalls/timeouts, enable boundary/element probes to localize flow stop.

Recommended support bundle:

  • error_code
  • repro_note
  • full pipeline_string
  • first 3-5 terminal bus errors (GraphReport.bus)
  • environment overrides used in run/validate

Customer graph performance artifact

For throughput/latency/power reporting, prefer the graph-run JSON export:

RunOptions opt;
opt.enable_board_power(); // graph-level power when supported by the board/SOM
Run run = graph.build(opt);

// run your normal push/pull loop inside a measurement window, then:
auto report = run.start_measurement().stop();
std::cout << report.to_text();

The export keeps scopes explicit:

  • run.graph_metrics.throughput_fps and run.graph_metrics.power are graph-level headlines.
  • run.node_metrics[] contains node/plugin latency only; node/plugin power is intentionally absent.
  • latency_semantics and aggregation tell you whether values are run-lifetime or measured-window deltas.
  • plugin_metrics_unattributed[] preserves kernel/plugin rows that could not be mapped to exactly one node.

For a measured window, use Run::start_measurement() and pass the returned MeasureReport to run_to_json(run, report, ...) / save_run_json(run, report, ...). Measured-window node min_ms/max_ms are marked unavailable because cumulative min/max counters cannot be subtracted exactly without window-local counters.

Power note: the current DVT board can validate option plumbing and JSON shape, but its wattage readings are not treated as numerically reliable. SOM hardware is the intended platform for power-number validation.

Common failures → fixes

SymptomLikely causeFix
missing ... pluginGStreamer plugin not foundCheck GST_PLUGIN_PATH, run gst-inspect-1.0 <plugin>
appsink 'mysink' not foundMissing terminal Output()Ensure Output is the last node in run/build pipelines
caps_override is set; renegotiation disabledcaps pinnedRemove caps_override or keep input caps fixed
tensor caps change not supportedTensor shape/dtype change at runtimeKeep tensor shape/dtype stable (no renegotiation)

For structured plugin errors and actionable hints, see Troubleshooting.