Raw NPU Output

Vision.beginRawOnly() and Vision.runRaw() run a model without a post-processor, overlay or metadata and return the output tensors for the sketch to decode.

#include <OV5640_Arduino.h>
#pragma neuron6 model="my_model.tflite" name=my_net

OV5640 camera;
NEURON6_DECLARE_MODEL(my_net);

void setup() {
  Serial.begin(115200);
  Vision.beginRawOnly(&NN_Instance_my_net);
}

void loop() {
  const LL_Buffer_InfoTypeDef *out = Vision.runRaw();
  if (!out) return;

  for (int i = 0; out[i].name != nullptr; i++) {
    const uint8_t *bytes = LL_Buffer_addr_start(&out[i]);
    uint32_t       size  = LL_Buffer_len(&out[i]);
    // decode
  }
}

beginRawOnly(model) initialises the camera, the NPU and the USB video stream, so the host receives the image and the console, but does not initialise the display and binds no post-processor. runRaw() captures a frame, runs inference and returns the output descriptor array, terminated by an entry with a NULL name. It returns NULL when no model is bound.

beginRawOnly() uses the default 256x256 stretched input. A model with another input size is run through Vision.begin(cfg) with the size set and runWith() with a post-processor whose run performs the decode.

#Tensor inspection

A sketch that prints each output's shape, quantisation and first values is the usual first step with a new model:

void loop() {
  const LL_Buffer_InfoTypeDef *out = Vision.runRaw();
  if (!out) return;

  static bool described = false;
  if (!described) {
    described = true;
    for (int i = 0; out[i].name != nullptr; i++) {
      Serial.print("out["); Serial.print(i); Serial.print("] ");
      Serial.print(out[i].name);
      Serial.print(" shape=");
      for (int d = 0; d < out[i].ndims; d++) {
        Serial.print(out[i].shape[d]);
        if (d + 1 < out[i].ndims) Serial.print("x");
      }
      Serial.print(" bytes="); Serial.print(LL_Buffer_len(&out[i]));
      Serial.print(" scale="); Serial.print(out[i].scale[0], 6);
      Serial.print(" zp=");    Serial.println(out[i].offset[0]);
    }
  }

  const int8_t *t0 = (const int8_t *)LL_Buffer_addr_start(&out[0]);
  for (int k = 0; k < 4; k++) { Serial.print(t0[k]); Serial.print(" "); }
  Serial.println();
  delay(500);
}

Names are assigned by the compiler and vary between exports. Tensors are identified by shape; two tensors with the same shape are distinguished by content magnitude or by order.

#Dequantisation

Integer outputs carry a scale and zero point per tensor. The float value is (raw - zero_point) * scale. A classifier requires only this followed by an argmax. A detector's tensor layout, anchors, strides and box encoding, depends on the model family and is what a post-processor encodes.

#Validity

Output buffers are in NPU-accessible memory and are valid from the return of runRaw() until the next runRaw() or run(). Data needed beyond that is copied.

#Timing

runRaw() blocks the calling task for one capture and one inference, yielding to other tasks while the NPU executes. A 256x256 detector takes a few milliseconds of NPU time plus the frame wait. ModelBench in extras/bench reports a per-frame breakdown.

#Conversion to a post-processor

A working decode in loop() is moved into an IPostProcessor by placing the decode in run, the output in emit_metadata, and replacing beginRawOnly with Vision.begin(model, &pp, cfg). The post-processor then participates in overlay drawing, metadata slots and multi-model composition. See Custom Post-Processors.