Anatomy of a Sketch

A sketch that uses neither the camera nor a neural network has the same structure as on any Arduino board. A vision sketch adds a fixed set of declarations before setup(). The object segmentation example, with comments removed, shows all of them:

#include <OV5640_Arduino.h>                    // 1
#include <PostProcess.h>                       // 2

#pragma neuron6 model="yolov8n_256_quant_pc_ii_seg_coco-st.tflite" name=yolov8_iseg   // 3

OV5640 camera;                                 // 4
NEURON6_DECLARE_MODEL(yolov8_iseg);            // 5

void setup() {
  Vision.begin(&NN_Instance_yolov8_iseg,       // 6
               PostProcess_YOLOv8_ISEG(0.4f, 0.5f));
}

void loop() {
  Vision.run();
}

#Camera include

OV5640_Arduino.h provides the OV5640 driver and causes the Arduino library scanner to link it. The global shutter sensor uses VD66GY_Arduino.h. A sketch without a camera omits the include.

#PostProcess include

PostProcess.h provides the bundled post-processors, the display overlay drawing functions and the builders that send results to Neuro Studio. Any sketch that runs a model, draws on the display or streams a sensor reading to the host includes it. A camera-only stream does not.

#Model pragma

#pragma neuron6 model="my_model.tflite" name=my_net

A prebuild hook scans the sketch for these directives, runs ST Edge AI Core on each file and compiles the generated network into the binary. During upload the weights are written to external flash before the application, and only if they differ from the blob already present.

The file is searched for in the sketch directory, then in a models/ directory beside the sketch, then in the core's models/ directory. Examples therefore name bundled models by bare filename.

A sketch may declare up to eight models. Commented-out pragmas are ignored.

#Camera object

OV5640 camera;                 // standard Neuro Vision carrier
OV5640 camera(CARRIER_TFT);    // TFT carrier

A single sensor object declared at file scope constitutes the camera configuration. Its constructor runs before setup(), performs no hardware access, registers the object as the sensor Vision drives, and records the CSI data lane mapping of the named carrier.

The carrier argument describes the wiring. The TFT carrier has its CSI lanes inverted relative to the standard carrier. A mismatch between the argument and the physical carrier causes a CSI synchronisation failure and no frames are received. Vision.begin() prints the sensor name and expected chip ID on the console when initialisation fails.

Exactly one sensor object is declared. A sketch with no camera declares none; the core then boots without a camera and prints a notice.

#Model declaration

NEURON6_DECLARE_MODEL(yolov8_iseg);

Instantiates the execution instance NN_Instance_yolov8_iseg that Vision runs. One declaration per model. Removing a model's pragma and its declaration together removes its weights from flash.

#Vision.begin and Vision.run

Vision.begin(model, pp) initialises the camera, the DCMIPP video pipes, the NPU, the video encoder and the USB stream, then initialises the post-processor. Vision.run() captures one frame into the model's input buffer, runs inference, runs the post-processor, draws the overlay and publishes metadata to the host. JPEG encoding and USB transmission run in background tasks at the camera frame rate, independently of loop().

A VisionConfig passed as a third argument overrides the defaults. See VisionConfig Reference. Other begin() overloads support sketches with no model, several models, or no post-processor.

#Implicit includes

No core header is included by the sketch. The build prepends Arduino.h, which on this core provides the C API, the Vision class, the RGB LED object and the region and tracker types used by cascades. #include <NeuroN6.h> is accepted and redundant.

#Minimal variants

Camera only, with no model or post-processor:

#include <OV5640_Arduino.h>
OV5640 camera;

void setup() {
  VisionConfig cfg;
  Vision.begin(cfg);
}

void loop() {
  delay(1000);   // frames are sent by background tasks
}

A sensor over I2C, with no camera:

#include <Wire.h>
#include <LSM6DSL_Arduino.h>

LSM6DSL imu;

void setup() {
  Serial.begin(115200);
  imu.begin();
}

void loop() {
  imuVec3 a = imu.readAccel();
  Serial.println(a.z, 3);
  delay(200);
}