Machine Vision Examples
The classical machine vision examples in the core, what each detects, the hardware each needs, and the pattern they share.
The core includes a set of machine vision examples that find things in the camera picture using the processor alone, with no neural network. They read QR codes, barcodes and printed markers, follow a coloured object, and detect motion. Each streams what it finds to Neuro Studio's Vision panel and can draw it on the on-board display.
The examples
| Example | Library | Detects | Camera | Display variant |
|---|---|---|---|---|
| QrReader | Quirc | QR codes: text, version, error correction level | OV5640 or VD66GY, autofocus on | QrReaderDisplay |
| BarcodeReader | Barcode1D | EAN-13, UPC-A and EAN-8 barcodes | OV5640, autofocus on | |
| AprilTagReader | AprilTag | tag36h11 markers: identifier, rotation, and position in space when calibrated | OV5640 or VD66GY | |
| BlobTracker | N6Imgproc | The largest patch of a chosen colour, with a smoothed position | OV5640 or VD66GY | BlobTrackerDisplay |
| FaceRecognition | FaceID | Faces, matched against enrolled people. Uses two neural networks and belongs with the neural network pages | OV5640 on the TFT carrier | Built in |
Display variants draw their result on the TFT carrier's panel instead of streaming it.
The shared pattern
Every example follows the same steps.
Configure the camera without a model.
Vision.begin(cfg)is called with aVisionConfigand no network. Thecfg.nnfields set the size and crop of the picture the algorithm receives, cropped and scaled by hardware. A square centre crop suits a QR code or a tag; a wide crop suits a barcode.Take a snapshot.
NN_capture_snapshot(buffer, size)copies one RGB888 frame into a buffer the sketch owns, in external PSRAM and aligned to 32 bytes.Run the decoder. The decoder is a plain C function over the buffer. It returns a list of results with a box for each, normalised from 0 to 1 across the analysed window.
Track.
n6_tracker_tsmooths the box from frame to frame and tolerates a few frames with no detection before giving up. It is the same tracker the neural network cascades use.Publish. The vision publisher sends each result's box, text and a thumbnail to Neuro Studio, and clears the panel on frames with nothing found. The result is also printed to the serial console, suppressing repeats.
Tune live. Detector thresholds are registered with the live tuning system, so they can be adjusted from Neuro Studio while watching the result, then copied back into the sketch.
What Neuro Studio shows
The Vision panel lists each result with its decoded text or identifier and a small picture of it cut from the frame. The result's box is drawn on the live video. The panel titles itself from the detector the sketch announces. A QR code that leaves the frame is removed from the panel on the next frame, because the sketch publishes an empty result set rather than nothing.
Camera settings that matter
- Focus. The OV5640's autofocus is enabled with
cfg.af = AF_CONTINUOUS. A blurred code or tag never decodes. - Fixed exposure and white balance. Colour tracking in particular requires the camera's automatic loops to be disabled after they settle, or the target hue drifts. See Capture Controls.
- Snapshot size. 256 pixels suits a QR code filling a quarter of the frame; 320 suits AprilTags; reading smaller subjects needs a larger snapshot rather than a lower threshold.
- Frame rate. Decoders run at a few frames per second on a 256 to 320 pixel snapshot.
FPS_30on the camera is ample.
Pages
- Snapshots and Image Primitives: capturing a frame and the N6Imgproc operations.
- QR Codes and Barcodes.
- AprilTags and Pose.
- Colour Blobs and Motion.
- Publishing Data from a Sketch: the vision publisher in detail.
Diagnostics
AprilTagDoctor, AprilTagSynthetic and ImgprocSelfTest in extras/bench isolate detector faults from imaging faults and test the primitives without a camera.