Software/Arduino Core/Classical Vision

Colour Blobs and Motion

Colour blob tracking with N6Imgproc, live tuning of colour thresholds, tracker smoothing, and motion gating.

intermediate2 min read

Colour blob tracking locates the largest region of a chosen colour in a frame. On the Neuro N6 it runs on the CPU in a few milliseconds per frame and leaves the NPU free.

Blob detection

The BlobTracker example, under File > Examples > N6Imgproc, has this structure:

cpp
#include <OV5640_Arduino.h>
#include <PostProcess.h>
#include <n6_imgproc.h>

#define DIM 256

static uint8_t g_snapshot[DIM * DIM * 3] __attribute__((aligned(32), section(".ext_psram")));

static int32_t TARGET_HUE = 25;      // degrees, 0 to 359
static int32_t HUE_TOL    = 18;
static int32_t MIN_SAT    = 90;      // 0 to 255
static int32_t MIN_VAL    = 60;

OV5640 camera;
static n6_tracker_t g_trk;

void setup() {
  VisionConfig cfg;
  cfg.camera    = CAMERA_VGA;
  cfg.nn.width  = DIM;  cfg.nn.height = DIM;
  cfg.nn.aspect = ASPECT_CROP_CENTER;
  Vision.begin(cfg);

  camera.setAutoWhiteBalance(false);
  n6_tracker_init(&g_trk, 0.5f, 5);

  n6_tune_hue("target hue", &TARGET_HUE);
  n6_tune_int("hue tol",    &HUE_TOL, 0, 180);
  n6_tune_int("min sat",    &MIN_SAT, 0, 255);
  n6_tune_int("min val",    &MIN_VAL, 0, 255);
}

void loop() {
  if (!NN_capture_snapshot(g_snapshot, sizeof g_snapshot)) return;

  n6_blob_cfg_t bc = { TARGET_HUE, HUE_TOL, MIN_SAT, MIN_VAL };
  n6_blob_t b;
  if (n6_blob_find_rgb888(g_snapshot, DIM, DIM, &bc, &b)) {
    n6_tracker_update_box(&g_trk, b.cx, b.cy, b.w, b.h);
  } else {
    n6_tracker_miss(&g_trk);
  }

  if (n6_tracker_is_tracking(&g_trk)) {
    const n6_roi_t *r = n6_tracker_roi(&g_trk);
    // r->cx, r->cy: smoothed position, 0 to 1 across the frame
  }
}

n6_blob_find_rgb888() gates each pixel on hue, saturation and value, builds a coarse density grid, flood fills the densest connected cluster, and returns its bounding box, dominant hue and size. It is approximate and fast enough to run on every frame.

Colour tuning

The four thresholds are registered with the live tuning system and appear as sliders in Neuro Studio. n6_tune_hue() identifies its parameter as a hue in degrees, and the host renders the colour rather than a number. The panel's Copy as code function produces the literals for the sketch. Values are not stored on the board. See Live Tuning and Assets.

White balance

Automatic white balance re-meters every frame and shifts the hue of the target object between frames. The example disables it after the camera has settled. Under fixed lighting, exposure and gain are also locked. See Capture Controls.

Tracker

The blob's box jitters by a pixel or two per frame, and a shadow can hide the object for a frame. n6_tracker_t smooths the box and tolerates short losses, as it does for neural network detections and AprilTags. A smoothing factor of 0.5 and a miss limit of 5 are typical starting values.

Motion gating

Motion detection answers whether the scene changed, which is cheaper than any detector and is used to gate expensive work:

cpp
static uint8_t g_gray[DIM * DIM] __attribute__((section(".ext_psram")));
static uint8_t g_prev[DIM * DIM] __attribute__((section(".ext_psram")));
static uint8_t g_diff[DIM * DIM] __attribute__((section(".ext_psram")));

n6img_rgb888_to_gray(g_snapshot, g_gray, DIM, DIM);
n6img_absdiff(g_prev, g_gray, g_diff, DIM, DIM);

n6img_stats_t st;
n6img_statistics(g_diff, DIM, DIM, NULL, &st);
memcpy(g_prev, g_gray, sizeof g_prev);

if (st.mean > 4) {
  Vision.run();        // run the model only when the scene changed
}

The N6Events library packages this as n6_motion_gate_t with a smaller working grid. See Events and SD Logging.

Exact components

When every object or its outline is required, n6img_find_components() labels pixels and returns each connected component as a box with area, largest first. It is slower than the blob finder and exact. The threshold and morphology steps that precede it are described in Snapshots and Image Primitives.

Publishing

The vision publisher draws the box on the video in Neuro Studio:

cpp
vnd_meta_vision_t vis;
vnd_meta_vision_begin(&vis, VND_META_SLOT_BLOB, META_BLOB_FRAME, g_payload, sizeof g_payload);
if (found) vnd_meta_vision_add(&vis, b.cx, b.cy, b.w, b.h, "orange", NULL, 0, g_snapshot, DIM, DIM);
vnd_meta_vision_finish(&vis);

The BlobTrackerDisplay example draws the box on the on-board panel instead.