Audio Event Detection

The N6AudioAI library runs ST's audio event detection model on the NPU. It converts microphone samples to the log-mel spectrogram the network expects and returns a score for each of ten classes: chainsaw, clock tick, crackling fire, crying baby, dog, helicopter, rain, rooster, sea waves and sneezing.

#Example

The AudioEventDetection example, under File > Examples > N6AudioAI, has this structure:

#include <OV5640_Arduino.h>
#include <PostProcess.h>
#include <Microphone.h>
#include <n6_aed.h>
#include <vnd_meta_audio.h>

#pragma neuron6 model="aed_yamnet_int8.onnx" name=aed

OV5640     camera;         // Neuro Studio overlays results on a video stream
Microphone mic;
NEURON6_DECLARE_MODEL(aed);

static int16_t chunk[512];

void setup() {
  VisionConfig cfg;
  cfg.camera = CAMERA_VGA;
  Vision.begin(cfg);                          // no model bound; the sketch runs it

  mic.begin();
  mic.streamToNS(false);                      // samples go to the sketch
  n6_aed_begin(&NN_Instance_aed, mic.sampleRate());
  n6_aed_set_threshold(0.5f);
}

void loop() {
  if (mic.available() >= 512) {
    uint32_t got = mic.read(chunk, 512);
    n6_aed_push(chunk, got);
  }
  if (!n6_aed_ready()) { delay(2); return; }

  n6_aed_run();
  int top = n6_aed_top();                     // class index, or -1 below threshold

  vnd_meta_audio_result(n6_aed_scores(), N6_AED_NUM_CLASSES, top, n6_aed_level(), 0.5f);
  vnd_meta_audio_mel(n6_aed_input_image(), N6_AED_NMEL, N6_AED_NFRAMES, 0);
}

Samples are pushed into a sliding window. n6_aed_ready() reports when a full window is available. n6_aed_run() computes the spectrogram and runs the network. n6_aed_scores() returns all class scores, n6_aed_top() the highest class above the threshold, and n6_aed_label(i) a class name. Neuro Studio's Audio panel shows the scores, the winning class and the spectrogram.

The camera runs even though the sketch does not use the image, because Neuro Studio presents audio results alongside a video stream.

#Frontend

The microphone runs at 32552 Hz and the model expects 16 kHz, so the library includes a fractional resampler. The log-mel spectrogram is 64 mel bands by 96 frames, matching the model's input. The frontend tables are regenerated from ST's published parameters by tools/gen_aed_tables.py, which verifies them against ST's own before emitting. n6_aed_begin() fails if the model's input is not 64x96 int8.

#Threshold

Sounds outside the ten classes are matched to the nearest class, so a confidence threshold is required. The example registers its threshold with the live tuning system so it can be adjusted from Neuro Studio in a noisy environment without rebuilding. See Live Tuning and Assets.

#Use as an event trigger

n6_aed_top() returns a class index, which makes it a trigger for the event recorder:

if (n6_aed_top() == AED_DOG) n6_events_fire(N6_EVT_SRC_AUDIO, 800);

See Events and SD Logging.

#Dependencies and licence

The library wraps ST's STM32 audio preprocessing library, vendored unmodified under BSD-3-Clause, with the few CMSIS-DSP functions the core declares but does not link. The model is ST's under SLA0044. See Licensing.