Neuro Vision Thermal Getting Started
The Neuro Vision Thermal module, combining a VD66GY global shutter camera with a FLIR Lepton 3.1R radiometric thermal camera.
The Neuro Vision Thermal is a dual camera module for the Neuro N6. It pairs the ST VD66GY global shutter visible light sensor with a FLIR Lepton 3.1R radiometric thermal camera. It has no strobe LED, no time of flight sensor, no display and no wireless module. It is the module used by the core's low power and event recording examples.
Specifications
| Visible camera | ST VD66GY, 1.5 MP global shutter, fixed focus. Carrier wires the CSI lanes inverted |
| Thermal camera | FLIR Lepton 3.1R, 160x120, radiometric, about 9 fps |
| Thermal interfaces | Command and control over the module I2C bus at address 0x2A through a level shifter, video over SPI (VoSPI) on connector pins that carry display signals on the TFT carriers |
| Thermal control | Power down on PD3, reset on PD9, chip select on PF4 |
| Strobe LED | None |
| Time of flight | None |
| Display | None |
| Wireless | None |
| Part number | OL-DEV-N6-005, hardware revision C |
The inertial unit, magnetometer and microphone are on the Neuro N6 board itself.
Circuit
The VD66GY seats in a Hirose BM28 30-pin connector with a 12 MHz oscillator, as on the ST Cam. The Lepton seats in a 32-pin socket and is clocked by a 25 MHz oscillator. BCT2057 regulators produce the Lepton's 1.2 V and 2.8 V supplies and a separate analogue 2.8 V. A TXB0106 level shifter carries the Lepton's SPI video link and AW39112 translators carry its I2C and the power-down and reset lines between the Neuro N6's 3.3 V and the Lepton's 2.8 V. The schematic is published under CERN-OHL-S v2; see Downloads.
Fitting
The module mates with the two board-to-board connectors on the Neuro N6 with the board unpowered. The Lepton is mounted inverted relative to the visible sensor; the driver's setFlip180(true) corrects its frames.
In a sketch
The carrier wires the CSI lanes inverted, which the camera declaration states:
#include <VD66GY_Arduino.h>
#include <NeuroVisionThermal.h>
VD66GY camera(VD66GY_CARRIER_THERMAL);
NeuroVisionThermal thermal;The ThermalCamera example, under File > Examples > NeuroVisionThermal, streams the Lepton to Neuro Studio as the main video. The GlobalShutterThermal example, under File > Examples > VD66GY, streams the visible sensor. See Thermal Camera and The Global Shutter Camera.
Thermal camera
The Lepton reports temperatures in hundredths of a kelvin per pixel in its radiometric mode. It requires about 950 ms to boot after power is applied and closes an internal shutter periodically for flat field correction, during which frames are degraded. It fails to boot if it observes I2C traffic to other addresses during its boot window, so the bus is claimed around powerOn(). It cannot boot across a Stop mode sleep and is power cycled between duty cycles.
Running a neural network on the visible camera at full rate contends with the thermal video link. cfg.nn.max_fps of about 22 keeps both healthy.
SD card
The low power and event recording examples log to an SD card on the header SPI bus with chip select on A5. SdFat is used with the two platform-specific SPI calls described in SPI.
Low power operation
This module is the reference for the power save library's measurements. A cycle of one visible frame, one thermal frame and one reading of the board's magnetometer every 15 seconds averaged 22.8 mA on battery. See Low Power.
Uploading over LTE
With a SIMCom A7683E modem on the header UART, the LteThermalUpload example posts a visible image, a thermal frame and a status record to a web server every thirty seconds. See LTE Modem.