Resolutions and Frame Rates

cfg.camera selects the sensor output mode. The selection determines the size of the streamed JPEG, whether the display can show the image, the scaling applied by the neural network pipe, and the memory allocated. The same mode values apply to both sensors and map to different native modes on each; the two tables below give the mapping per sensor. The frame rate ceiling in every mode is set by the sensor readout, not by the JPEG encoder or the USB link.

#OV5640 modes

CAMERA_QVGA and CAMERA_VD66_FULL are VD66GY modes and map to WVGA on the OV5640.

The display pipe has no downscaler. Modes larger than 800x480 disable the display automatically; USB streaming continues at full resolution.

#VD66GY modes

The global shutter sensor is read out as crops of its 1124x1364 array. Shared mode values map to the nearest crop and the JPEG reports its real size.

The ceilings were measured on hardware. The modes are readout bound: a global shutter reads the entire array each frame regardless of the output crop, so VGA is not faster than XGA, and the full frame tops out at 45 fps.

#Frame rate

cfg.fps = FPS_30;

The enumeration values equal the frame rate: FPS_15, FPS_30, FPS_45, FPS_60, FPS_90, FPS_120, FPS_240. FPS_DEFAULT selects the mode's native maximum.

A mode can be slowed below its maximum by added vertical blanking but cannot be run faster. A request above the ceiling is clamped with a [CAM] warning on the console. The higher values exist for the VD66GY and clamp on the OV5640. FPS_240 selects the maximum for any mode.

A lower frame rate leaves more time per frame for the sketch, reduces PSRAM traffic and reduces current. On TRANSPORT_LCD the frame rate also sets the panel refresh, clamped to 30 to 60 Hz.

#Effects of the resolution

Memory. The JPEG encoder reads from a line buffer in on-chip RAM sized to the mode at start: about 100 KB at WVGA and about 480 KB at 1080p. JPEG output buffers in PSRAM range from about 1 MB at VGA to 2.5 MB at 5 MP. When a resolution cannot be accommodated alongside a large model and the display, Vision.begin() prints a diagnostic and does not start the camera.

CSI link. The STM32N6 CSI-2 host implements MIPI CSI-2 version 1.3 over a D-PHY with two data lanes at up to 2.5 Gbit/s per lane, and the DCMIPP behind it is rated for sensors up to 5 MP at 30 fps. Each OV5640 mode runs the receiver at a mode-specific rate, using register values taken from the mainline Linux OV5640 driver. The VD66GY runs a fixed 804 Mbps per lane.

Neural network input. The hardware downscaler manages a ratio of about 8x. When the camera to model ratio exceeds this, as 2592 to 256 does, the pipe decimates first. Decimation drops pixels rather than averaging them, so the model input is softer than at lower resolutions. Inference works at every mode; XGA gives the best detection quality. For wide modes, cfg.nn.aspect = ASPECT_CROP_CENTER avoids squashing the frame into a square input. See Aspect Modes and Coordinates.

#Selection guide

#Run-time changes

The mode cannot be changed after Vision.begin(). The pipes, the line buffer and the JPEG buffers are sized at initialisation. The crop seen by the model can be changed per pass with Vision.runRoi().