Image capture and 60fps color object targeting with an 8-bit microprocessor and a CMOS camera.
This is a simple demo to illustrate a larger idea: computer vision with limited resources enables emerging applications for consumer products.
The goal is to prove the ability of the F360 to perform machine vision tasks. The C8051F360 syncs to video from a digital camera, reads pixel values, makes decisions about objects in the scene, communicates by serial port, and drives displays or servos by GPIO.
Microcontroller: SiliconLabs C8051F360
Camera: Omnivision OV7720
The serial port connection to the host PC is not shown, it wasn’t connected in this photo. The VGA connector in the image is for a related project.
The Silicon Labs C8051F360 runs at 100Mips, which makes it a perfect mate for a camera running at 60fps, 640×480 resolution. The data rate for demosaiced RGB is approx. 48MHz. The F360 takes two cycles per port read, or 50MHz max. The 24MHz clk from the camera is input to the F360, and the F360 internal PLL converts the 24MHz clk to a 96MHz system clock.
Download project files here.
Camera -> Microcontroller -> PC
The F360 communicates captured pixels and target data to the host PC via a virtual com port supplied by Silicon Labs.
All signals pass through the 8051.
Code in C and Assembly.
Due to loop overhead, any pixel can be read from the camera, but not every pixel in sequence.
It takes a minute to transfer an image over the serial port, but the video motion targets tracking the red toy car are transferred at full speed for the full frame rate, 60 times a second.
The Omnvision OV7720 eval. board is wired directly to the F360 port pins.
Connections between camera and microcontroller:
F360 input P1.7-P1.0 – Camera Data [7:0]
F360 input P0.4 – Camera Hsync
F360 input P0.5 – Camera Vsync
F360 input P0.6 – Camera Clk
F360 Gnd – Camera Gnd
Sampling the camera image with the F360:
The F360 cache had to be flushed before horizontal and vertical sync routines or the results were erratic.
F360 synced to the camera, every pixel in its place.
Filtering for a color window:
Image from pixels captured by the F360.
Pixel values filtered so that only red pixels are captured.
Camera view. The F360 will perform a very simple grouping of red pixels to fit a target box on the remote controlled red car.
The 8051 filters the RGB input and marks the red object at 60fps. Only the target coordinates are sent to the host PC. The coordinates are updated at 60fps, so the motion of the target box tracking the car is full speed and is seen on the host PC. The image above is misleading as the moving targets are superimposed on the static red pixels.
It takes a minute to transfer an image over the serial port, but the video motion targets tracking the red toy car are transferred at full speed for the full frame rate, 60 times a second. The first part of the video below shows the full-speed targets from the F360 and the last half of the video shows an image being transferred slowly to the host PC.
Advantages of directly connecting a CMOS digital camera to a microcontroller:
- Two components
- Easy to program
- Quick to prototype
- Low power
Disadvantages of directly connecting a CMOS digital camera to a microcontroller:
- Processor is fully occupied while sampling pixels for the object
- Cannot sample every pixel in sequence
- Very limited number of colors/shapes/objects can be detected (e.g. only one color detected in this project)
If a microcontroller like the F360 had a pixel-processing peripheral, you would get the best of both worlds – easy, low-cost, low-power vision with high resolution, high frame rate, multiple object targeting.
Download project files here.