I’ve spent the last few years working on patents, presentations, and demos. I could write three books. Being committed to an idea, an invention, and an unfunded project is like climbing a mountain and falling off a cliff at the same time.
My first approach was to directly sell or license ChipSight (a vision peripheral for pixel processing) to a semiconductor company so it could be used as a microcontroller peripheral. Then when engineers or OEMs want to develop a vision product, they just connect a CMOS camera to the vision peripheral and write the application.
There’s a chicken-and-egg problem. The chip companies that would be the best fit for ChipSight have no vision customers because they have no vision peripheral. So they wonder, what is computer vision and what are the markets?
What happens when computer vision is suddenly available as a platform for applications?
Now I am talking directly to OEMs; some of them have carried the vision for consumer vision for years. Unfortunately the semiconductor companies can only offer them big, complicated solutions. The OEMs are ready to take the next step in using cameras as sensors, but they are suffering the chronic pain of trying to squeeze TI-OMAP/Intel/ADI parts into $5.00 products. If only the MSP430 had a vision peripheral.
Next comes this website, hosting the idea of consumer vision for those interested. I will post some past projects and whatever else helps make computer vision more commonly available.
I just finished a computer vision demo using the Texas Instruments Stellaris DK-LM3S9B96 – FPGA+camera development kit. Some of the functions I included are motion targeting from RAW pixel values (with tricky filtering and no demosaicing), converting the motion pixel data into endpoints, office lighting luminance measurement with some calibration tools and a new color transform that counteracts color absorption error (reflectance spectra), another color transform for the color segmentation windows, drivers for the host PC, drivers for the ARM, drivers for the FPGA, and drivers for the camera. The FPGA had some room left so I added ChipSight on top for object filtering and feature encoding with adjustable-precision extents. Adding ChipSight took a day, the new color windows took weeks. I’ll go into some of the functions later.
The Stellaris is powerful: it can easily handle all kinds of algorithms, applications, user interfaces, and networking. It would be a crime to use up its resources on pixel processing.
After the motion and luminance functions, I did a quick, simple exercise of ChipSight. The Stellaris running at 50MHz took 40uS to read the list of features from ChipSight and pick out the red, green, and yellow stars. At 15fps x 40uS, approx. 0.06% load on the ARM. 99% of the Stellaris is free to do wonderful things. Like running a small robot with several sensors and wireless.
Stellaris computer vision, shape and color:
Before I forget, and best of all — the new Lattice MachXO2 CPLDs are perfect for mating with WLCs.
Let’s build something.
{ 2 comments… read them below or add one }
Your vision sensor technology can more efficiently replace motion sensor that use RF. Maybe it is time to go down the route of producing and selling competing product that use your vision sensor tech to replace other less efficient sensor.
Thanks for commenting!
Plus image sensors can replace multiple different sensors at the same time. For example instead of a sensor for drowsy driver detection, and an occupant weight sensor for airbag force adjustment, use one camera with a view of the front seat area.
Or use the camera on mobile devices for multi-touchless and true gesture control instead of phone rubbing.