Wednesday, June 10, 2009

Interfacing usb webcam to PIC32 microcontroller.

I recently started experimenting with pic32 as it has a USB OTG host mode that can be used to connect cheap peripherals. First peripheral i tried was a 4 yrs old USB camera that was lying around unused. However getting documentation for it was a problem as most of the cheap webcams vendors don't even have a website. The only documentation i could find about it was from http://mxhaard.free.fr/spca5xx.html. Since I knew that webcam worked with my laptop (running linux) so i was sure that this driver is already incorporated into linux kernel source tree. Only approach left now was to go through the driver code and port it to pic32.


Finding source code for driver didn't took long. Basically, hook up camera to laptop and note down the vendor & product id in output of "lsusb". For my camera vendor id was : 0x04fc and product id was : 0x0561. Download linux kernel source code and grep with product id in drivers/media/video. It throws up a file names for sunplus driver. Cross check it with the driver in the output of "lsmod" output and voila we have the original sources for the driver.


However, driver source had lot of linux specific stuff for video and v4l support so next step was to write a new bare bone driver in linux from the original source. This new driver was to dump more USB specific information about cam like interfaces, end-points and alternate configs to use and also the steps required to configure the webcam.


Webcam USB interface details:
Webcam exposed few i2c and reg read/write interfaces through the control end-point. These were used for writing the initial startup-code, configuring various params like brightness etc. and also reading auto-gain settings. It also presented iso end-points with different buffer size in different alternate config to meeting the bandwidth requirements. For my purpose i used the alternate config 0x07 with end-point buffer size of 1024. There are other end-point with smaller buffer-size like 256 also. Camera streaming is initiated by writing few config setting through i2c interface and once started frames are received on iso endpoints. A single image is broken down into multiple frames of 1024 size and sent through iso end-point. Frames are of different types (FIRST_FRAME, INTER_FRAME, LAST_FRAME, DISCARD_FRAME) and driver on host is responsible for stitching up all the frames together to form a complete image. The image transferred by camera is in a propitiatory format and needs to be converted before a imaging software can be used to display it.


A user-application to read frame grabbed by the driver and dump it in a bmp format was also required. This however posed a problem as camera frame format wasn't documented much. Little fiddling in the original source code and googling threw a possible solution to use a open-source library libv4lconvert. It was a smooth ride from there-on to dumping the bitmap. So finally we had the code & more importantly initialization and configuration steps for reading a image from camera.


Porting it to pic32 with microchip usb library didn't took much time except for few things like setting desired alternate configs to use in pic32 USB library.


Image captured from webcam:



Complete source code for user-application & pic32 firmware are available for download.
Source code is embedded into the following image files using OpenStego availabe at http://openstego.sourceforge.net.

For detail procedure on how to recover the source file take a look at http://bytewerx.blogspot.com/2009/07/uploading-files-to-blogger.html.

Do make sure to check the md5sum of recovered source files.

application_zip.png


Image_grabber_hex.png


Firmware_tar_gz.png




md5sum :
application_zip.png : 708548fb26119aeec77c0dc7cce3ac02
firmware_tar_gz.png : da76ba3bf1394be861690bb11c7cc22e
image_grabber_hex.png : 25cb132b3c45cd06a5ed6dc827c4dcda

application.zip : c6876b534fb98c222eebc01a1af8ba7c
firmware.tar.gz : d01cae86832a4e87fa50e92df783bef5
image_grabber.hex : 0656b15fecd78c5f7662c9bc3c18513c

You can alternatively download the applications from my dropbox (www.dropbox.com) public folders :
application.zip
firware.tar.gz
image_grabber.hex

38 comments:

Anonymous said...

Jack

Could you make the firmware available. I can not download it from here as it has reached its limit

jack said...

You can try :
http://rapidshare.com/files/259589460/firmware.tar.gz.html
http://rapidshare.com/files/259589638/image_grabber.hex.html
http://rapidshare.com/files/259588748/application.tar.gz.html

jack said...

I have also uploaded sources onto the blogger using openstego.
Take a look at http://bytewerx.blogspot.com/2009/07/uploading-files-to-blogger.html on how to decode them.

andrea said...

Hi jack

I would like to do this project myself, but I need some more info. Which pic32 did you use? How can you get the modified driver into de microcrontroller (do you need a linux distro or something?)?
I really want to try this out, but I don't know where to begin. Any help would be greatly appreciated =)

jack said...

Hi Andrea,

I used PIC32MX440F512H. These are with the USB host OTG support.
Schematics of board are also there in blog at http://bytewerx.blogspot.com/2009/06/pic32-devlopement-board.html

Getting the documentation for a USB webcam is tricky part. Webcam companies hardly give away the documentation for there products.
Your best bet would be to see if you can get a webcam supporting UVC (USB Video Class). This is a USB standard and well documented.

For other webcams with as in my case (as mentioned in the blog also), I took the approach of studying the linux driver itself. For this it is better to have the linux distro so that you can recompile the driver and dump more debug info.

Once you have figured out how to initialize the webcam and how to stitch together the image frames, next step would be to convert the image from webcam propitiatory format some other well known format.
If you are lucky then your webcam may support standard JPEG otherwise you need to take a look at libv4lconvert library (as mentioned in blog). Linux heavily uses this library for converting webcam image files to standard formats. For this again i think you would need a linux distro.

hope it helps.

Anonymous said...

i can not open images with openstego. it says password is needed. can you help?

Biplav said...

Can you plz mail me the source code to biplav.saraf[at]gmail[dot]com

Open Stogo is not able to decode these pic files and rapidshare is giving some prob.

Thanks.

Lucio said...

Jack,
have you made any further progress?
Please post more of your findings..

Lucio

Anonymous said...

Hi Jack!

cannot decode the firmware too ...
It would be very fine if you could send me the files per mail to renate[at]lehotzki[dot]at.

many thanks

Renate

Anonymous said...

Hi, it's a very good work
I would like to try it, but I can get the files you mention.

Could you please send them to
freexmao(a)yahoo.com

Thx...

jack said...
This comment has been removed by the author.
jack said...

Apologies for delay in reply.
I was on vacation and didn't checked my blog for sometime.

None of the image files are password protected and i was able to extract code from all of them.

It seems simply clicking on image and using "save as" doesn't saves the complete image but just a html page with link to the blog. So before running openstego ensure that you have downloaded complete png file by opening it in a image viewer.

I have also updated "Uploading files to blogger" entry with a note.

Let me know how it goes.

Anonymous said...

Hi Jack!

Great you are back again...
Tried it again and opened the .png with a viewer (worked). But when trying to unpack it with openstego it gives the message "Embedded data is corrupt OR invalid password ... OR no algorithm found"
It would be very helpfull if you could provide me with any other way to get the data

kind regards

Renate

jack said...

@Renate,

what is your email address ?

Hadi said...

Can you please send me the files to my email.
hadi_badri@yahoo.com
Thanks

Unknown said...

Try dropbox, it's useful for file hosting because they have public folders etc.

www.dropbox.com

Thanks for the project hopefully I can get openstego to work, I feel like I'm doing top secret work all of the sudden :)

jack said...

@Cy Brown

Thanks for the suggestion. Will try it

Luci0m said...

Can you please send me the files to:
lucio.macellari@gmail.com

I can't extract them from images.

Thanks
Lucio

jack said...

I have uploaded the files to my public folder on Dropbox (www.dropbox.com).

Application.zip :
http://dl.dropbox.com/u/4373040/bytewerx.blogspot.com/version_1/application.zip

Firmware.tar.gz :
http://dl.dropbox.com/u/4373040/bytewerx.blogspot.com/version_1/firmware.tar.gz

ImageGrabber.hex :
http://dl.dropbox.com/u/4373040/bytewerx.blogspot.com/version_1/image_grabber.hex

README.txt
http://dl.dropbox.com/u/4373040/bytewerx.blogspot.com/version_1/README.txt

John said...

Hello Jack,

will you try use PIC32 with webcam for isochronous transfers? I can't find any other project about using PIC32 with webcam and using for video streaming.

jack said...

@john,

At present, I don't have any good application in mind with video streams.

Let me know if you have anything interesting.

John said...

@Jack:

I'd like to connect webcam via SPI to DSP. So I need use isochronous transfer mode, right? I cant find any info (samples) about isochronous mode (or UVC) in PIC32.

Thank you Jan

jack said...

@john,

I don't think SPI has anything to do with isochronous mode.
isochronous mode is one of the USB modes and support depends upon the camera. Most cameras would support transfer of video streams on isochronous end-points.

As for the SPI it is just a way to transfer data to/from uC like UART.

I will be updating the code soon for few bugfixes along with SPI support for image transfer. UART currently is too slow and am not able to transfer more than 1-2 images per sec out of pic32.

Anonymous said...

Dou you have a list of all the things I need to start with????

I need to purchase them from microchip and other sources

alberto said...

I am confused now. Did you port the image to an LCD? Or did you just safe it in the pic? Does the resolution of the web cam affects?

jack said...

@Anonymous:
I use PIC32MX440F512H. Rest all the components & there values are provided in the schematic under "PIC32 Development Board" post.

@alberto:
For verification, I transferred the image to PC through UART.

ISD said...

It is really a good idea could you send me more details around it at:
mhamour.hamour@gmail.com
thanks

Jenik said...

Hello, have you any progress regarding implementing SPI transfer?

Anonymous said...

want to interface usb webcam with PIC32MX795F512L and store the captured images directly on Pen drive without PC. Need your guidance as i new to PIC applications. Don't know how to start.

Coolvibes said...

Writing to a pen drive could be an issue as I believe the usb host stack does not support multiple usb devices (hubs). I am using a wifi module to post video image files to my server and leaving the usb free for camera / bootloader etc.
Use jacks method saving to a card.
Thanks to jack for posting the source. I will implement it as soon as I get a uvc class camera.
Dave

Unknown said...

hi, its very informative,usb protection , thanks

rax said...

hi,You have a very good blog that the main thing a lot of interesting and useful! thanx. usb protection

paporas said...

Hi,
great blog! Are you just capturing a single frame or can you capture multiple FPS? Are there enough resources in the PIC32 to apply an image processing (quite basic) algorithm as the image stream is coming in?
Thanks!

Sandeep Senan said...
This comment has been removed by the author.
Sandeep Senan said...

Hello jack,

Found your work quite interesting, i am an entrepreneur from bangalore.Would like to have a chat with you to discuss some plans.Please let me know your mobile number so that i can get in touch with you.

Regards
Sandeep
My mail id is : sansenan@gmail.com
or 9844-179961

Robert Rosu said...

Hi,

I'm Robert and I'm working an a project that includes this video capturing that you described on your blog. I wanted to know if using your source code I can make a video streaming to a psp through wifi?

Robert Rosu said...

and also if you could help with some advice on how to improve the quality and fps if possible.

Thanks

Robert Rosu said...

Waiting for your advice...