Using 2 webcams

[hudvin]

Is it possible to get near realtime stereo video using RB?

[PaulL]

I doubt it. Your useable frame rates will be pretty low (and the resolution will need to be low), and I would guess any processing of dual images will be... Problematic. There's a "sticky" post at the top regarding OpenCV. You're obviously welcome to try, though.

Curious, why are you wanting stereoscopic vision?

[veltrop]

I'm doing stereo vision with the Roboard, but because it's so slow I offload the stereo matching to a different computer on the network. Framerates are then limited by the bandwidth. 1~2 fps with 802.11g and 8fps for 802.11n for a pair of 640x480 images. Multiply that fps by 4 for 320x240.

I just did a quick test of doing the stereo matching on the Roboard instead of offloaded, it took 30 seconds to process a frame for 128 disparities with a pair of 640x480 images. If the disparities were taken down to 32 and the images to 320x240, I bet you could get about 1 frame every 2 seconds.

So it doesn't look good for realtime stereo on the Roboard.

Here's some background info on my project btw:
http://taylor.veltrop.com/robotics/khrhumanoidv2.php?topic=veltrop-ros-pkg#stereo

The stereo matching is using ROS, which uses OpenCV. The source is freely available and the download information is also at the above link.


Here's the live output of the visualization of the Robots perception of its body in relation to the stereo point cloud. It's looking at a tatami floor, my hands, a computer, and an orange tin box.

[IgorT]

So it doesn't look good for realtime stereo on the Roboard.

One question....

Since you're offloading the work to your more powerful desktop anyway, why not use two cheap Wireless Video "Spy" Cameras, with transmitters and two receivers on the other end?

Instead of congesting the RoBoard with sending a data stream over Wi-Fi, why not simply have the cameras do their work and transmit a video signal over the air to their respective receivers and then feed the video signal from the receivers into a TV-Tuner card or any card with simple AV inputs, and let the tuner or video inputcard digitize it for your image processing?

Basically, except for the final processing part neither computer would be slowed down even a little...

Then, assuming your desktop is powerful enough to do the processing, simply send the extracted data or even only just commands back to the RoBoard whichever way you're doing it...


The resolution won't be quite 640-480, the cameras all seem to have 380 lines vertical resolution, but it's still quite a bit more than 320x200...



I think it should be doable, the only issue could be finding a video input card with more than one AV channel which can operate simultaneously, or perhaps having to use two such cards.. Some of the cameras and their receivers have selectable channels, up to four could run at the same time


Just a thought. Perhaps you already thought of this but found a problem i missed, i don't know, maybe it's worth taking a look tho.

Here are more cameras....

I haven't tried searching for video input cards yet tho.



EDIT: Here's a four channel video only input card...

Not too expensive - i selected the cheaper model without audio...
And it can display multiple channels at once as it's meant for security cameras...


The only thing i'm not too sure about is how to feed the video into your processing software efficiently and in real-time.

Unless of course the software you're using supports video capture cards. If it does it should all be very simple.


Speaking of "video capture cards" - that's the term i should have searched for instead of "video input card". Probably gives more results, perhaps something even cheaper and more suitable for your idea...

[veltrop]

I have used a setup with wirelessly transmitting cameras:

http://taylor.veltrop.com/robotics/khrhumanoid.php#flowchart
http://taylor.veltrop.com/robotics/khrhumanoid.php#stereo
http://taylor.veltrop.com/robotics/khrhumanoid.php#head
http://taylor.veltrop.com/robotics/khrhumanoid.php#24ghz

There were too many problems, mostly related to the radio signal.
900mhz models had low video quality. 2.4ghz models had slightly better video quality but they interfered with 802.11b and bluetooth. They would also turn to static if someone in my building used their microwave. I could actually see the bluetooth frequency hopping as jumping horizontal lines in the video stream. Also, you can get these security cameras on multiple channels so they theoretically won't interfere with each other. But when they are only several centimeters apart on a stereo rig, they -will- interfere with each other.

I would recommend against attempting stereo vision with wireless security cameras.

[IgorT]

I would recommend against attempting stereo vision with wireless security cameras.

I see... So it's not that easy afterall...

Well i did find one other thing tho.. A wi-fi camera, that logs on to your WLAN and creates a video server that you can access with other computers on the network.

That thing should be interference free, i'd imagine. But the store i saw it in only had the one and it was used, even damaged...

That should be capable of transmitting the video smoothly tho, if that's what it's made for, but strangely i saw no info on resolution and framerate. Likelly to be very low.


I would imagine that for stereoscopic image analysis resolution and sharpness of the images are very important, since the computer has to match areas in the two images to create a 3D map.... It would be easier for the comp if it only had to match blurry blobs, but the resolution of the resulting map would suffer horribly.


Thee is one more thing tho... Recently i saw an article, on RobotDreams i believe, about someone using a stereo camera setup from an X-Box add-on, and also it's processor and possibly the software, to put it on a robot and have it go around mapping the room in 3D.
The thing can also see people and recognize gestures - that's what it's meant for in the X-Box thingy...

Did you seethat perhaps?
The camera setup was very wide there, since apparently they simply just removed the entire front panel from the device. But perhaps it could be reduced in size a lot more...

[IgorT]

I found it but not on robot-dreams.

It's called Kinect. Here someone put it on a robot vacuum and let it explore:
http://www.vg247.com/2010/11/18/people- ... -gestures/


But since you're actively researching this field you probably already know all about it...

[veltrop]

Yeah already got an Kinect too

It comes apart into several smaller pieces nicely but it won't fit well or safely on a Kondo KHR sized robot, however I think it could be done with a lot of effort. It would probably have to be plugged into a remote computer with a view of the room containing the robot, or used as a remote control device.

BTW ROS already has a nice ready to use stack for it.

Keep in mind they don't function outdoors because the infrared gets washed out, and other infrared sources can ruin the data, but those probably aren't concerns for most people.

After taking it apart it looked really close to being able to fit on my KHR, but not really feasable. A slightly larger humanoid or wheel based robot shouldn't have much of a problem integrating it.

Another issue is that to be useful for such a small robot, it needs to perceive extremely close distances, especially for hand-eye coordination. I haven't tried to max out the close range on my Kinect, but I highly doubt the robot could use it to see its immediate (within 15-20 cm) surroundings. The FOV is quite narrow; horizontal 57 degrees, vert 43 degrees IIRC. It could be used for mapping a room though.

The narrow baseline stereo vision on my robot is optimized for quite the opposite scenario, within about 1.5 meter is is accurate, but beyond that it is unusable.

Anyway, I'd to see a Kinect rebuilt for a humanoid, once I do some more reasonable experiments with it I can let you all know if it's any good with the small hardware we are all using, since most of this post was speculation. But I'm working on so many other tangents on the robot right now that this on the back-burner.