Using OpenCV Vision Library on RoBoard

[roboard offline]

Hi, the following is some information for everyone who is interested in using OpenCV on RoBoard.

Testing OpenCV on RoBoard is very easy (provided that you have installed WinXP on RoBoard): Download OpenCV and install it into RoBoard. Run OpenCV's sample programs to see the results (you may need to connect a USB webcam first).

Currently, we sugguest that you use OpenCV v1.0. Some functions of OpenCV v1.1pre1 use Intel SSE instructions and can fail on RoBoard, because RoBoard's CPU does not support both MMX and SSE instructions. (By the way, when compiling your RoBoard programs, do not enable the Pentium MMX or i686 optimization options; otherwise, your programs could fail on RoBoard.)

In our test, many OpenCV sample programs, such as CamShift and Face detection, run well on RoBoard, except the speed may be somewhat slow (due to these samples taking a high image resolution).

Keep in mind that RoBoard's performance is about 800MIPS; it may not perform fast enough in scenarios that may require INTENSE processing. However you should be OK for general purpose vision processing at low~medium res.
For your reference, we have tried the OpenCV CamShift function on RoBoard with 160*120 images captured from a USB webcam in realtime; we got the frame rate between 20fps and 30fps.

See the CamShift test at:
http://www.youtube.com/watch?v=JdnRUkWIsxw

phpBB [media]

[roboard offline]

The following is some information for people who are interested in rebuilding OpenCV for RoBoard:

Since RoBoard's CPU doesn't support MMX, SSE, and SSE2 instructions, you need to disable the option in OpenCV's source code. To do this, find the following code segment in cxtypes.h of OpenCV 1.1pre1 or 2.0:

Code: Select all

  #if ((defined WIN32 || defined _WIN32 || defined WIN64 || defined _WIN64) && \
      (_MSC_VER >= 1400 || defined CV_ICC)) \
      || (defined __SSE2__ && defined __GNUC__ && __GNUC__ >= 4)
    #include <emmintrin.h>
    #define CV_SSE2 1
  #else
    #define CV_SSE2 0
  #endif

and change it to:

Code: Select all

  #if ((defined WIN32 || defined _WIN32 || defined WIN64 || defined _WIN64) && \
      (_MSC_VER >= 1400 || defined CV_ICC)) \
      || (defined __SSE2__ && defined __GNUC__ && __GNUC__ >= 4)
    //#include <emmintrin.h>
    #define CV_SSE2 0
  #else
    #define CV_SSE2 0
  #endif

  #undef __MMX__
  #undef __SSE__

Now you can rebuild OpenCV using, e.g., VC2005 to get an OpenCV binary compatible to RoBoard.

Note that some 3rd party libraries used by OpenCV may still employ MMX, SSE, and SSE2 instructions for performance optimization. But most of OpenCV core functions in the rebuilded version should work fine on RoBoard.

[limor]

OpenCV released their V2.0 yesterday. it is based on the 1.1 which has been around in SVN. have you tested the new version for compatibility ?

[roboard offline]

OpenCV released their V2.0 yesterday. it is based on the 1.1 which has been around in SVN. have you tested the new version for compatibility ?

We have tried several OpenCV 2.0 functions, such as face detection, on RoBoard, and they still work fine.

Some functions that originally use SSE2, e.g., cvSmooth() with CV_MEDIAN method, also work on RoBoard in the rebuilded version.

[IgorT offline]

For your reference, we have tried the OpenCV CamShift function on RoBoard with 160*120 images captured from a USB webcam in realtime; we got the frame rate between 20fps and 30fps.

See the CamShift test at:

Wait, the entire RoBoard is doing nothing more than tracking an object on camera and moving the servos accordingly and the frame rate is no higher than 20-30?

How much processing powe,r does that leave for other potentialy complex algorithms trying to run on it?


I was kinda expecting more... If the resolution already has to be only 160x120, why not use the HaViMo for simple object detection and tracking, freeing up the RoBoard for other complex processing (if needed of course)?

Does OpenCV offer much more than the HaViMo board? Examples perhaps?



Otherwise, why does the tracker arm in the video twitch so much? Is it because of reduced processor power due to image processing?


Did you ever try simple face detection? How fast would that work, together with servo tracking? Hopefully a lot faster...

[shervin.emami offline]

I think 20-30fps is reasonable for what it is doing. Its rare to get any higher than 30fps when doing any sort of computer vision task even on a PC, let alone a tiny embedded device!

Something you should consider is that most USB webcams don't work faster than 20-30 FPS even on a PC, so this in itself will limit your max speed even if you don't do any image processing or controlling servos.

The standard face detection system in OpenCV usually runs between 2-15 FPS on a PC (if you use typical settings which are fairly slow but very accurate), so its obviously going to be slower on the RoBoard, but maybe not that much slower.


Cheers,
Shervin Emami.
http://www.shervinemami.co.cc/

[PaulL offline]

Otherwise, why does the tracker arm in the video twitch so much? Is it because of reduced processor power due to image processing?

I believe there are two reasons for the twitching:

1 - Lack of accel / decel. I believe the moves in the video are linear interpolated, as in the RoboIO RCServo functions. OpenCV says the image has moved, the servo move code says "move to x / y position", and with no accel / decel, the servo starts moving at a fixed speed, and stops moving from the fixed speed, causing the mechanism to show that inertia truly is a real-world phenomenon.

2- Not the mechanism's fault per se, but the mechanism has some flex in it, and the sudden starts and stops result in some flex at start and end of moves.

The higher the speed / greater the mass, the more accel / decel is needed for smooth movement.

If you watch the video, for smaller, slower corrections, the jerkiness is much less, and you can see it does not accelerate and decelerate for any of the corrections.

[roboard offline]

From OpenCV 2.1, the type of the host CPU is automatically recognized on-fly. So the new OpenCV now can work on RoBoard without rebuild.

[bermooda offline]

hi
im using opencv 2.2 , but it shows just white frames!
and it works fine on windows 7,vista,xp on the other PCs and laptops.
i have tried to fix cxtypes.h file, but i couldn't find it!
the point is that on laptop's windows xp when i change windows color mode to 16-bit, it shows white frames just like my RB-110. but in 32-bit mode it works fine.
unfortunately when i change color mode to 32-bit on my RB-110, the problem of white frames is still remaining!

any clues?

P.S.
i have tried both on windows xp SP2 and sp3.

[roboard offline]

...
unfortunately when i change color mode to 32-bit on my RB-110, the problem of white frames is still remaining!
...

Hi, bermooda,

we have tried OpenCV 2.2 on RB-100 & RB-110 and everything worked normally. We guess that your issue is possibly caused by your camera driver, and you may try another camera or try the older OpenCV 2.1 again.

[bermooda offline]

thanks for reply

my camera is ok when i use other apps, like my old application written in C# using AForge.NET library. its plug and play A4Tech camera and no driver needed to capture video at all.

[robaldo offline]

i have rb110
i want to know what is the FPS of camera for 160*120 captured image size?
as u told its 25 fps
but i need to know it is 25 fps with process or without process?
is it directly using open cv?
i want to use open cv on my roboard and the FPS (with processing image) is really important to me
please help
thanks

[PedroR]

Hi Robaldo

25 fps seems the standard FPS when capturing normal video (wihtout processing).

If you add OpenCV processing on top of that the number of FPS is likelly to be reduced significantly.

There is no exact way of telling how many FPS it will do as it depends on the number of processes you will have running on your system as well as the type of vision processing you'll be eprforming (for example for BLOB tracking you should get higher FPS than Face tracking - both are OpenCV applications).

I would recommend that you test first if you have access to a Roboard.

Regards

[robaldo offline]

yes i knew that
as i saw in the video , there is some processes and the FPS is more than 15 (i think)
its enough for me using open cv and processing the image!
testing is the best way to know what i need

thanks