Giving Your Robot Eyes: Image Processing on the SBC with OpenCV

Having a webcam is all well and good, but what can we do with it? Well as it turns out, rather a lot. Computer vision (CV) is a big part of advanced robotics and CV algorithms have become the subject of much interest in recent years. Allowing your robot to “see” its surroundings is one of the most powerful things you can do to allow it to perform complex tasks. One of the most prominent tools in CV development is OpenCV. OpenCV is a set of libraries developed by Intel to aid in image processing tasks. Luckily for us, OpenCV works well on the SBC and can be easily installed.

Note that many image processing applications are extremely computationally intense and may be too much for the SBC to handle. This makes it important to test your application as early in the development cycle as you can so you can get a good feeling of how the SBC will cope with the amount of work you are requesting of it.

This post is going to walk through setting up OpenCV, and then writing a small program that detects red objects in an image.

 

Setting up OpenCV

In order to use OpenCV on your SBC you will first need to install the libraries and development files. To do this we need to configure the SBC a little bit:

Make sure the C/C++ Development tools/headers are installed to work with OpenCV

Ensure the C/C++ development tools are installed.

OpenCV SSH2

Ensure the SSH server is enabled.

Then connect to the SBC via an SSH terminal. Since our desktop computer is using Windows, we’re using PuTTY. Mac and Linux users can use the openSSH terminal application. If you are uncertain on how to connect to an SBC via SSH, we encourage you to check out our getting started on SBC video, which devotes some time to the topic. Once we’ve connected to the SBC, we’ll install all of the necessary openCV libraries with:

apt-get update
apt-get upgrade
apt-get install libopencv-core2.3 libopencv-core-dev libopencv-dev libcv2.3 libhighgui2.3 libcvaux2.3 libcv-dev libhighgui-dev libcvaux-dev

If apt-get cannot find these packages that may mean there are newer versions available. By running the following commands, you’ll be able to see if the version in the repository is something other than 2.3:

apt-cache search opencv

In order to make sure OpenCV installed properly, check to see that the files are in the right location.

ls /usr/include/opencv

In particular you are looking for cv.h and highgui.h. It’s possible that a different version of OpenCV may put highgui.h in a separate folder and give it a slightly different name (highgui_somecharacters.h). If you do not see both cv.h and highgui.h in /usr/include/opencv then you may have to search around a bit to find them.

OpenCV check

The header files are where we expected them to be.

 

Using OpenCV in your program

Once you have installed OpenCV you can begin writing your code. Let’s start with an example we already have: detecting red pixels with opencvtest.c

#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <include/cv.h>
#include <include/highgui.h>

int main(int argc, char *argv[])
{
	int redpixelcount = 0;

	if(argc < 4){
		printf("Usage error: a.out <REDLOWERLIMIT> <GREENUPPERLIMIT> <BLUEUPPERLIMIT>\n");
		exit(1);
	}	

	CvCapture* capture = cvCaptureFromCAM(CV_CAP_ANY);
	if(!capture){
		fprintf(stderr, "Error: capture is NULL \n");
		getchar();
		return -1;
	}

	IplImage* img = cvQueryFrame(capture); //load image from webcam capture
	IplImage* copy = cvCreateImage(cvGetSize(img),8,3); //create a new image of 8 bit 3 channel

	CvScalar s, c;

	printf("Searching for Red pixels...\n");

	for(int i=0;i<(img->height);i++){ //in the 2d array of the image.. count till the vertical pixel reaches the height of img
		for(int j=0;j<(img->width);j++){ //in the 2d array of the image.. count till horizontal pixel reaches the width of img
			s=cvGet2D(img,i,j); //get the RGB values of img's i,j, into a scalar container, s
			if((s.val[2]>atoi(&*argv[1]))&&(s.val[1]<atoi(&*argv[2]))&&(s.val[0]<atoi(&*argv[3]))){//if(RED>redlower && GREEN<greenupper && BLUE<blueupper)

				c.val[2] = s.val[2];
				c.val[1] = s.val[1];
				c.val[0] = s.val[0];
				cvSet2D(copy,i,j,c); //make the corresponding pixel 100% red in the img copy
				redpixelcount++;
			}
			else
			{
				c.val[2] = 255;
				c.val[1] = 255;
				c.val[0] = 255;
				cvSet2D(copy,i,j,c);
			}
		}
	}
	printf("Red pixel count is: %d \n", redpixelcount);

	cvSaveImage("original.jpg",img);
	cvSaveImage("redDetect.jpg",copy);

	/*cvNamedWindow("Input",CV_WINDOW_AUTOSIZE); //make a new window
	cvShowImage( "Input", img ); //Display img in a window named “Input”
	cvNamedWindow( "Output", CV_WINDOW_AUTOSIZE ); //Create a window “Output”
	cvShowImage( "Output", copy ); //Display copy in a window named “Output”
	cvWaitKey(); //Wait till the user presses a key or cvWaitKey(10) will wait for 10ms*/
	cvReleaseImage( &img );
	cvReleaseCapture(&capture);
	return 0;
}

You can download the full source code here.

 

A note on compiling

To compile you will need to link the correct libraries. To do this use the pkg-config tool. You may need to install it with apt-get before you can use it.

g++ opencvtest.c `pkg-config --cflags --libs opencv`

If you are not copying and pasting these commands, but rather typing them out manually, then you need to ensure that you use backticks (`) rather than the more common front tick (‘). It may also be necessary to link to the highgui.h separately based on what path it got installed to (likely opencv2 or highgui)if you had to search for it previously. Note that since OpenCV is designed for C++ you will need to use g++ rather than gcc in order to compile. It should have been installed when we enabled the C/C++ development tools.

Results

To run the program, we will execute the binary file we just compiled. We do this with:

./a.out 100 75 75

What that means is it will accept any pixel with an RGB value of >99, <76, <76 for the processed image. The program will output two JPG files; original.jpg which is the original image from the webcam, and redDetect.jpg which is the processed version. As you can see below, it did a pretty good job of singling out the red pixels. These numbers could be fine-tuned a bit so that it picks up less of the robot’s outline, but it obviously works in principle.

original

The original image, as taken by the webcam.

Red detected after processing with OpenCV on the Phidgets SBC

The processed image, with red pixels isolated.

Getting these two images can be easily done with the file editor functionality of the SBC web configuration tool:

Downloading the images with the SBC web configuration tool.

Downloading the images with the SBC web configuration tool.

This is only a simple example of some of the things that be done with computer vision. In a bigger application processing of the images coming from the webcam can be done for a more practical purpose, and in a more real time fashion. For example, one could have a robot that detects red objects and tries to move towards them. The applications of computer vision are limitless, so go ahead and experiment!

Advertisements
Tagged with: , ,
Posted in How-Tos
One comment on “Giving Your Robot Eyes: Image Processing on the SBC with OpenCV
  1. […] always an easier option to ship the order to Finland and pick it up from there. No exaggeration. UPS even advises customers not to ship any dutiable items to individuals outside major cities in Russia. For this reason, […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: