Using PointGrey Research Cameras with OpenCV 2.1.1 in Macs

Point Grey Research makes really good industrial cameras that are suitable for many kinds of image processing researches. The big con for me is that they don't or plan to offer support for Mac OS X. Luckily with the new Quick Time libraries (QTKit.framework) we can use those cameras.

There are two alternatives, you can become a native citizen and use Objective-C, Cocoa and QTKit directly or you can use OpenCV (a C/C++ image processing library) that uses QTKit internally.

OpenCV is an open source project for image processing, computer vision, machine learning and many other areas, is very useful and developed by a very active group!. Since version 2.1 OpenCV uses QTKit.framework to get frames into images (IplImages).

This is just a couple of notes you should take in count when using this unsupported cameras, you have two options:

Option 1: Modify your program

When using PGR cams in macs you MUST set a width and height to the Video Capturer like this:

using namespace cv;
int main(int, char**)
{
 VideoCapture cap = VideoCapture(1);
 cap.set(CV_CAP_PROP_FRAME_WIDTH, 320); //LINE ADDED
 cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240); //LINE ADDED
 if(!cap.isOpened()) return -1;

 namedWindow("edges",1);
 for(;;)
 {
  Mat frame;
  cap >> frame;
  imshow("edges", frame);
  if(waitKey(30) >= 0) break;
 }
 return 0;
}

If you don't set a width and height then the window will never appear. No errors, nothing happens. Probably you will be debugging for hours or end up by using another camera.
If don't want to modify every single program you have, you simply modify OpenCV, is less than 10 lines including building and installing. These is the best part of open software stuff!

Option 2: Modify OpenCV

Find the file to modify

Locate file "cap_qtkit.mm" and open it. It should be in "opencv/modules/highgui/src/" or just do a search.
Find startCaptureDevice() method implementation, it should look something like this:

int CvCaptureCAM::startCaptureDevice(int cameraNum){
  ...
}

Modify the file

Inside startCaptureDevice() method you will find these lines:

NSDictionary *pixelBufferOptions ;
if (width > 0 && height > 0) {

We are gonna modify the code just before those line as in the picture:

Here is the code:

//start of modification
if (width == 0 && height == 0 && [[device description] isEqualToString:@"Camera"]) {
 width = 640; height = 480;
}
//end of modification

Now just built and compile OpenCV and your are done!
(You can refer these instructions if you don't know how, also make sure you have your cmake updated)

Explanation

PGR cams are not made to work with macs, but happily they conform to some protocols (I believe one of them is ieee1394) so they actually do!. They problem seems to be in some of them default parameters
So when we are just about to initialize a PGR cam, we simply set good defaults.

How to recognize a PGR cam? I have realized that when printing all devices descriptions PGR cameras shows a string Camera.
So printing all devices in a MacBook with a PGR cam will show something like this:

"Built-in Input",
"Built-in Microphone"
"Built-in iSight"
"Camera"

That is why we check for a device whose description is Camera and its width and height hasn't been set.

More on PGR cams and their pixel formats:

I have tested with QTKit.framework and Objective-C and only a reduced group of pixel format seems to  work with my PGR cam (Grasshopper) . I said seem to work because I didn't check the pixels, I just used a QTCaptureView object to see the results.

kCVPixelFormatType_32ARGB
kCVPixelFormatType_32BGRA
kCVPixelFormatType_32ABGR
kCVPixelFormatType_32RGBA
kCVPixelFormatType_64ARGB

Only those worked for me, but I didn't test all of them (they are too many!) Even though this is more than necessary to work with raw pixels.

References

• Using PGR Imaging Products under Macintosh OS X.