Nao Robot: Visual debugging of the vision variables

One of the more interesting things of the Nao Robot SDK is, probably,  the Vision Module and all the work it can do for you. For example, Nao can detect faces and provide you the coordinates of the different faces with a lot of face features. However, I have found  few information about how to use the variables provided by Nao, so I have needed to use the old method of try/error to understand the variables.

Accessing the variables

Thanks to the ALMemoryProxy accessing to the FaceDetected variable is very easy:

mMemoryProxy = new AL::ALMemoryProxy("NaoRobotsIP", 9559);
AL::ALValue face = mMemoryProxy->getData("FaceDetected");

Understanding the variables

(Disclaimer) I have not found official documentation about what the information inside FaceDetected means and this explanation is based in my experiments and maybe I have get some mistakes.

All parameters have the centre of the image as origin and they have not units. We will pay attention only to the ShapeInfo variable:

  • alpha: Vertical offset in relation to the image height with origin in the centre of the image
  • beta: Horizontal offset in relation to the image width with origin in the centre of the image
  • sizeX: The face size in relation to the image height. It should be the width (as we are talking about the X axis), but in this case the rectangle around the face is wider than tall
  • sizeY: The face size in relation with the width
Reference system for the Vision Module variables

Reference system for the Vision Module variables

Results

Once we have transformed the variables provided by Nao, we can draw a simple rectangle around the faces detected using OpenCV.

// Face Detected [1]/ First face [0]/ Shape Info [0]/ Alpha [1]
float alpha = face[1][0][0][1];
float beta = face[1][0][0][2];
float sx = face[1][0][0][3];
float sy = face[1][0][0][4];

// sizeX / sizeY are the face size in relation to the image
float sizeX = mImageHeight * sx;
float sizeY = mImageWidth * sy;

// Centre of face into the image
float x = mImageWidth / 2 - mImageWidth * alpha;
float y = mImageHeight / 2 - mImageHeight * beta;

cv::Point p1(x - (sizeX / 2), y - (sizeY / 2));
cv::Point p2(x + (sizeX / 2), y + (sizeY / 2));

cv::rectangle(mImage, p1, p2, cv::Scalar(255,255,255));

Nao camera debugging 1

Conclusions

The docs explain that the face detection algorithm is provided by OKI. It can save you a lot of work (computational work) if you want to avoid to transfer images over the network and use the OpenCV classic face detection algorithm to know how many faces are or where are they.

However, the values related with others features, like eyes, are no so useful:

Eye Centre (X) Eye Centre (Y) Eye Top Limit (X) Eye Top Limit (Y) Eye Bottom Limit (X) Eye Bottom Limit (Y)
0.112876 -0.121094 0.126156 -0.128013 0.126156 -0.114174

As can be appreciated, the values for the TopLimit and TopBottom coordinates are almost the same. The debugging result is a point over the eye and not a rectangle as it could be expected.

After several test with different faces I am not very satisfied with the results obtained. The Nao’s algorithm provide a good approximation of the faces coordinates but the faces features are inaccurate. This implies you will need OpenCV (or any other library) to search for features.

Leave a Reply

Your email address will not be published. Required fields are marked *