Reminder

• The tracker uses CORRECTED images.

  • Be sure to calibrate your camera and subscribe to the image topic after correction (see image_proc).

• The tracker uses an initial pose.

  • You will need to use either the GUI or an initial estimation to start the tracker.

Tips and tricks

How fast is the tracking?

Each time a new pose is computed, a new pose is published. Therefore, to see your tracking frequency you can run:

$ rostopic hz /tracker_mbt/result

There is no ideal tracking frequency as it depends on how fast your object moves. However, if the frequency is less than ~25hz, the tracking will probably fail.

The tracking initialization fails

If the initialization using the GUI fails, it is either due to the model (bad format, failed to parse...) or due the camera calibration parameters.

The tracker uses the K and P matrices information as well as the camera distorsion model. Make sure that your camera is correctly calibrated (using camera_calibration for instance). See also the message documentation sensor_msgs/CameraInfo.

The tracking fails at the first iteration after initialization

Make sure the model normals are correctly set. The tracker eliminates invisible faces. If normals are wrong, the tracker will remove all the object faces.

My tracking is too slow!

Several factors can slow down your tracking.

First, you will never be able to track at a higher frequency than the rectified image topic frequency. Tracking is only done once per image and stops until a new image is received.

Therefore, you should take a look at your rectified image topic frequency by running:

# Topic name should be changed to match the topic you use.
$ rostopic hz /camera/image_rect

Once you have checked that the tracking frequency is way lower than the rectified image topic frequency, it means that you may have problem.

• Is the grabbing and processing on different computer? This setup is not
  • recommended. Make sure in the tracker node output that no warning appears regarding message synchronization. The node only tracks when receiving synchronized image and camera calibration data. If your network is too slow, it may happen only once in a while.
• Is your model too complex? You can try to reduce the number of lines
  • in your model.

• Change the tracking parameters. Decrease the mask_size,

  • the sample_step and increase the ntotal_sample to run the algorithm faster.

Use image_transport to use compressed video streams

If the tracker is on a remote machine, you may use the additional _image_transport parameter to use compressed video streams. See image_transport documentation for more details about compressed streams.

It will reduce the amount of data passing through the network but will induce a delay in the transmission and add CPU load on the remote machine which has to compress the flow.

However, do not use it when launching the tracker node. The camera device and tracker node should be on the same machine to avoid delays and in this case, it is not necessary to compress the stream. On the opposite, it will decrease the tracking quality.

On the opposite, you can used compressed streams for the client and viewer. It is not recommended for the client as it may decrease the click precision but is highly recommended for the viewer.

The viewer ensures that the projected object pose and camera data are synchronized which mean that even if a delay is induced by the stream compression, displayed information will remain consistent.

The camera calibration is crucial

The main idea behind the virtual visual servoing is to project lines from a 3d model and compare them with the image lines. If the camera rectification is bad, it will be impossible to find a pose where the projected lines and the real one are at the same location. Therefore the algorithm will fail.