Contents
The main task of the Visual Net is the segmentation of the whole set of 2D feature points tracked between frames into an unknown number of clusters.
The Visual Net provides two core components: clustering of features according to different criteria (represented by so-called predictors) and 3d estimation of the clustered features using structure from motion. However, structure from motion can be switched off if a 3d sensor such as kinect is used.
Package Description
The package contains only one node, called visual_net. In the following we explain which parameters can be set, and how to handle input and output data.
Information Flow
- Visual net subscribes to a feature topic, where some kind of feature tracker publishes a set of visual 2D or 3D features (as a PCL point cloud) at every time step
If certain (definable) thresholds are exceeded, namely that the global_motion of the features is sufficient and that at least min_cluster_size feature move by an amount of min_motion pixels/cm (depending on the sensor), the segmentation is triggered. If a 2d sensor is used, the structure from motion computation is triggered, too.
- After computation, visual net publishes the segmentation and (if structure from motion is used) the estimated 3d positions of the features.
Parameters
/number_features
type: int
- description: Set how many features will be published (can be set dynamically using topic /iap/num_features). The parameter is also used by visual net, feature tracker etc.
/video_sensor_type
type: string
description: Which video sensor you are using. Possible values are kinect (kinematic analyzer will subscribe to topic camera/rgb/image_color) and usb_cam (kinematic analyzer will subscribe to topic camera/image_raw)
/min_cluster_size
type: int
description: min_cluster_size features have to move at least min_motion to trigger the segmentation (and min_global_motion has to be exceeded)
/min_motion
type: double
description: min_cluster_size features have to move at least min_motion to trigger the segmentation (and min_global_motion has to be exceeded)
/min_global_motion
type: double
- description: The minimal global motion to trigger a segmentation (if also min_motion and min_cluster_size are exceeded, see above)
/segmenting_in_3d
type: bool
- description:
/visual_net/external_trigger
type: bool
- description:
/3d_from_sensor
type: bool
description: If true, kinematic analyzer subscribes to topic /iap/feature_set_3d, otherwise to /iap/estimated_feature_set_3d
If /3d_from_sensor is true, visual net also expects data from the following topics: /video_height /video_width /focal_length
Predictors and parameters:
/visual_net/use_rm_predictor
type: bool
- description: Use (2D) relative motion predictor
/visual_net/rm_min_dist
type: double
- description: Distance for 2D relative motion predictor
/visual_net/use_rm3d_predictor
type: bool
- description: Use (3D) relative motion predictor
/visual_net/rm3d_min_dist
type: double
- description: Distance for 3D relative motion predictor
/visual_net/use_fm_predictor
type: bool
- description: Use fundamental matrix predictor
/visual_net/fm_num_hypo
type: double
- description: Number of hypothesis for fundamental matrix predictor
/visual_net/fm_num_trials_per_hypo
type: double
- description: Number of trials per hypothesis for fundamental matrix predictor
/visual_net/use_sd_predictor
type: bool
- description: Use short distance predictor
/visual_net/sd_min_dist
type: double
- description: Minimal distance for short distance predictor
/visual_net/use_ld_predictor
type: bool
- description: Use long distance predictor
/visual_net/ld_min_dist
type: double
- description: Minimal distance for long distance predictor
/visual_net/ld_max_dist
type: double
- description: Maximal distance for long distance predictor
/visual_net/use_t_predictor
type: bool
- description: Use triangulation predictor
/visual_net/use_csegm_predictor
type: bool
- description: Use color segmentation predictor
/visual_net/use_csimil_predictor
type: bool
- description: Use color similarity predictor
ROS Subscribers
/iap/feature_set_2d
type:sensor_msgs/PointCloud2
- description: A set of 2D features at a time.
/iap/feature_set_3d
type: sensor_msgs/PointCloud2
- description: A set of 3D features at a time.
/iap/visual_net/run_segm_and_sfm
type: std_msgs/Empty
- description: Trigger segmentation and structure from motion computation
ROS Publishers
/iap/segmentation
type: sensor_msgs/PointCloud2
description: A segmentation of the features into clusters. Returns a list of the same length as indicated by parameter number_features, containing the cluster ID for every feature.
/iap/estimated_feature_set_3d
type: sensor_msgs/PointCloud2
- description: If structure from motion is activated, visual net publishes the 3D feature set here
ROS Services
None.
ROS Messages
None. See iap_common.