melodic noetic

ROS Mesh Tools

We introduce a set of tools to make 3D environment mesh representations accessible and manageable in ROS. We provide RViz tools to visualize and annotate huge meshes in combination with generated textures and different cost layers, which are based on the geometric analyses of the environment, or which represent different sensor readings, e.g. RGB image or even hyper-spectral image textures.

Introduction

Over the last years, the Knowledge Based Systems Group has developed an extensive set of tools to automatically generate triangle meshes from 3D laser scans. These tools are implemented in the freely available Las Vegas Surface Reconstruction Toolkit (LVR) and have been successfully applied in different robotic contexts.

With terrestrial and mobile laser scanning it is possible to create 3D point clouds of large environments in short time. However, the amount of collected data is too large to be processed on a mobile robot. To make this data available on mobile robots, we suggest to compute 3D triangle meshes from point cloud data as corresponding map representations. Such maps have several advantages over raw point cloud data and voxel representations. They overcome the discretization of voxel maps and deliver topologically connected surface representations, which ease segmentation, annotation and enable intuitive visualization.

However, to make the maps accessible in ROS, corresponding message definitions and tools for storing, handling and editing such maps are required.

Message Definitions

We divided the mesh structure in its geometry, textures and cost layers. These different components are then associated using a UUID. This structure enables passing the geometry at first to RViz and other nodes. Theses nodes may analyze the geometry, e.g the roughness or height difference and compute corresponding mesh cost layers which are published as individual message with a corresponding mesh UUID. Thereby other nodes can associate these layers with a certain mesh object. In RViz a certain layer can then be selected by the user and visualized coloring the corresponding costs or displaying the textures. In the following gives an overview on most important mesh messages:

• MeshGeometry (Stamped) mesh_msgs/MeshGeometryStamped: Geometric structure of a triangle mesh using a buffer vertices, normals and faces

• MeshVertexColors (Stamped) mesh_msgs/MeshVertexColors: Vertex color information of a corresponding mesh

• MeshVertexCosts (Stamped): mesh_msgs/MeshVertexCosts: Vertex cost information of a corresponding mesh coloring the mesh using a rainbow color scheme.

• MeshClusterLabel: mesh_msgs/MeshClusterLabel Grouping a set of triangles / faces of a corresponding mesh using the face indices, optional cluster label

• MeshTexture: mesh_msgs/MeshTexture Texture using an Image and an ID

• MeshMaterial: mesh_msgs/MeshMaterials Color and optional texture ID

• MeshVertexTexCoords: mesh_msgs/MeshTexCoords Texture coordinate

• MeshMaterials (Stamped): mesh_msgs/MeshMaterials Combining the materials with texture coordinates and clusters of a corresponding mesh

• Feature: mesh_msgs/Feature A feature at a specific location and its feature descriptor

• MeshFeatures mesh_msgs/MeshFeatures: List of features for a corresponding mesh

RViz Plugins

We developed several RViz plugins to display 3D meshes together with different information layers (RGB textures, semantic labels, trafficability, etc.). It is possible to render the received meshes in different modes (lighting, materials and wireframe). The supported modes are selectable based on availability in the tree view. For path planning, we also implemented an interactive tool to set navigation goal poses on the mesh surface. To generate semantic maps, we implemented an interactive tool that can be used to select connected clusters of triangles and assign semantic labels to them. Some screenshots and videos demonstrate these plugins (see Supplementary Material).

RViz Mesh Plugin

The TexturedMesh displays a mesh with optional a fixed color, textures, vertex colors or vertex costs. Additionally, it can display the wire-frame as well as the vertex normals of the mesh. The MeshGoal Tool provides the possibility to select a geometry_msgs/PoseStamped on the surface of the mesh.

RViz Map Plugin

The Mesh Display displays labeled faces. The ClusterLabel Display displays labeled clusters. The ClusterLabel Tool enables the labeling of certain faces using different selection and de-selection methods as shown in a video (see Supplementary Material. The ClusterLanel Panel allows to manage your clusters and label names as well as the corresponding colors.

Videos

Mesh Cost Layer

This Video shows different cost layers which are used for path planning in outdoor terrain. It shows one partly reconstructed scan in the botanical garden of Osnabrück

Mesh Textures

This Video shows a textured dataset of the botanical garden of Osnabrück including footpaths

Label Tool

This video shows the labeling tool, which is used to cluster faces to an object or a union, (in the example a door) using a mesh inside a building

Report a Bug

Use GitHub to report bugs or submit feature requests. [View active issues]