[Documentation] [TitleIndex] [WordIndex



This package contains the ROS-independent library which provides the actual scene-recognition functionality of Implicit Shape Model (ISM) trees. It is referred to both by asr_ism for Passive Scene Recognition and by asr_recognizer_prediction_ism for Active Scene Recognition.


This library consists of three main components to record, train and recognize a scene, additionally it provides a tool-set to manipulate and generate the used data.

Main components

Recorder: Provides an interface to store sets of objects to a sqlite database. Each set of objects represents a configuration of objects for a certain scene.

Trainer / CombinatorialTrainer: The Trainer and the CombinatorialTrainer are interchangeable, both create a scene model represented by an ISM from the object poses stored in the recorded database and write it into a sqlite database. The ISM created by the CombinatorialTrainer should have a shorter computing time when it comes to recognizing a scene compared to the one created by the Trainer.

Recognizer: Given a set of objects, the Recognizer calculates the most likely scenes that those objects are part of. These calculations use the scene models created by the Trainer/CombinatorialTrainer.


DataCleaner: Delete the recordings or the models or both from the database.

DataMerger: Merge multiple databases into one.

MarkerRotator: Rotate the pose of objects which are determined by a marker e.g. asr_aruco_marker_recognition.

PoseInterpolator: Generates additional recording data between objects by interpolating the pose of two objects between consecutive recorded object sets.

RecordedObjectsTransformer: Transform the absolute pose of an entire scene relative to a world coordinate frame.

RotationInvariantObjectsRotator: Rotate Objects in the database which are rotation invariant to the y-Axis around this y-Axis to a given direction. As result these objects get distinct object poses.

Data Models

Sqlite database

The ism library uses sqlite to save its recordings and trained models, which can be differentiated by the prefixes “model_” and “recorded_”. Note: It's not necessary that a database contains both, recordings and models.

Tables for Recordings

recorded_patterns(id, name) - Stores the representing name for a pattern.

recorded_sets(id, patternId) - Stores a recorded object set at a point in time for a certain pattern.

recorded_poses(id, objectId, setId , px, py, pz, qw, qx, qy, qz) - Stores the object estimation pose obtained from a object recognizer during the recording process.

recorded_objects(id, type, observedId, setId, resourcePath) - Stores the object estimation meta-data obtained from an object recognizer during the recording process.

Tables for Models

model_patterns (id, name, expectedMaxWeight) - Stores trained patterns.

model_objects(id, type) - Store all objects which appeared while training the model.

model_votes(id, objectId, patternId, observedId, radius, qw, qx, qy, qz, qw2, qx2, qy2, qz2,qpw, qpx, qpy, qpz, qpw2,qpx2,qpy2,qpz2, trackIndex, weght) - Stores the votes generated by the trainer.


XML is used to represent a scene, a pattern or just a set of objects for convenient use in simulation or to just store a certain configuration of objects for further use. E.g. use the XML to publish the contained object-set as recognized object estimations with the provided tool of the asr_fake_object_recognition package.



  <Object type="" id="" mesh="" angles="quaternion"> x, y, z, qw, qx, qy, qz</Object>
  <Object type="" id="" mesh="" angles="euler"> x, y, z, alpha, beta, gamma</Object>





First a little overview of the directories and after that a overview of the main interfaces, to encourage the user to use this library in his own projects or contribute to the existing projects.

Directory Overview


Implemented tools which can be used via terminal, but all necessary tools were ported to ros nodes provided by asr_ism. Using the nodes instead of the terminal version is recommended.


This directory contains the actual source/functionality provided by the library.

Main Interfaces


General usage:

1. Include ISM/recorder/Recorder.hpp

2. Create a new Recorder instance:

Recorder(const std::string& dbfilename)

3. Insert objects of a recorded object configuration into the database:

void insert(const ObjectSetPtr& set, const std::string& patternName)


General usage:

1. Include ISM/heuristic_trainer/Trainer.hpp

2. Create a new Trainer instance:

Trainer(std::string dbfilename, bool dropOldModelTables)

3.1.Train ism for all patterns inside the database:

void trainPattern()

3.2. Train ism only for the given pattern (patternName):

void trainPattern(const std::string& patternName)

! Before using the trainPattern method, the training parameters should be set. An example how to set those parameters can be found at asr_ism/src/trainer.cpp


General usage:

1. Include ISM/'combinatorial'_trainer/'CombinatorialTrainer'.hpp

2. Create a new CombinatorialTrainer instance:

CombinatorialTrainer(CombinatorialTrainerParameters params)

3. Train ism for all patterns inside the database:

std::map<std::string, std::pair<double, TreePtr> > learn()


General usage:

1. Include ISM/recognizer/Recognizer.hpp

2. Create a new Recognizer instance:

Recognizer(const std::string& dbfilename, double bin_size, double maxProjectionAngleDeviation, int raterType = 0)

3. Recognize most likely pattern for a given object configuration:

const std::vector<RecognitionResultPtr> recognizePattern(const ObjectSetPtr& _objectSet_, const double filterThreshold = 0.0, const int resultsPerPattern = -1, const std::string targetPatternName == "")

ROS Nodes

This package only provides the library without any nodes, however the package asr_ism implements nodes for the actual usage of the recognition system.

2024-07-13 12:37