[Documentation] [TitleIndex] [WordIndex] 

API review

Proposer: Mirza Shah

Timeline: Please submit feedback by September 5, 2012.

Reviewers:

Contents

  1. API review
    1. API Overview
      1. Motivation
      2. Solution
      3. Solution Design Details
    2. API
      1. Interface 1 - Global Functions
      2. Interface 2 - Class Loader
    3. Question / concerns / comments
    4. Meeting agenda
    5. Conclusion

See the APIReviewProcess for guidelines on doing an API review.

The current implementation can be found at GitHub: ros/plugins.

API Overview

The plugins package is part of an endeavor to refactor the existing plugin system in ROS known as pluginlib. plugins provides a ROS-agnostic way of loading and unloading dynamically-linked libraries during run-time, inspecting what classes (i.e. plugins) are contained within those libraries, and being able to create objects of those classes with only the definition to a base class (i.e. no need for derived class declaration).

Motivation

In its current state, pluginlib has a few shortcomings that we would like to resolve:

  1. Dependency on the ROS build system meaning that applications that want to utilize pluginlib but not use the ROS environment cannot do so.
  2. Since each pluginlib uses separate named manifests for each class registration the plugin XML also contains this magic name (the name attribute of the class tag). This makes it impossible to use the shared library without knowledge of these magic names and introspect it for available classes.
  3. It is not thread-safe

  4. It utilizes a hacked version of the third-party Poco library which is undocumented and included as part of the baseline code.

Solution

In order to fix the above mentioned shortcomings, the following has been done:

  1. Break up pluginlib into two libraries: plugins and pluginlib. plugins is a new ROS independent package (for which this API review page is being done) for loading/unloading plugins. pluginlib keeps the same interface but sits on top of the new plugins package.

  2. Since the new plugins library does not depend on ROS it does not use/require any XML files to work. The library is able to acquire loadable symbols simply by loading a shared library (where as the old pluginlib required the knowledge about the names of the named manifests which is stored in the plugin XML).

    • pluginlib will still utilize the export tags and plugin XML files to discover plugins of specific types in the ROS environment.

  3. The new plugins library is designed from the beginning to be thread-safe.

  4. The new plugins package utilizes a stock version of Poco directly from the Ubuntu repos.

Solution Design Details

The implemented solution does not break packages that depend on pluginlib as pluginlib's API remains the same. However the implementation of it does.

API

To make a class exportable as a plugin, one must utilize the REGISTER_CLASS macro defined in "plugin_register_macro.h" within their cpp file. For example, if we have a base class called "Animal" and a derived class "Dog" ("Dog" is the plugin), we mark "Dog" as a plugin within the implementation .cpp file as follows: REGISTER_CLASS(Dog, Animal)

Once a class is registered, if it is contained within a runtime library, it will be seen by the plugin system during load time. Note this is an important improvement over the existing pluginlib system where plugin exports have to be explicitly declared within the plugin XML file in addition to utilizing a registration macro.

The API to plugins offers two ways to load plugins:

  1. a global function interface (plugins.h)
  2. a class-based interface (class_loader.h)

Interface 1 - Global Functions

The first approach to loading plugins is a simple set of functions declared within plugins.h: 

template <typename Base> boost::shared_ptr<Base> createInstance(const std::string& derived_class_name);
template <typename Base> std::vector<std::string> getAvailableClasses()
bool isLibraryLoaded(const std::string& library_path);
void loadLibrary(const std::string& library_path);
void unloadLibrary(const std::string& library_path);
void unloadAllLibraries();

Users use the interface as follows:

  1. Include "plugins.h" within their source code.

  2. Open a library via loadLibrary() passing the name of a runtime library

  3. Ask the plugin system which plugins are available for a given base class via getAvailableClasses()

  4. Users create instances to plugins via the createInstance() function call

  5. Once done, the users close libraries they opened using unloadLibrary() or unloadAllLibraries(). Note if a library is in use by a ClassLoader (next section), then the library won't really be unloaded for real until the last user of the library is finished.

Interface 2 - Class Loader

The second interface is similar, but the functions described above belong to a class known as ClassLoader defined with class_loader.h. Though the global functions in interface 1 are easy to use, sometimes it is desirable to open libraries only within a limited scope. Libraries loaded via a ClassLoader can only use the plugins defined within those libraries if opened through that respective ClassLoader's methods. 

class ClassLoader
{
  public:
    template <class Base>
    std::vector<std::string> getAvailableClasses()
    template <class Base>    
    boost::shared_ptr<Base> createInstance(const std::string& derived_class_name)
    bool isLibraryLoaded(const std::string& library_path);
    bool isLibraryLoadedByExternalObject(const std::string& library_path);
    void loadLibrary(const std::string& library_path);
    void unloadLibrary(const std::string& library_path);
};

Users use the ClassLoader as follows:

  1. Include the file "class_loader.h" within your source code

  2. Create an instance of a ClassLoader

  3. Invoke the loadLibrary(), createInstance(), and unloadLibrary() methods in the same way as the global interface

  4. Note the ClassLoader will automatically unload libraries in during destruction. Libraries opened by other class loaders or the global interface will stay in memory until the last user is finished.

Question / concerns / comments

Enter your thoughts on the API and any questions / concerns you have here. Please sign your name. Anything you want to address in the API review should be marked down here before the start of the meeting.

Mirza - I'm fairly happy with how the plugins API came out, though there are some things I'm considering changing/adding.

Bill - I'm really not sure that "plugin exports have to be explicitly declared within the plugin XML file" is a negative. Unless there is some other mechanism, I don't see how the new system works with rospack plugins --attrib=plugin rviz for example. How are plugins in the new system detected?

It is not clear how these changes effect our use cases, but since you are making changes I'd like to propose some ideas.

https://github.com/IHeartRobotics/iheart-ros-pkg/tree/master/ihe_hardware/urdf_compose

manifest.xml 

   1 <package>
   2   ....
   3   <export>
   4     <urdf_compose urdf="urdf/cup_holder.urdf"/>
   5   </export>
   6 </package>

https://github.com/TurtleBot-Mfg/ros-system-tools/blob/master/rosmetalaunch manifest.xml 

   1 <package>
   2   ....
   3   <export>
   4     <launch_manager launch="button_interface.launch"/>
   5   </export>
   6 </package>

For some reason it looks like both of these use cases currently work and we would like to maintain and formalize support for this.

Also, this approach may work well for the appmanager to locate .app files provided by various packages.

Jack - Will nodelets be able to use the new plugins interface?

Jack - How does a plugins host obtain the names of shared libraries it might want to load?

Jack - I don't understand the purpose of the second, Class Loader, interface. What is the use case? What kind of scope is implied?

Vincent Rabaud - I'd like to see some const correctness in there: I am not sure about all cases but the following should, imho, be const: getAvailableClasses(), isLibraryLoaded, isLibraryLoadedByExternalObject. I could see why createInstance and the load/unload modify some internal struct.

Vincent Rabaud - I'd like a typedef for the plugin name: this way, when we switch to C++11 and have UTF8 strings, it will be transparent.

Names seem to be consistent elsewhere. Why different function names here? 

bool ClassLoader::isLibraryLoadedByExternalObject(const std::string& library_path)
{
  return(plugins::plugins_private::isLibraryLoadedByAnybody(library_path));
}

Can we use self instead of me? 

void ClassLoader::unloadAllLibrariesInUseByMe()

It is nice to call the base constructor here too: 

    AbstractMetaObject(const char* name) : name_(name)
    {
    }

and you can inherit privately from boost::noncopyable as well, to avoid having the private constructors, copy , etc. Or better yet, have AbstractMetaObjectBase in herit from boost::noncopyable (privately).

This line should not be in the code: 

   std::cout << "MetaObject being destroyed for class type = " << (this->name()) << std::endl;

and all other lines that use std::cout / cerr.

In plugin_proxy.cpp (which I think it should be renamed to plugin_core.cpp) there are a number of functions that have static members, and each function returns one member. This makes it necessary to write code that does not look very solid (although it might be) where the functions need to be called in a very precise order. Various bells ring for me when I see things like setting the active class loader. I think it would be cleaner if you had a struct like so: 

struct PluginCoreStorage
{
 boost::mutex m;
 BaseToFactoryMapMap instance;
 LibraryVector instance;
....
};

and then one single function that does 

PluginCoreStorage& getStorage(void)
{
  static PluginCoreStorage storage;
  return storage;
}

Given the functionality in meta_obejct.cpp, could you use a map instead of a vector for ClassLoaderVector? This is to avoid O(n) searches. This ownership issue however could be resolved with the shared_ptr business I mentioned above.

What if each loaded library has it's corresponding data in a struct, and there is a shared pointer to that struct in each ClassLoader that needs that lib? The actual unloading happens in the destructor of this struct, so when destroying a class loader, the shared pointer will do the magic (decement ref count and destroy if needed).

Meeting agenda

To be filled out by proposer based on comments gathered during API review period

Conclusion

Package status change mark change manifest)

2024-12-28 15:09