Contents
Writing Qualification Tests
Developers can use the qualification system to write a test for almost any robot or robot sub-assembly. Developers can also write configuration scripts, to allow easy reconfiguration of a part (see Component Configuration for details).
Outline Test Structure
To write a qualification test, start by making a folder in the qualification/tests directory, with the name of a component. In that folder, make a file called 'test.xml'. In the case of the simple_example test, it looks like this:
<test check-assembly="true" id="my-test" > <instructions>instructions.html</instructions> <pre_startup>pre_startup_script.launch</pre_startup> <startup>startup.launch</startup> <subtest>test1.launch</subtest> <subtest>test2.launch</subtest> <subtest>test3.launch</subtest> <shutdown>shutdown.launch</shutdown> </test>
Each launch file is given as the basename of the file in the directory of test.xml. The qualification manager will look for the specified files using the path of your directory. For example, "../caster_test/caster.launch" would load qualification/tests/caster_test/caster.launch.
Parts of Qualification Tests
Our test contains five parts:
- Instructions, given as an HTML file
- Prestartup scripts: anything we need to do before we start our tests
- Startup scripts, which generally launch drivers needed such as pr2_etherCAT
- Subtests, which run any type of test on the device and send the analysis to the qualification system
- Shutdown scripts, which stop any processes we started (mainly the power board)
Except for the instructions file, these scripts and subtest are all roslaunch files.
Instructions
The qualification GUI will show the user the instructions first. All instructions must contain the following items:
- Picture of component
- Identifying information of the component, to make sure that the user if plugging in the correct one
- Connecting instructions (power, etherCAT cables)
- Test details, what to look for, etc
The instructions should have the test name as an H2 heading, with images roughly 640 by 480. A sample instruction file is below:
<html><body> <H2>Instructions File</H2> <H3>Instructions Subheader</H3> <img src="my_image.png" width="640" height="480"> <H3>Verifying Component Type</H3> <p>Put verification information here...</p> <H3>Running Tests</H3> <p>Put stuff about running tests here.</p> </body></html>
Try to follow the above example as a "style guide" for instructions.
Prestartup Scripts
Prestartup scripts generally do things like enable the power supply, and program or configure MCB's. These steps are an important part of checking mechanical components. The qualifcation GUI will launch the prestartup scripts, in order specified in the test.xml file after the instructions.
When launching the prestartup scripts, the qualification system will advertise a pr2_self_test_msgs/ScriptDone service on the topic prestartup_done. All prestartup scripts must call this service when finished. The prestartup script should fill the ScriptDone/result field as follows:
- 0/OK: Prestartup completed successfully
- 1/FAIL: Prestartup failed (ex: MCB's failed to program)
- 2/ERROR: Prestartup recorded an error (ex: Power node wouldn't launch correctly)
Failure and error terminations will stop the test (call shutdown script and stop all launched scripts).
Prestartup failures will be logged in the inventory system for each part, since they reflect the part's status, but errors are not. Do not let your prestartup script return an error code unless you are sure that it is not due to a bad component. For example, a power node failure is an error, but a bad MCB is a failure.
Prestartup scripts do not run tests. Generally, they are done before tests to prepare components for qualification by turning on power and configuring MCB's.
Startup Scripts
Startup launch files are called before subtests and after prestartup scripts. Startup scripts activate drivers such as hokuyo nodes and pr2_etherCAT. The drivers called in the startup script will remain active during all qualification subtests and are shutdown after a test is finished.
<launch>
<!-- robot descriptions param -->
<param name="robot_description" command="$(find xacro)/xacro.py '$(find qualifica\
tion)/tests/head_test/head.xml'" />
<!-- Loads realtime machine and PR2_etherCAT -->
<include file="$(find qualification)/tests/init.machine" />
<include file="$(find qualification)/tests/pr2_etherCAT.launch" />
<!-- Head Calibration -->
<node pkg="mechanism_bringup" type="calibrate.py"
args="$(find qualification)/tests/head_test/head_cal.xml" />
</launch>Use the files qualification/tests/init.machine and qualification/tests/pr2_etherCAT.launch for setting up test machines and using pr2_etherCAT respectively.
Qualification Machines
The file init.machine file loads two machines, localhost and a test_host. The test host is determined by the environment variable ROS_TEST_HOST, which is set by the qualification system.
Subtests
For more details on writing qualification subtests, see the Writing Subtests page.
Subtests form the backbone of the qualification system. Each subtest is started by a launch file, and calls the 'test_result' service of type invalid message type for SrvLink(srv/type) with its results. Since the qualification GUI does not do any analysis itself, it is up to the subtests to analyze results and output graphs and data.
Shutdown Scripts
Typically shutdown scripts disable the power board after a qualification test. Like prestartup scripts, shutdown scripts call the shutdown_done service which takes a pr2_self_test_msgs/ScriptDone service. Any ScriptDone.result other than RESULT_OK will send an error message to the user. There can be only one shutdown script.
Adding Test to Qual System
Once a test is written, with a complete test.xml file and associated launch scripts, it must be added to the list of qualification tests. The file qualification/tests/tests.xml contains all the tests. Add new tests in the following format:
<test serial="1234567"
file="my_test/test.xml" descrip="My Test" />- "serial" allows the GUI to look up the test by the first seven digits of the serial number. Do not add more digits.
- "file" gives the path to the qualification test.xml file, after qualification/tests
- "descrip" is a short description, max 80 characters, to allow a user to distinguish two tests for the same serial
Verifying Your Test
After your test file is loaded into the tests.xml file, you can call "make test" in the qualification package to verify this file. This will make sure your XML file parses correctly and that your launch files exist.
To parse your launch files as part of a build test use the rosbuild macro rosbuild_add_roslaunch_check() to the CMakeLists.txt file in qualification.
Debugging Your Test
It can be useful to debug your test by running it without the entire qualification system. You can roslaunch it manually, and use a viewer to view the results.
rosrun qualification display.py
This will display the results of the qualification test is the same format as the normal qualification system.
Debug Mode
To run the qualification system in a "debug" mode, use the argument "--debug" to run the qualification GUI.
rosrun qualification qualify --debug
Do not use debug mode for production testing.
Stress Testing
You can stress test you qualification test by running it "Continuous" mode. While in debug mode, select "Continuous Testing" in the debug menu. Your test will run continuously, and generate a log of success and fail runs.