Skip to content

Adding New Tool

Jump to bottom
ManosMpampis edited this page Dec 15, 2021 · 22 revisions

New tools added to the OpenDR GitHub repo should provide the following files:

  • source code to apply the tool for training and inference
  • documentation of the tools including:
    • a short description of the method itself. It is not necessary to give all the details, but it is good to link to the related paper or reference if any.
    • details description of the API needed by the user to apply the tool to her scenario
    • step-by-step description about how to run the example
  • unit tests
  • example (preferably using Webots) of how to apply the tool to a scenario
  • at least one pretrained model ready for inference

The following instructions will provide paths using the following wildcards:

  • <type>: tool type, e.g., control, perception, planning
  • <task_name>: task addressed by the tool, for example "pose_estimation"
  • <algorithm_name>: name of the algorithms used to solve the task, for example, "lightweight_open_pose"

Source Code

New Tool Public API

New tools should follow the OpenDR code guidelines and ensure a common interface:

Files

The source code of the tool has to be stored in the /src/<type>/<task_name>/<algorithm_name> folder. The expected files are:

  1. README.md: containing a short description of the task and algorithms (1-2 lines), the list of the sources of third-party code (if any), and the modifications to the original code
  2. <algorithm_name>_learner.py: main class providing the entry point to the functionality
  3. algorithm/: folder containing the algorithm implementation
  4. dependencies.ini: list of dependencies needed to run the tool. See Dependencies section for more details about the format.

CI tests

The files in /src are tested by the CI (Continuous Integration) system to check the compatibility with the OpenDR guidelines. Tests on original files taken from third-party that are copied and pasted without modifications can be skipped. In particular, the OpenDR license test should be skipped because it is now allowed to change license and copyrights of third-party code. It is possible to skip tests for a folder by adding the path to the skippedDirectoryPaths variable in the test itself.

Third-party Code and License

If not modified and available on a public GitHub repo, third-party code should be linked (as a "submodule") instead of copied in the OpenDR repository. If not available on a GitHub repo or some modifications are applied:

  • commit the algorithms sources in the OpenDR repo
  • specify the link to the original code in the tool README.md file, the author, and list all the modifications

OpenDR license doesn't have to be applied on third-party code!

OpenDR Apache License version 2.0 is a permissive license: work can be redistributed even for commercial use. So, third-party code integrated into the OpenDR toolkit should also be permissive. Non-commercial licensed code cannot be integrated. Common compatible licenses are:

Dependencies

Dependencies will be automatically installed using a script and the Makefile system. For each tool, a dependencies.ini file should be created listing all the dependencies required to run the method. This file should be written using the INI format. Three different sections are supported:

  • runtime: includes dependencies required to execute the tool
  • compilation: includes dependencies required to compile the C/C++ code. Python tools require runtime dependencies only.
  • device: includes the opendr_device key

In the runtime and compilation section, the following keys can be used:

  • python: listing the python modules. The value syntax should be the standard pip requirements file format
  • python-dependencies: listing additional python packages that will be installed before the python dependencies to resolve prerequisites
  • linux: listing all the system packages required on Linux systems that will be installed using the system package manager
  • macOS: not implemented yet
  • windows: not implemented yet

In the device section, the following keys can be used:

  • opendr_device: with valid options cpu and gpu. If the listed dependencies strictly require a GPU, the key has to be set to gpu. Otherwise, i.e., if a CPU suffices, the key should be set to cpu or can be omitted.

Here is a sample dependencies.ini file.

[runtime]
python=torch==1.7.1
       opencv-python
linux=unzip wget

[compilation]
linux=g++

[device]
opendr_device=cpu

Documentation

All the documentation needed by the user should be added in the /docs/reference folder, i.e. in the Reference Manual of the OpenDR toolkit. A single file named after the method should be added and should contain these informations:

  1. Description of the method
  2. Specifications of the API
  3. Instructions to run the example

Adding documentation directly in the code is recommended to make the code more readable and understandable. But this is not considered part of the OpenDR API documentation needed to use the toolkit.

When adding a new documentation file, please also make sure to add it to the documentation index: https://github.com/tasostefas/opendr_internal/blob/master/docs/reference/index.md

Description of the method

This includes a description of the method: what is the purpose of the method, which are the techniques used, an overview of the performance, etc. This is not intended to be a detailed description of how the method is implemented or all the research behind it, but it should give enough information to the user to understand if this method could be applied to his scenario. It is good to link to the related paper or similar so that interested users can look into the details.

Specifications of the API

All the public methods of the tool have to be listed and described. This includes the description of the function arguments, returned values, and functionality.

Instructions to run the examples

The last section of the documentation page should include the step-by-step instructions to run the provided example. A short description of the example and the expected result should able be included.

Unit tests

Unit tests are run by the CI system regularly (e.g., every night) or upon request from a GitHub Pull Request. For each tool, we should provide some tests to check the main tool functionalities and check for regression issues. The test file should be added at /test/sources/tools/<type>/<task_name>/<algorithm_name>. For Python implementation, the Python unittest framework is used.

If you add a new folder for testing your tool, then make sure that it is included in the test package list at: https://github.com/tasostefas/opendr_internal/blob/master/.github/workflows/tests_sources.yml#L54-L66

If the test needed specific dependencies besides the tool dependencies, they should be added in /tests/sources/requirements.txt. This file uses the standard pip requirements file format.

Example

The example files have to be added to a dedicated folder in the /projects folder. For example: /projects/<type>/<task_name>/algorithm_name The example should be as simple as possible but still providing a meaningful application where the tool's potentialities are demonstrated. It should preferably be a Webots project.

Keeping in mind that OpenDR is a set of many different tools it is important to carefully choose the dependencies and try when possible and meaningful to reuse dependencies already included instead of adding new ones. New dependencies needed to run this specific example and that are not generically needed to run the tool, should be listed in the /projects/<type>/<task_name>/<algorithm_name>/dependencies.ini file. More information about listing dependencies is specified in the Dependencies section.

The example should be designed in such a way that it is as simple and fast to execute as possible but still providing a meaningful application example.

PR checklist

Please check all these points before contributing a tool:

  1. The main functionality is provided in a Learner-based class.
  2. infer(), train() and eval() work correctly with OpenDR data types (e.g., Dataset, Image, Target, etc.).
  3. save() and load() functions are provided.
  4. Any binary models or data are removed from the contributed files. Binary models and/or data should go to the FTP server.
  5. There is no duplicate functionality in the code that is provided elsewhere (e.g. pip package, other repo that can be linked, etc.). If so, this should not be re-included in the repository.
  6. There is no leftover functionality that is not going to be used in the contributed code. If so, this should be removed to make the code easier to maintain.
  7. Appropriate tests to ensure that the code runs correctly (at least one interface check that checks that model works correctly with OpenDR datatypes and at least one functionality check with a pre-trained model).
  8. A continuous integration test for the tool has been added to .github/workflows/tests_sources.yml.
  9. Documentation has been included.
  10. At least one pre-trained model is provided.
  11. A meaningful name has been used for the PR.
  12. Make sure you update the documentation index (please also refer to https://github.com/tasostefas/opendr_internal/blob/master/docs/reference/index.md)
  13. Make sure that an appropriate __init__.py file has been created to expose the public interface of your implementation.

Nice to have (not mandatory at the moment):

  1. C-based Inference Interface
  2. DL model optimization targeting specific platforms based on a third-party tool (e.g., ONNX, TensorRT, etc.)
Clone this wiki locally