Adds an initial StableBaselines3 RL environment as an example #2667
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR adds support for the Reset API to the test fixture. As `TestFixture` is one of the main ways one can get access to the ECM in python when trying to write some scripts for Deep Reinforcement Learning I realized that without `Reset` supported in the `TestFixture` API, end users would have a very hard time using our python APIs (which are actually quite nice). For reference I'm hacking a demo template here: https://github.com/arjo129/gz_deep_rl_experiments/tree/ionic Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
This allows us to reset simulations without having to call into gz-transport making the code more readable from an external API. Depends on #2647 Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
A lot of things are not working. Particularly when `ResetAll` is called, the EnableVelocityChecks does not trigger the phyics system to populate the velocity components. This is a blocker for the current example. Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
19 tasks
Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
|
So the above code should be able to train a RL model even on a potato. Currently I've got the algorithm to successfully balance a cart pole. There are some open issues however that will block this from being merged. Primarily, my main concern is that I've hacked together an API for running the gui client.
|
arjo129
added a commit
that referenced
this pull request
Dec 20, 2024
* Adds support for Reset in test fixture This PR adds support for the Reset API to the test fixture. As `TestFixture` is one of the main ways one can get access to the ECM in python when trying to write some scripts for Deep Reinforcement Learning I realized that without `Reset` supported in the `TestFixture` API, end users would have a very hard time using our python APIs (which are actually quite nice). For reference I'm hacking a demo template here: #2667 --------- Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
arjo129
commented
Feb 5, 2025
Signed-off-by: Ubuntu <arjoc@intrinsic.ai>
Signed-off-by: Ubuntu <arjoc@intrinsic.ai>
…ample' into arjo/examples/rl_example
Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
|
@osrf-jenkins run tests |
|
Can someone please review this? |
iche033
reviewed
Apr 16, 2025
examples/scripts/reinforcement_learning/simple_cart_pole/README.md
Outdated
Show resolved
Hide resolved
examples/scripts/reinforcement_learning/simple_cart_pole/README.md
Outdated
Show resolved
Hide resolved
examples/scripts/reinforcement_learning/simple_cart_pole/cart_pole_env.py
Outdated
Show resolved
Hide resolved
examples/scripts/reinforcement_learning/simple_cart_pole/README.md
Outdated
Show resolved
Hide resolved
examples/scripts/reinforcement_learning/simple_cart_pole/README.md
Outdated
Show resolved
Hide resolved
examples/scripts/reinforcement_learning/simple_cart_pole/cart_pole.sdf
Outdated
Show resolved
Hide resolved
Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
github-project-automation
bot
moved this from In progress
to In review
in Core development
Signed-off-by: Arjo Chakravarty <arjoc@intrinsic.ai>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR provides an incredibly basic example of how to use gazebo with StableBaselines3 for RL. This example is that of the classic cartpole which is commonly used as a "getting started" task in reinforcement learning. The python script trains a simple model using python to balance a cart pole. We use the gui to visualize it.