Skip to content

feat(doc): add guidance for developers about using autoware evaluator #562

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 12, 2024

Conversation

brkay54
Copy link
Member

@brkay54 brkay54 commented Jun 3, 2024

Description

Related issue:

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The Reviewers must check the checkboxes below before approval.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

@brkay54 brkay54 added type:documentation Creating or refining documentation. component:planning Route planning, decision-making, and navigation. component:simulation Virtual environment setups and simulations. labels Jun 3, 2024
@brkay54 brkay54 requested a review from xmfcx June 3, 2024 06:48
@brkay54 brkay54 self-assigned this Jun 3, 2024
@brkay54 brkay54 marked this pull request as draft June 3, 2024 06:48
@brkay54
Copy link
Member Author

brkay54 commented Jun 3, 2024

@xmfcx I used the Others section for this documentation but I couldn't be sure if it is the correct place or not.

@brkay54 brkay54 force-pushed the add-autoware-evaluator-guide branch from ac6352a to 3063f85 Compare June 25, 2024 09:37
@brkay54 brkay54 added the tag:deploy-docs Mark for deploy-docs action generation. (used-by-ci) label Jun 25, 2024
@brkay54 brkay54 marked this pull request as ready for review June 25, 2024 09:41
@brkay54 brkay54 force-pushed the add-autoware-evaluator-guide branch from da57a7f to 69b2b4b Compare June 25, 2024 10:02
@brkay54 brkay54 changed the title feat(doc): Add guidance for developers about using autoware evaluator feat(doc): add guidance for developers about using autoware evaluator Jun 25, 2024
@brkay54 brkay54 force-pushed the add-autoware-evaluator-guide branch from 69b2b4b to a80ee19 Compare June 25, 2024 10:07
@brkay54
Copy link
Member Author

brkay54 commented Jun 25, 2024

cc @xmfcx @mitsudome-r
Hi,
The document is ready for review, but there are some blockers:

  1. Regression Test Procedures:

    • In the issue created by Fatih, one of the conditions was: Procedures for running regression tests to ensure no disruptions are described.
    • To achieve this, we must download all scenarios from Autoware Evaluator to run them locally. However, during our discussion with Igata-san in the ODD WG, we decided this wouldn't be feasible because map versions in the cloud can vary, and a scenario depends not only on the map but also on its specific version.
    • Alternatively, developers can execute tests in the cloud for their PRs using Autoware Evaluator.
  2. Access to the AWF Project:

    • We discussed with @mitsudome-r -san in previous weeks that the AWF Project will be made public (accessible without login), but this hasn't been implemented yet. Once it is, I may need to update the documentation links. Currently, this process is mentioned as Under Development in the documentation.

Except for these points, the document is ready for review. If there is any missing point or possible improvement, let me know.

@brkay54 brkay54 force-pushed the add-autoware-evaluator-guide branch from a034c5d to 695d322 Compare June 27, 2024 11:15
@brkay54
Copy link
Member Author

brkay54 commented Jun 27, 2024

cc @xmfcx @mitsudome-r
After we discussed with Igata-san how people should be invited to the AWF project, I updated the documentation. Until make the AWF project more open, we decided to ask the new user to join ODD WG once to share his/her interest for now (Because of the AWF project's limited resources).
Currently, this PR is ready for review.

Copy link
Contributor

@ahmeddesokyebrahim ahmeddesokyebrahim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much @brkay54 for the amazing efforts and the very useful documentation.
The documentation is easy to follow and very descriptive.
I have some minor comments that would be better to have a look to them before merging your PR.
Thanks again and let me know if you have any thoughts for my comments.

Berkay Karaman and others added 5 commits July 10, 2024 13:21
Signed-off-by: Berkay Karaman <berkay@leodrive.ai>
Signed-off-by: Berkay Karaman <berkay@leodrive.ai>
Signed-off-by: Berkay Karaman <berkay@leodrive.ai>
Signed-off-by: Berkay Karaman <berkay@leodrive.ai>
@brkay54 brkay54 force-pushed the add-autoware-evaluator-guide branch from 226b781 to 10cffa3 Compare July 11, 2024 00:06
Copy link
Contributor

@xmfcx xmfcx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge, any additional changes can be made in a separate PR.

Thanks for everyone's work.

@xmfcx xmfcx merged commit 040cb32 into main Jul 12, 2024
8 of 9 checks passed
@xmfcx xmfcx deleted the add-autoware-evaluator-guide branch July 12, 2024 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:planning Route planning, decision-making, and navigation. component:simulation Virtual environment setups and simulations. tag:deploy-docs Mark for deploy-docs action generation. (used-by-ci) type:documentation Creating or refining documentation.
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants