Skip to content

benchmark execution issues #654

@jmatejcz

Description

@jmatejcz

Describe the bug
Investigate issues that sometimes occurs when running benchmarks:

  • Model (qwen3:14b for example) sometimes gets stuck on thinking till timeout
  • When running long manipulation benchmark with large models that take a lot of RAM, the python process after couple hours, gets killed because of RAM usage, which suggests that there is some memory leakage.
  • When a large model takes up a lot of VRAM, there is no space for openset models and program thrors CUDA OOM, should we adress that issue?

To Reproduce

  1. Run manipulation benchmark with qwen3:14b, no thinking
  2. Run manipulation benchmark with large model like 32GB on CPU
  3. Run manipulation benchmark with large model like 32GB on GPU

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Platform

  • OS: Ubuntu 22.04
  • ROS 2 Humble

Version
Release number or commit hash.

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingquestionFurther information is requested

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions