Skip to content

Commit 18481c5

Browse files
I've created a comprehensive README.md for the project.
This commit introduces the initial `README.md` file. The README includes the following sections: - Project Title and Overview - Features: Highlighting real-time pose detection, tracking, hardware acceleration, and example applications. - Technologies Used: Listing C++, Rust, Bazel, TensorRT, CVCUDA, and key Rust crates. - Prerequisites: Detailing necessary software like Bazel, NVIDIA libraries (TensorRT, CVCUDA, CUDA Toolkit), Rust, and a C++ compiler. - Building the Project: Instructions on how to build using Bazel and notes on configuration. - Running the Examples: Guidance on executing image and video demos, emphasizing the need for a TensorRT plan file. - Project Structure: A brief overview of key directories and their contents. - License: A placeholder indicating the license is not yet specified. This README aims to provide you and other developers with essential information to understand, build, and run the project.
1 parent 0ac3cec commit 18481c5

File tree

1 file changed

+127
-0
lines changed

1 file changed

+127
-0
lines changed

README.md

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# Real-Time Pose Detection and Tracking
2+
3+
This project implements a system for real-time human pose detection and tracking in images and videos. It leverages hardware acceleration for efficient processing.
4+
5+
## Features
6+
7+
* Real-time pose detection (bounding boxes and keypoints).
8+
* Multi-object tracking using an adapted ByteTracker algorithm.
9+
* Hardware acceleration using NVIDIA TensorRT for inference and CVCUDA for vision pre/post-processing.
10+
* Cross-language implementation with C++ for performance-critical detection and Rust for tracking and application logic.
11+
* Example applications for processing both static images and video streams.
12+
13+
## Technologies Used
14+
15+
* **Programming Languages:** C++, Rust
16+
* **Build System:** Bazel
17+
* **Core Libraries & Frameworks:**
18+
* NVIDIA TensorRT: For high-performance deep learning inference.
19+
* NVIDIA CVCUDA: For CUDA-accelerated computer vision tasks.
20+
* OpenCV (indirectly via CVCUDA and for image/video handling concepts)
21+
* `cxx.rs`: For seamless C++/Rust interoperability.
22+
* **Key Rust Crates:**
23+
* `image`: For image loading, manipulation, and saving.
24+
* `video-rs`: For video decoding.
25+
* `minifb`: For simple GUI window creation to display video output.
26+
* `nalgebra`: For numerical calculations, particularly in the tracking component.
27+
28+
## Prerequisites
29+
30+
Before building and running this project, please ensure you have the following installed:
31+
32+
* **Bazel:** The build system used for this project. (Follow official Bazel installation instructions for your OS).
33+
* **NVIDIA TensorRT:**
34+
* The detection engine relies on TensorRT for model inference.
35+
* The project is configured to look for TensorRT in `/usr` (headers in `/usr/include/...` and libraries in `/usr/lib/...`). If your installation path differs, you may need to adjust the paths in `repositories/repositories.bzl` and potentially `detector/BUILD.bazel`.
36+
* **NVIDIA CVCUDA:**
37+
* Used for CUDA-accelerated computer vision operations.
38+
* Similar to TensorRT, it's expected to be in `/usr`. Adjust paths in `repositories/repositories.bzl` if needed.
39+
* **NVIDIA CUDA Toolkit:** Required by TensorRT and CVCUDA. Ensure your driver and toolkit versions are compatible.
40+
* **Rust Toolchain:** Install Rust and Cargo using `rustup`.
41+
* **C++ Compiler:** A modern C++ compiler (e.g., g++) compatible with your CUDA Toolkit and TensorRT versions.
42+
* **pkg-config:** Often used by build systems to find libraries.
43+
44+
**Note:** Specific versions of CUDA, TensorRT, and CVCUDA might be implicitly required for compatibility. While not explicitly versioned in the build files, it's recommended to use recent versions and ensure they are mutually compatible.
45+
46+
## Building the Project
47+
48+
Once all prerequisites are met, you can build the project using Bazel:
49+
50+
```bash
51+
bazel build //...
52+
```
53+
54+
This command will build all targets, including the detector library, tracker library, and the examples.
55+
56+
**Notes on Configuration:**
57+
58+
* If your NVIDIA libraries (TensorRT, CVCUDA, CUDA) are not installed in standard system paths (e.g., `/usr/local` instead of `/usr`), you might need to:
59+
* Adjust the `path` attribute in the `new_local_repository` rules within `repositories/repositories.bzl`.
60+
* Ensure your `LD_LIBRARY_PATH` includes the paths to the shared libraries.
61+
* Potentially modify include paths or library paths in the `cc_library` rules within `detector/BUILD.bazel` if the Bazel auto-configuration doesn't pick them up correctly.
62+
* The project uses `cc_library` rules which often rely on Bazel's C++ toolchain auto-configuration. If you have multiple compilers or specific toolchain requirements, you might need to configure your Bazel C++ toolchain (e.g., via `.bazelrc` or command-line flags).
63+
64+
## Running the Examples
65+
66+
The project includes examples to demonstrate pose detection on images and videos.
67+
68+
**Important:** The examples require a pre-built TensorRT engine plan file for the pose detection model. You will need to obtain or generate this file yourself. The specific model architecture and input/output tensor configurations will depend on the plan file you use. The example code expects the model to output bounding boxes and keypoints in a compatible format.
69+
70+
### Image Demo
71+
72+
The image demo processes a single image, performs pose detection, visualizes the results (bounding boxes and keypoints), and saves the output as `output.png`.
73+
74+
**Command:**
75+
76+
```bash
77+
bazel run //examples:image_demo -- <path_to_tensorrt_plan_file> <path_to_your_image>
78+
```
79+
80+
Replace `<path_to_tensorrt_plan_file>` with the actual path to your TensorRT model plan and `<path_to_your_image>` with the path to the image you want to process.
81+
82+
Example:
83+
```bash
84+
bazel run //examples:image_demo -- /path/to/model.plan /path/to/my_image.jpg
85+
```
86+
87+
### Video Demo
88+
89+
The video demo processes a video file, performs real-time pose detection and tracking, and displays the annotated video in a window.
90+
91+
**Command:**
92+
93+
```bash
94+
bazel run //examples:video_demo -- <path_to_tensorrt_plan_file> <path_to_your_video>
95+
```
96+
97+
Replace `<path_to_tensorrt_plan_file>` with the path to your TensorRT model plan and `<path_to_your_video>` with the path to the video file.
98+
99+
Example:
100+
```bash
101+
bazel run //examples:video_demo -- /path/to/model.plan /path/to/my_video.mp4
102+
```
103+
104+
Press 'q' in the window to quit the video demo.
105+
106+
## Project Structure
107+
108+
A brief overview of the key directories in this project:
109+
110+
* `detector/`: Contains the core pose detection logic.
111+
* `detector/src/engine.cpp, .hpp`: C++ code for the TensorRT inference engine.
112+
* `detector/src/rtmo.cpp, .hpp`: C++ implementation details for the RTMO model processing.
113+
* `detector/src/rtmo.rs`: Rust bindings for the C++ RTMO detector functions using `cxx.rs`.
114+
* `detector/src/lib.rs`: Rust library module for the detector.
115+
* `tracker/`: Contains the object tracking logic.
116+
* `tracker/src/byte_tracker.rs`: Implementation of the ByteTracker algorithm in Rust.
117+
* `tracker/src/kalman_filter.rs`, `lapjv.rs`, etc.: Supporting components for the tracker.
118+
* `examples/`: Contains runnable demo applications.
119+
* `examples/src/image_demo.rs`: Demonstrates pose detection on a static image.
120+
* `examples/src/video_demo.rs`: Demonstrates pose detection and tracking on a video stream.
121+
* `repositories/`: Contains Bazel definitions for managing external dependencies like TensorRT and CVCUDA.
122+
* `BUILD.bazel`: Root Bazel build file.
123+
* `MODULE.bazel`: Bazel module file.
124+
125+
## License
126+
127+
The license for this project is not yet specified. Please assume it is proprietary unless a LICENSE file is added to the repository.

0 commit comments

Comments
 (0)