This is an additional moudle for LiDAR SLAM, employed for individual tree segmentation and tree atrributie calculation (Position, DBH, and Stem Curve), named TreeSemantic.
Two modes included in this repository.
-
The first is processing LiDAR single frame faster than 10 Hz. This mode segmented points to indivadul trees, and get their position and bounding-box.
-
The second is processing the key frame from any SLAM system, and calculating servel tree traits (Position, DBH, and Stem curve). After that, the usful products, such as DEM, marteloscopes, and tree traits table, are outputed to file. This mode is running at 1 Hz. I have tested with servel popular SLAM systems, including LIO-SAM, Fast-LIO2, Faster-LIO, and Point-LIO.
The mission on single frame mode is indvidual tree segmentation. As shown in the video (click followin figure), different trees are segmented and uniqully colored in real-time. The ground points are colored by brown. The total informations, i.e., total number of trees, area of each scan, are displayed on top-left. The bounding-box is simultaneously shown with extracted tree. Additionally, ID and height with coarse precision are also shown along with the bounding-box.
The mission on key frames mode includes the tree segmentation, attribute extraction (DBH and stem curve), and products generation (DEM, Marteloscopes).
As shown in the video (click followin figure), top is terminals to launch TreeSemantic, Fast-LIO2, and play ROS bag data. Left shows online results of TreeSemantic processing, right shows the results of Fast-LIO2. In left, different trees have unique colors and the ground points are colored in brown. The white cube represents bounding-box, the black circle represents DBH, white points discribe the stem curve. Additionally, total information, such as tree's number, scaning area, are displayed in the top left. The DBH, height, and tree position are displayed along with bounding-box.
The ground points are filtered in the real-time. After this module ends, the ground points are saved to file and employed to generate Digital Elevation Models (DEMs). It is usful for slope, aspect, and visibility analysis.
Ground points
DEM
Tree points also saved to a file and segmented into individual instants. Here is the example of extracted ground and individual trees. The ground points are colored in yellow, tree points are colored randomly.
The stem curve points are also calculated. In this study, multiple profiles along with the stem are selected and then fitted as a circle. The center points of circle are regarded the stem curve.
As show cases, and use for field visits, a figure of representing tree's characteristic in forest stand are intantly generated after program ends, named Marteloscopes.
All tree characteristics descripted above, such tree positions, DBH, stem curve, are saved into a CSV file.
Hence, whole procedure of forest inventory is performed at real time, the raw data and production are obtained at the same time.
πππ : If you are interested to tested them with our module, here is a manual for quick start.
The code have been tested on Ubuntu 18.04. The third-party library are listed as follows:
PCL >=1.8.1
Eigen >=3.3.4
OpenCV >=3.3.1
CSF (Cloth Simulation Filter)
Patchworkpp
small_gicp
Boost >=1.7.5 # Needed by CGAL library
CGAL 5.6
If you use ubuntu 18.04, You can install pcl using a simple way.
sudo apt update
sudo apt-get install libpcl-devOr you can compile through the source code: https://github.com/PointCloudLibrary/pcl
Similarly, you can install the eigen library through apt way.
sudo apt update
sudo apt-get install libeigen3-devOr you can just download and extract Eigen's source code: https://eigen.tuxfamily.org/index.php?title=Main_Page
You can install the OpenCV library through apt way.
sudo apt update
sudo apt install libopencv-dev python3-opencv
# check opencv version
pkg-config --modversion opencvCSF (Cloth Simulation Filter) is an algorithm for LiDAR ground filtering. It usually employed to the Airborne Lidar data, but also shown good performance on the Terrastrial-, Mobile-, and even Spaceborne- Lidar. In this program, CSF is ultilized to re-fine the ground points in the final phase.
You can follow the instruction of CSF.
mkdir build #or other name
cd build
cmake ..
make
sudo make installRelated papers: W. Zhang, J. Qi*, P. Wan, H. Wang, D. Xie, X. Wang, and G. Yan, βAn Easy-to-Use Airborne LiDAR Data Filtering Method Based on Cloth Simulation,β Remote Sens., vol. 8, no. 6, p. 501, 2016.
Patchwork++, an extension of Patchwork, is a fast, robust, and self-adaptive ground segmentation algorithm on 3D point cloud.
In this program, Patchwork++ is ultilized to identify the ground points in real-time. Although its precision is low, its high efficiency is needed for us. The influence of False Negtative (FN) and False Positive (FP) ground points can be ingnored in this project.
You can install Patchwork++ through following commands.
# download patchwork++
git clone https://github.com/url-kaist/patchwork-plusplus.git
# in patchwork-plusplus directory
cd cpp && mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j 16
# install to your system
sudo make instalOr you can refer this Link: https://github.com/url-kaist/patchwork-plusplus/tree/master/cpp The Python version and ROS Node based on ROS1 and ROS2 also can be tested, as decripted in link: https://github.com/url-kaist/patchwork-plusplus.
small_gicp is a header-only C++ library providing efficient and parallelized algorithms for fine point cloud registration (ICP, Point-to-Plane ICP, GICP, VGICP, etc.). The C++ version of small_gicp can be installed by following commands.
sudo apt-get install libeigen3-dev libomp-dev
cd small_gicp
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release && make -j
sudo make installIn this program, small_gicp is employed to re-fined the key-frame pose calculated from SLAM algorithm.
Teaser++ also is a fast and certifiable 3D registration algorithm and can be used to re-fine the key-frame pose. However, Teaser++ is often employed to transform the point cloud at object-level, and not adopted in this project. If you are interested to Teaser++, you can install this alogrithm through following commands:
sudo apt install cmake libeigen3-dev libboost-all-dev
git clone https://github.com/MIT-SPARK/TEASER-plusplus.git
cd TEASER-plusplus && mkdir build && cd build
cmake .. && make
sudo make installThe reference link is https://github.com/MIT-SPARK/TEASER-plusplus.
Boost is a foundamental library for various applications. Unfortunatly, different projects commonly need different boost versions. In this program, the CGAL 5.6 need Boost β₯1.75.
# download
wget https://boostorg.jfrog.io/artifactory/main/release/1.75.0/source/boost_1_75_0.tar.gz
tar -xzf boost_1_75_0.tar.gz
cd boost_1_75_0
# install depend modules
sudo apt update
sudo apt install build-essential g++ python3-dev libicu-dev libbz2-dev zlib1g-dev
# define local path
./bootstrap.sh --prefix=/usr/local/boost_1_75
# install boost 1.75
sudo ./b2 --prefix=/usr/local/boost_1_75 install
# verify boost
ls /usr/local/boost_1_75
# it should output "include lib"
# check the version of boost
cat /usr/local/boost_1_75/include/boost/version.hpp | grep "BOOST_LIB_VERSION"
# it should output // BOOST_LIB_VERSION must be defined to be the same as BOOST_VERSION
#define BOOST_LIB_VERSION "1_75"After that, the Boost 1.75 has been installed to /usr/local/boost_1_75. Then, Boost's path need to be pre-defined in our CMakeLists.txt, such as the BOOST_ROOT, BOOST_INCLUDEDIR, and BOOST_LIBRARYDIR.
set(BOOST_ROOT "/usr/local/boost_1_75")
set(BOOST_INCLUDEDIR "${BOOST_ROOT}/include")
set(BOOST_LIBRARYDIR "${BOOST_ROOT}/lib")
find_package(Boost 1.75.0 REQUIRED)
CGAL is an open source software project that provides easy access to efficient and reliable geometric algorithms in the form of a C++ library. It can be installed through following commands. You can use these commoands, if your operation system is later than Ubuntu 20.04.
sudo apt update
sudo apt-get install libcgal-devBut, CGAL 4.11 is corresponding to Ubuntu 18.04 system, which is too old for this program. You can install CGAL 5.6 through source code. Download the release version, then compile and install to the "/usr/local/include".
cd CGAL-5.6
mkdir build && cd build
cmake ..
make installhttps://doc.cgal.org/5.6.2/Manual/usage.html#section_headeronly
Notic: The Boost library in ubuntu 18.04 is 1.65.1 version, but CGAL 5.6 need Boost later thanγ1.75. So, you should install another Boost in different location. It shoud be mentionded that you shouldn't remove the original version. Another programs still need Boost 1.65.1.
Similarly to our program, the local path of boost package should set in CMakeLists.txt.
set(BOOST_ROOT "/usr/local/boost_1_75")
set(Boost_INCLUDE_DIR "${BOOST_ROOT}/include")
set(Boost_LIBRARY_DIR "${BOOST_ROOT}/lib")
# set local path before find Boost package
find_package(Boost REQUIRED)Clone the repository and catkin_make:
cd ~ && mkdir TreeSemantic_ws && cd TreeSemantic_ws
mkdir src && cd src
git clone https://github.com/xchwang1998/tree_semantic.git
cd ..
catkin_make
source devel/setup.bashTreeSemantic has two modes, the single-frame segmentation and the key-frame process, include segmentation, attribute calculation, and outputing products.
If you only want run the single-frame processing module, you can launch the run_single_frame.launch file.
roslaunch tree_semantic_pkg run_single_frame.launchOtherwise, if you want run whole process to extract the tree attributes in real-time and output the produtions instantlly. You can launch run.launch file.
roslaunch tree_semantic_pkg run.launchThen, you also need launch a SLAM algorithm to publish key_frame_info. Such as the popular SLAM algorithms:
# lio-sam
roslaunch lio_sam run.launch
# fast-lio 2
roslaunch fast_lio mapping_ouster128.launch
# faster-lio
roslaunch faster_lio mapping_ouster128.launch
# point-lio
roslaunch point_lio mapping_ouster128.launchThe structure of key_frame_info is shown as follow:
# Cloud Info
Header header
# cloud messages
sensor_msgs/PointCloud2 key_frame_cloud_ori
sensor_msgs/PointCloud2 key_frame_cloud_transed
sensor_msgs/PointCloud2 key_frame_poseskey_frame_cloud_ori is the original point cloud, key_frame_cloud_transed is the point cloud transed into world coordinate of SLAM, key_frame_poses is all poses of key frames.
Above popular SLAM algorithms should be revised to publish key_frame_info message. Here are four revised repositories for you.
LIO-SAM: https://github.com/xchwang1998/LIO-SAM-Forest-Inventory
Fast-LIO2: https://github.com/xchwang1998/FastLIO_Forest_Inventory
Faster-LIO: https://github.com/xchwang1998/Faster_LIO_Forest_Inventory
Point-LIO: https://github.com/xchwang1998/PointLIO_Forest_Inventory
If you are interested to the detail of revise part, you can refer this CSDN blog.
Notice: Since TreeSemantic module and SLAM algorithm may confilict in sometimes, the computation source can be distributed before launch them. The launch commands are subsquentlly revised as:
# launch LIO-SAM, use core 0,1,2,3
taskset -c 0-3 roslaunch lio_sam run.launch
# launch TreeSemantic, use core 4,5,6,7
taskset -c 4-7 roslaunch tree_semantic_pkg run.launch-
We used the hand-held personal laser scanning device to obtain data from Wuhan University, China.
-
We test TreeSemantic on the public dataset, TreeScope.
To evoluate the results of extracted tree attributes, such as DBH, height, and stem curve, an open-source software is employed. This software is developed by MSpace-WHU, named ParameterEvaluation. You can visit this project for more detail.
There are some issues should be noted.
The OusterPointXYZIRT data format is designed for Ouster point cloud. ring attribute format is uint8_t in order version of Ouster_ROS driver, and are revised to uint16_t in new version, such as TreeScope. Hence, we should use correct format for different rosbag file.
// use
uint16_t ring;
// or use
uint8_t ring;If you use this repository, please cite our paper.
If you use ParameterEvaluation software, please cite this DOI: https://zenodo.org/records/15245849
If you have any questions, please do not hesitate to contact me: xchwang@whu.edu.cn
- Support more SLAM algorithms
- Support more tree attributes calculation
- Expand vertical FoV of laser scanning, to obtain the canopy information of forest
- Combined with deep learning algorithm to recongnize tree attributes in real-time