Welcome to raycloudtools Discussions!

[thomas-lowe]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[ShukhratSh]

Hello Thomas, Thank you for sharing this amazing toolset. I have extensive TLS datasets, and each plot contains several gigabytes of data. However, when attempting to run Rayextract, I encountered a memory error issue. I was wondering if there is a way to process the data using RayCloudTools in smaller 10m by 10m tiles? While I can loop over each tile, I'm concerned that this approach might introduce edge effects, as it seems there is no way to incorporate neighboring tiles in rayextract. Any suggestions or guidance would be greatly appreciated. Thank you!

[thomas-lowe]

Hi Shukhrat, if you are talking about rayextract trees then you're in luck. In the scripts folder there is a script called rayextract_trees_large.sh, which splits a large cloud into tiles of user-specified width. Then it runs the rayextract terrain and rayextract trees on each tile and combines the result back into one trees file. Note that you would need treetools<https://github.com/csiro-robotics/treetools> to combine the treefiles into one. This mitigates against edge artifacts by including an overlap between tiles (set at 5 metres in the .sh, it can be adjusted to the maximum tree width), after trees are reconstructed whole trees are cropped to the non-overlapping cell based on their trunk location. This does a good job with edge artifacts. If you just want to do rayextract terrain then you can make a copy of this bash file, and modify it to just extract terrain for each tile. Modelling software like meshlab or blender can combine all the meshes into one. Note that the for loop in this bash script can be changed to a parallel for, if you want to make use of parallelism. Thomas Lowe Senior Experimental Scientist Data61 | CSIRO

…

________________________________ From: Shukhrat Shokirov ***@***.***> Sent: Tuesday, 1 August 2023 2:10 AM To: csiro-robotics/raycloudtools ***@***.***> Cc: Lowe, Tom (Data61, Pullenvale) ***@***.***>; Author ***@***.***> Subject: Re: [csiro-robotics/raycloudtools] Welcome to raycloudtools Discussions! (Discussion #1) Hello Thomas, Thank you for sharing this amazing toolset. I have extensive TLS datasets, and each plot contains several gigabytes of data. However, when attempting to run Rayextract, I encountered a memory error issue. I was wondering if there is a way to process the data using RayCloudTools in smaller 10m by 10m tiles? While I can loop over each tile, I'm concerned that this approach might introduce edge effects, as it seems there is no way to incorporate neighboring tiles in rayextract. Any suggestions or guidance would be greatly appreciated. Thank you! — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKSVQ4EXJFWSWV42CEHM47TXS7KFXANCNFSM4UZ7G66A>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[ShukhratSh]

Hello Thomas,

I greatly appreciate your prompt response. I saw a tool called "raysplit," but I'm struggling to understand its usage. When I applied it to a 1-hectare plot with a 24 GB dataset, I obtained unexpected results. Specifically, I used the following command: raysplit cloud.ply grid 10,10,0 1 (but also tried to change grid to 20m, 50m). Strangely, it generated 4-5 tiles of size 10m by 10m, along with one tile of the same size as the original 24 GB file, but the tile size of 10m by 2m and 700 000 000 points which is unrealistic for this small area of the forest point cloud. But the point cloud clearly represents some trees. Then again it throws a memory error when running rayextract trees on that tile. Could you help me understand why I am observing this behavior? Your insights would be very valuable.
Thanks,
Shukhrat

[thomas-lowe]

Hi Shukhrat, I have not experienced that error before.
The raysplit grid command has to keep many file pointers open at once, that number currently can go as high as 256 (after which it splits in batches with the latest code). That number works on Ubuntu which I'm testing on, but it may be that your operating system isn't coping with that many open file pointers. You can reduce the value in raysplitter.cpp, it is called max_open_files.

If that isn't the problem then you might want to send me the file if it isn't sensitive data, but better to reduce its size first. For example if you run raydecimate cloud.ply 100 rays first, does it still happen? That would be an easier file to look at.

Lastly, I changed raysplitter about 2 weeks ago, so depending on when you last pulled, you might want to try the other version (before, or after the change).

[ShukhratSh]

Hi Thomas, rayextract_trees_large.sh worked with another dataset perfectly. I think the previous issue was related to the data. So please ignore my previous request. Thanks a lot for your help.

[thomas-lowe]

No probs. You can try rayinfo cloud.ply to see if there is any obvious problem. For instance look at the bounding box.
Also, the raysplit cloud.ply grid command tells you how many cells it is attempting to split into, see if it is what you would expect.