Python + Open3D: fully automatic point cloud registration?

[LocoMaxim]

I'm building a pipeline to align and merge multiple 3D scans (.PLY files) captured from different angles.
I'm using Open3D and ICP, but I want to automate everything — no manual point picking. Has anyone done multi-cloud registration (3+ scans) fully hands-free?
Any libraries, sample scripts, or tips appreciated!

[timohl]

The Multiway registration tutorial worked for me, but I did some modifications:

• I preprocessed my clouds rotating them to an initial alignment. Since I am using a turntable with 24 scan directions, I have roughly the same angles and am able to pre-align the clouds. If you do not have any initial alignment you could look into Global Registration (since ICP might fall into wrong local minima).
• I also have to remove background (since it is not turning with the turntable) since I am labeling for semantic segmentation anyway, I start with just labeling background and removing it before fine-registration with ICP. If you do not use a turntable, you can probably skip this step. This is the only manual step for me (which can hopefully be replaced or aided by a machine learning model later)
• For pairwise-registration I used the new tensor-based Multi-Scale-ICP with TransformationEstimationPointToPlane. Make sure to call the estimate normals function on your clouds if you happen to not have normals in your .ply
• For some scans, especially when the rotation angles are too large and the overlap between consecutive scans is too low, I would observe severe misalignments (causing a planar surface to show as multiple layers in the combined cloud). To fix this, I added a final step after the Multiway-Registration tutorial iteratively integrating scans while recomputing registration on the integrated cloud as target. Here an outline how this algorithm looks like:

# Previous transforms come from multi-way registration example
integrated_cloud = scans[0]
new_transforms = [previous_transforms[0]]
for scan, previous_transform in zip(scans[1:], previous_transforms[1:]):
    new_transform = my_icp(scan, integrated_cloud, initial_transform=previous_transform)
    integrated_cloud += scan.transform(new_transform)
    integrated_cloud = integrated_cloud.voxel_downsample()
    new_transforms.append(new_transform)

I basically just followed the Open3D tutorials (which already gave good results) and improved my pipeline when I found misalignments in the results.
Your results may vary, especially depending on your scan data, and you might find your own problems.

[LocoMaxim]

Hi! Thanks a lot for sharing your experience with multiway registration and Open3D — it was really helpful for a project I'm working on. I'm using a setup with a turntable and a fixed scanner to capture small, detailed objects. In your opinion, is this setup enough to accurately reconstruct all surfaces? Did you run into issues with hidden or hard-to-reach areas? And if so, did you find any effective solutions (like tilting the object, doing multiple passes, etc.)? I'm also considering a configuration with 5 fixed scanners arranged around a static object. Have you ever tried something like that? Any suggestions on: the optimal placement of the scanners for good coverage? how to manage data fusion from fixed positions (without a turntable)? I’d really appreciate any tips if you have time to share — thanks again! Alex Il giorno mer 9 apr 2025 alle ore 13:01 Tim Ohliger < ***@***.***> ha scritto:

…

The Multiway registration <https://www.open3d.org/docs/release/tutorial/pipelines/multiway_registration.html> tutorial worked for me, but I did some modifications: - I preprocessed my clouds rotating them to an initial alignment. Since I am using a turntable with 24 scan directions, I have roughly the same angles and am able to pre-align the clouds. If you do not have any initial alignment you could look into Global Registration (since ICP might fall into wrong local minima). - I also have to remove background (since it is not turning with the turntable) since I am labeling for semantic segmentation anyway, I start with just labeling background and removing it before fine-registration with ICP. If you do not use a turntable, you can probably skip this step. This is the only manual step for me (which can hopefully be replaced or aided by a machine learning model later) - For pairwise-registration I used the new tensor-based Multi-Scale-ICP <https://www.open3d.org/docs/release/tutorial/t_pipelines/t_icp_registration.html#Multi-Scale-ICP-Example> with TransformationEstimationPointToPlane. Make sure to call the estimate normals function on your clouds if you happen to not have normals in your .ply - For some scans, especially when the rotation angles are too large and the overlap between consecutive scans is too low, I would observe severe misalignments (causing a planar surface to show as multiple layers in the combined cloud). To fix this, I added a final step after the Multiway-Registration tutorial iteratively integrating scans while recomputing registration on the integrated cloud as target. Here an outline how this algorithm looks like: # Previous transforms come from multi-way registration exampleintegrated_cloud = scans[0]new_transforms = [previous_transforms[0]]for scan, previous_transform in zip(scans[1:], previous_transforms[1:]): new_transform = my_icp(scan, integrated_cloud, initial_transform=previous_transform) integrated_cloud += scan.transform(new_transform) integrated_cloud = integrated_cloud.voxel_downsample() new_transforms.append(new_transform) I basically just followed the Open3D tutorials (which already gave good results) and improved my pipeline when I found misalignments in the results. Your results may vary, especially depending on your scan data, and you might find your own problems. — Reply to this email directly, view it on GitHub <#7221 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BREW5SHU447KAB5G6XUXXIT2YT4ZFAVCNFSM6AAAAAB2NHTEPCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENZXGY3TOOA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[timohl]

In my setup, I can only scan from the side with a laser scanner on a linear axis.
This means if any surfaces on the target object are parallel to the laser line, they cannot be captured, or at least only the outline is captured with a hole on the surface where the laser cannot reach.
Optimally, I would like to add another sensor from top or diagonal to capture those areas, but for now I have to live with that.
Maybe tilting the linear axis could help, but I have not tried that yet.

With multiple views, you should always choose one reference view to transform all other views to.

In order to compute the initial transformations on the turntable (and also with fixed positions) it is a good idea to place some kind of reference object.
I found that my scanner is a tiny bit tilted compared to the turntable.
Therefore, I used a thin stick like object with a small mount on the bottom, which I placed vertically roughly in the middle of the turntable (like this: _|_ ).
Then I recorded from all angles and computed the mean position at two fixed heights.
With this I got the rotation axis and could compute the center and rotation matrix.

Not sure if this is standard procedure, but it worked for me. :)
I have also seen reference cubes with known dimensions used for that.

[LocoMaxim]

Hi again! Thanks a lot for your previous reply — it was really helpful and gave me a lot of clarity on some key aspects of my setup. I hope you don’t mind if I ask a few more questions, especially as I’m refining the scanning pipeline for a project involving small, detailed objects (some with complex geometry like glasses). 1. Have you tried using global registration methods (like FPFH or RANSAC) instead of ICP for initial alignment? Did you find any cases where they performed better? 2. How do you typically choose the reference view when doing multi-view registration? Do you stick to one fixed view or recompute it dynamically? 3. You mentioned using a vertical stick to estimate the rotation axis — that was brilliant! Do you think a setup with 5 fixed scanners (arranged around a static object) could also benefit from a similar calibration approach? Or would you suggest another strategy to align those views accurately? 4. Lastly, if you had to rebuild your setup from scratch, is there anything you’d definitely do differently, either in terms of hardware or processing? Thanks again for sharing your experience — I really appreciate it! Message ID: ***@***.***

…

com>

[timohl]

1. No, I have not tried global registration yet and I do not think I need it, since I have good initial alignment. I think it is more helpful if your scanner position is not fixed (e.g., a handheld device). Since I have well defined positions and rotation angles using turntable, I just measure the initial transformations once and save them.
2. I am just using the first capture as reference (turntable angle 0°)
3. Thanks :) For 5 fixed scanners, I would probably just take a reference object with distinguishable geometry (if you have access to 3D printing you could create a fancy one) and use point pair picking in CloudCompare or something like this. It does not have to be super accurate if you want to use ICP as a next step, but should give a good initial alignment. Make sure that there is some overlap between consecutive scanner positions, since ICP does not work if there is very low overlap (e.g., from 0° and 180°). The vertical stick solution just works for me since I am using one orthogonal scanner and a turntable.
4. Definitely adding some kind of additional top view, but that is also a budget question, since right now I can just use one scanner. For the processing, I am still improving it and working on it right now, since I still have problems with some objects that are very thin and partially parallel to the scan direction and provide bad overlap between consecutive scans. For most objects I am happy with the processing right now.

[LocoMaxim]

Hey! Hope you're doing well I’ve been making good progress thanks to your previous suggestions — they’ve been incredibly helpful. If you don’t mind, I’d love to ask a few more questions based on some recent developments: Have you ever tried building a fully automated pipeline (load → register → merge → export)? I’m experimenting with Open3D and CloudCompare scripting — wondering if you found a good modular approach for it. Do you notice significant differences between regular ICP and multi-scale ICP (point-to-plane)? Especially for small and thin objects like glasses, I’m trying to find the most stable setup. When using ICP after manual point picking, do you usually tweak specific parameters to avoid misalignment or instability? Have you worked with objects that are reflective or partially transparent (like glasses)? I’m considering using matte spray — curious if you’ve tested anything like that. In terms of calibration: Do you think using printed markers or fiducials (like ArUco codes) could help with initial alignment, or do they just overcomplicate things? Lastly, have you ever evaluated the quality of your final alignment against a CAD model or real measurements? I’m looking for practical ways to validate results beyond visual inspection. Thanks again for sharing your experience — it really helps to learn from someone working on a similar level. Wishing you smooth scans ahead! Il giorno gio 10 apr 2025 alle ore 15:07 Tim Ohliger < ***@***.***> ha scritto:

…

1. No, I have not tried global registration yet and I do not think I need it, since I have good initial alignment. I think it is more helpful if your scanner position is not fixed (e.g., a handheld device). Since I have well defined positions and rotation angles using turntable, I just measure the initial transformations once and save them. 2. I am just using the first capture as reference (turntable angle 0°) 3. Thanks :) For 5 fixed scanners, I would probably just take a reference object with distinguishable geometry (if you have access to 3D printing you could create a fancy one) and use point pair picking in CloudCompare or something like this. It does not have to be super accurate if you want to use ICP as a next step, but should give a good initial alignment. Make sure that there is some overlap between consecutive scanner positions, since ICP does not work if there is very low overlap (e.g., from 0° and 180°). The vertical stick solution just works for me since I am using one orthogonal scanner and a turntable. 4. Definitely adding some kind of additional top view, but that is also a budget question, since right now I can just use one scanner. For the processing, I am still improving it and working on it right now, since I still have problems with some objects that are very thin and partially parallel to the scan direction and provide bad overlap between consecutive scans. For most objects I am happy with the processing right now. — Reply to this email directly, view it on GitHub <#7221 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BREW5SAYTTNYDBH7TLKIOQD2YZUKNAVCNFSM6AAAAAB2NHTEPCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENZZGE2TSMA> . You are receiving this because you authored the thread.Message ID: ***@***.***>