Skip to content

[GSoC] Loop closure detection. #3002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: 5.x
Choose a base branch
from

Conversation

zihaomu
Copy link
Member

@zihaomu zihaomu commented Jul 11, 2021

Merge with: opencv/opencv#20608

[GSoC] Loop closure detection.

Hi, this is a draft PR and only contains some implementation ideas.
DNN model (HF-Net) is used as a feature extractor in my project. I try to make the model support all platforms, but unfortunately, only the OpenVINO backend is currently supported. And more backend models will be supported in the future.

The submap.cpp was moved to opencv/opencv/modules/3d/include/opencv2/3d/detail/submap.hpp.
So I made a new PR 20608 for opencv/opencv.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • There is reference to original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake
opencv=loop_closure_detection_submap

@savuor savuor self-assigned this Jul 14, 2021
@zihaomu zihaomu marked this pull request as ready for review August 9, 2021 13:24
Comment on lines 124 to 140
template<typename MatType>
void Submap<MatType>::addEdgeToSubmap(const int tarSubmapID, const Affine3f& tarPose )
{
// duplicate check.
auto& iter = constraints.find(tarSubmapID);

// if there is NO edge of currSubmap to tarSubmap.
if(iter == constraints.end())
{
// Frome pose to tarPose transformation
Affine3f estimatePose = tarPose * pose.inv();

// Create new Edge.
PoseConstraint& preConstrain = getConstraint(tarSubmapID);
preConstrain->accumulatePose(estimatePose, 1);
}
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @savuor. When the LOOP is detected, I need to add new edges and constraints. Is this implementation right?

@zihaomu zihaomu force-pushed the loop_closure_detecion branch from 6f66d71 to d14c636 Compare August 13, 2021 15:39
@@ -136,6 +143,10 @@ class LargeKinfuImpl : public LargeKinfu

int frameCounter;
Affine3f pose;

// Loop Closure
bool useLCD;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need useLCD field? We can check lcd pointer instead

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for code reviewing. it will be updated in the next version.

@zihaomu
Copy link
Member Author

zihaomu commented Aug 26, 2021

The compilation error is the use of an unimplemented function, which has been implemented at the PR 20608 for opencv/opencv.

@alalek
Copy link
Member

alalek commented Aug 26, 2021

@zihaomu Builds still fail with both PRs.

@zihaomu
Copy link
Member Author

zihaomu commented Aug 27, 2021

@zihaomu Builds still fail with both PRs.

@alalek Hi, Thanks for your reply. From the compiled information, it seems that there is something wrong with CN CI System.: https://build.opencv.org.cn/job/precommit/job/windows10/3115/execution/node/9/log/

@alalek
Copy link
Member

alalek commented Aug 29, 2021

@zihaomu
Copy link
Member Author

zihaomu commented Aug 30, 2021

Hi @alalek. Thank you, the code has been updated, but it still fails on CI on ARM platform. I don't know how to deal with that. https://pullrequest.opencv.org/buildbot/builders/precommit-contrib_android/builds/14815/steps/compile%20release/logs/stdio

@alalek
Copy link
Member

alalek commented Aug 30, 2021

@zihaomu Required builds are green now.

@asmorkalov asmorkalov added this to the 5.0 milestone Oct 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants