Merge remote-tracking branch 'upstream/3.4' into merge-3.4

Author: alalek

doc/py_tutorials/py_calib3d/py_epipolar_geometry/py_epipolar_geometry.markdown

@@ -79,7 +79,7 @@ from matplotlib import pyplot as plt
 img1 = cv.imread('myleft.jpg',0)  #queryimage # left image
 img2 = cv.imread('myright.jpg',0) #trainimage # right image
 
-sift = cv.SIFT()
+sift = cv.SIFT_create()
 
 # find the keypoints and descriptors with SIFT
 kp1, des1 = sift.detectAndCompute(img1,None)
@@ -93,14 +93,12 @@ search_params = dict(checks=50)
 flann = cv.FlannBasedMatcher(index_params,search_params)
 matches = flann.knnMatch(des1,des2,k=2)
 
-good = []
 pts1 = []
 pts2 = []
 
 # ratio test as per Lowe's paper
 for i,(m,n) in enumerate(matches):
     if m.distance < 0.8*n.distance:
-        good.append(m)
         pts2.append(kp2[m.trainIdx].pt)
         pts1.append(kp1[m.queryIdx].pt)
 @endcode

doc/tutorials/imgproc/erosion_dilatation/erosion_dilatation.markdown

@@ -84,57 +84,198 @@ This tutorial's code is shown below. You can also download it
 Explanation
 -----------
 
--#  Most of the material shown here is trivial (if you have any doubt, please refer to the tutorials in
-    previous sections). Let's check the general structure of the C++ program:
+@add_toggle_cpp
+Most of the material shown here is trivial (if you have any doubt, please refer to the tutorials in
+previous sections). Let's check the general structure of the C++ program:
+
+@snippet cpp/tutorial_code/ImgProc/Morphology_1.cpp main
+
+-#   Load an image (can be BGR or grayscale)
+-#   Create two windows (one for dilation output, the other for erosion)
+-#   Create a set of two Trackbars for each operation:
+    -   The first trackbar "Element" returns either **erosion_elem** or **dilation_elem**
+    -   The second trackbar "Kernel size" return **erosion_size** or **dilation_size** for the
+        corresponding operation.
+-#  Call once erosion and dilation to show the initial image.
+
+
+Every time we move any slider, the user's function **Erosion** or **Dilation** will be
+called and it will update the output image based on the current trackbar values.
+
+Let's analyze these two functions:
+
+#### The erosion function
+
+@snippet cpp/tutorial_code/ImgProc/Morphology_1.cpp erosion
+
+The function that performs the *erosion* operation is @ref cv::erode . As we can see, it
+receives three arguments:
+-   *src*: The source image
+-   *erosion_dst*: The output image
+-   *element*: This is the kernel we will use to perform the operation. If we do not
+    specify, the default is a simple `3x3` matrix. Otherwise, we can specify its
+    shape. For this, we need to use the function cv::getStructuringElement :
+    @snippet cpp/tutorial_code/ImgProc/Morphology_1.cpp kernel
+
+    We can choose any of three shapes for our kernel:
+
+    -   Rectangular box: MORPH_RECT
+    -   Cross: MORPH_CROSS
+    -   Ellipse: MORPH_ELLIPSE
+
+    Then, we just have to specify the size of our kernel and the *anchor point*. If not
+    specified, it is assumed to be in the center.
+
+That is all. We are ready to perform the erosion of our image.
+
+#### The dilation function
+
+The code is below. As you can see, it is completely similar to the snippet of code for **erosion**.
+Here we also have the option of defining our kernel, its anchor point and the size of the operator
+to be used.
+@snippet cpp/tutorial_code/ImgProc/Morphology_1.cpp dilation
+@end_toggle
+
+@add_toggle_java
+Most of the material shown here is trivial (if you have any doubt, please refer to the tutorials in
+previous sections). Let's check however the general structure of the java class. There are 4 main
+parts in the java class:
+
+- the class constructor which setups the window that will be filled with window components
+- the `addComponentsToPane` method, which fills out the window
+- the `update` method, which determines what happens when the user changes any value
+- the `main` method, which is the entry point of the program
+
+In this tutorial we will focus on the `addComponentsToPane` and `update` methods. However, for completion the
+steps followed in the constructor are:
+
+-#  Load an image (can be BGR or grayscale)
+-#  Create a window
+-#  Add various control components with `addComponentsToPane`
+-#  show the window
+
+The components were added by the following method:
+
+@snippet java/tutorial_code/ImgProc/erosion_dilatation/MorphologyDemo1.java components
+
+In short we
+
+-#  create a panel for the sliders
+-#  create a combo box for the element types
+-#  create a slider for the kernel size
+-#  create a combo box for the morphology function to use (erosion or dilation)
+
+The action and state changed listeners added call at the end the `update` method which updates
+the image based on the current slider values. So every time we move any slider, the `update` method is triggered.
 
-    -   Load an image (can be BGR or grayscale)
-    -   Create two windows (one for dilation output, the other for erosion)
-    -   Create a set of two Trackbars for each operation:
-        -   The first trackbar "Element" returns either **erosion_elem** or **dilation_elem**
-        -   The second trackbar "Kernel size" return **erosion_size** or **dilation_size** for the
-            corresponding operation.
-    -   Every time we move any slider, the user's function **Erosion** or **Dilation** will be
-        called and it will update the output image based on the current trackbar values.
+#### Updating the image
 
-    Let's analyze these two functions:
+To update the image we used the following implementation:
 
--#  **erosion:**
-    @snippet cpp/tutorial_code/ImgProc/Morphology_1.cpp erosion
+@snippet java/tutorial_code/ImgProc/erosion_dilatation/MorphologyDemo1.java update
 
-    -   The function that performs the *erosion* operation is @ref cv::erode . As we can see, it
-        receives three arguments:
-        -   *src*: The source image
-        -   *erosion_dst*: The output image
-        -   *element*: This is the kernel we will use to perform the operation. If we do not
-            specify, the default is a simple `3x3` matrix. Otherwise, we can specify its
-            shape. For this, we need to use the function cv::getStructuringElement :
-            @snippet cpp/tutorial_code/ImgProc/Morphology_1.cpp kernel
+In other words we
 
-            We can choose any of three shapes for our kernel:
+-# get the structuring element the user chose
+-# execute the **erosion** or **dilation** function based on `doErosion`
+-# reload the image with the morphology applied
+-# repaint the frame
 
-            -   Rectangular box: MORPH_RECT
-            -   Cross: MORPH_CROSS
-            -   Ellipse: MORPH_ELLIPSE
+Let's analyze the `erode` and `dilate` methods:
 
-            Then, we just have to specify the size of our kernel and the *anchor point*. If not
-            specified, it is assumed to be in the center.
+#### The erosion method
 
-    -   That is all. We are ready to perform the erosion of our image.
-@note Additionally, there is another parameter that allows you to perform multiple erosions
-(iterations) at once. However, We haven't used it in this simple tutorial. You can check out the
-reference for more details.
+@snippet java/tutorial_code/ImgProc/erosion_dilatation/MorphologyDemo1.java erosion
 
--#  **dilation:**
+The function that performs the *erosion* operation is @ref cv::erode . As we can see, it
+receives three arguments:
+-   *src*: The source image
+-   *erosion_dst*: The output image
+-   *element*: This is the kernel we will use to perform the operation. For specifying the shape, we need to use
+    the function cv::getStructuringElement :
+    @snippet java/tutorial_code/ImgProc/erosion_dilatation/MorphologyDemo1.java kernel
 
-    The code is below. As you can see, it is completely similar to the snippet of code for **erosion**.
-    Here we also have the option of defining our kernel, its anchor point and the size of the operator
-    to be used.
-    @snippet cpp/tutorial_code/ImgProc/Morphology_1.cpp dilation
+    We can choose any of three shapes for our kernel:
+
+    -   Rectangular box: CV_SHAPE_RECT
+    -   Cross: CV_SHAPE_CROSS
+    -   Ellipse: CV_SHAPE_ELLIPSE
+
+    Together with the shape we specify the size of our kernel and the *anchor point*. If the anchor point is not
+    specified, it is assumed to be in the center.
+
+That is all. We are ready to perform the erosion of our image.
+
+#### The dilation function
+
+The code is below. As you can see, it is completely similar to the snippet of code for **erosion**.
+Here we also have the option of defining our kernel, its anchor point and the size of the operator
+to be used.
+@snippet java/tutorial_code/ImgProc/erosion_dilatation/MorphologyDemo1.java dilation
+@end_toggle
+
+@add_toggle_python
+Most of the material shown here is trivial (if you have any doubt, please refer to the tutorials in
+previous sections). Let's check the general structure of the python script:
+
+@snippet python/tutorial_code/imgProc/erosion_dilatation/morphology_1.py main
+
+-#  Load an image (can be BGR or grayscale)
+-#  Create two windows (one for erosion output, the other for dilation) with a set of trackbars each
+    -   The first trackbar "Element" returns the value for the morphological type that will be mapped
+        (1 = rectangle, 2 = cross, 3 = ellipse)
+    -   The second trackbar "Kernel size" returns the size of the element for the
+        corresponding operation
+-#  Call once erosion and dilation to show the initial image
+
+Every time we move any slider, the user's function **erosion** or **dilation** will be
+called and it will update the output image based on the current trackbar values.
+
+Let's analyze these two functions:
+
+#### The erosion function
+
+@snippet python/tutorial_code/imgProc/erosion_dilatation/morphology_1.py erosion
+
+The function that performs the *erosion* operation is @ref cv::erode . As we can see, it
+receives two arguments and returns the processed image:
+-   *src*: The source image
+-   *element*: The kernel we will use to perform the operation. We can specify its
+    shape by using the function cv::getStructuringElement :
+    @snippet python/tutorial_code/imgProc/erosion_dilatation/morphology_1.py kernel
+
+    We can choose any of three shapes for our kernel:
+
+    -   Rectangular box: MORPH_RECT
+    -   Cross: MORPH_CROSS
+    -   Ellipse: MORPH_ELLIPSE
+
+Then, we just have to specify the size of our kernel and the *anchor point*. If the anchor point not
+specified, it is assumed to be in the center.
+
+That is all. We are ready to perform the erosion of our image.
+
+#### The dilation function
+
+The code is below. As you can see, it is completely similar to the snippet of code for **erosion**.
+Here we also have the option of defining our kernel, its anchor point and the size of the operator
+to be used.
+
+@snippet python/tutorial_code/imgProc/erosion_dilatation/morphology_1.py dilation
+@end_toggle
+
+@note Additionally, there are further parameters that allow you to perform multiple erosions/dilations
+(iterations) at once and also set the border type and value. However, We haven't used those
+in this simple tutorial. You can check out the reference for more details.
 
 Results
 -------
 
-Compile the code above and execute it with an image as argument. For instance, using this image:
+Compile the code above and execute it (or run the script if using python) with an image as argument.
+If you do not provide an image as argument the default sample image
+([LinuxLogo.jpg](https://github.com/opencv/opencv/tree/master/samples/data/LinuxLogo.jpg)) will be used.
+
+For instance, using this image:
 
 ![](images/Morphology_1_Tutorial_Original_Image.jpg)
 
@@ -143,3 +284,4 @@ naturally. Try them out! You can even try to add a third Trackbar to control the
 iterations.
 
 ![](images/Morphology_1_Result.jpg)
+(depending on the programming language the output might vary a little or be only 1 window)

modules/calib3d/doc/calib3d.bib

@@ -40,6 +40,14 @@ @article{penate2013exhaustive
   publisher={IEEE}
 }
 
+@inproceedings{Terzakis20,
+  author = {Terzakis, George and Lourakis, Manolis},
+  year = {2020},
+  month = {09},
+  pages = {},
+  title = {A Consistently Fast and Globally Optimal Solution to the Perspective-n-Point Problem}
+}
+
 @inproceedings{strobl2011iccv,
   title={More accurate pinhole camera calibration with imperfect planar target},
   author={Strobl, Klaus H. and Hirzinger, Gerd},

modules/calib3d/include/opencv2/calib3d.hpp

@@ -471,6 +471,7 @@ enum SolvePnPMethod {
                               //!<   - point 1: [ squareLength / 2,  squareLength / 2, 0]
                               //!<   - point 2: [ squareLength / 2, -squareLength / 2, 0]
                               //!<   - point 3: [-squareLength / 2, -squareLength / 2, 0]
+    SOLVEPNP_SQPNP       = 8, //!< SQPnP: A Consistently Fast and Globally OptimalSolution to the Perspective-n-Point Problem @cite Terzakis20
 #ifndef CV_DOXYGEN
     SOLVEPNP_MAX_COUNT        //!< Used for count
 #endif
@@ -946,6 +947,9 @@ It requires 4 coplanar object points defined in the following order:
   - point 1: [ squareLength / 2,  squareLength / 2, 0]
   - point 2: [ squareLength / 2, -squareLength / 2, 0]
   - point 3: [-squareLength / 2, -squareLength / 2, 0]
+-   **SOLVEPNP_SQPNP** Method is based on the paper "A Consistently Fast and Globally Optimal Solution to the
+Perspective-n-Point Problem" by G. Terzakis and M.Lourakis (@cite Terzakis20). It requires 3 or more points.
+
 
 The function estimates the object pose given a set of object points, their corresponding image
 projections, as well as the camera intrinsic matrix and the distortion coefficients, see the figure below
@@ -1069,6 +1073,7 @@ a 3D point expressed in the world frame into the camera frame:
          - point 1: [ squareLength / 2,  squareLength / 2, 0]
          - point 2: [ squareLength / 2, -squareLength / 2, 0]
          - point 3: [-squareLength / 2, -squareLength / 2, 0]
+    -  With **SOLVEPNP_SQPNP** input points must be >= 3
  */
 CV_EXPORTS_W bool solvePnP( InputArray objectPoints, InputArray imagePoints,
                             InputArray cameraMatrix, InputArray distCoeffs,
@@ -2598,10 +2603,10 @@ CV_EXPORTS void convertPointsHomogeneous( InputArray src, OutputArray dst );
 floating-point (single or double precision).
 @param points2 Array of the second image points of the same size and format as points1 .
 @param method Method for computing a fundamental matrix.
--   **CV_FM_7POINT** for a 7-point algorithm. \f$N = 7\f$
--   **CV_FM_8POINT** for an 8-point algorithm. \f$N \ge 8\f$
--   **CV_FM_RANSAC** for the RANSAC algorithm. \f$N \ge 8\f$
--   **CV_FM_LMEDS** for the LMedS algorithm. \f$N \ge 8\f$
+-   @ref FM_7POINT for a 7-point algorithm. \f$N = 7\f$
+-   @ref FM_8POINT for an 8-point algorithm. \f$N \ge 8\f$
+-   @ref FM_RANSAC for the RANSAC algorithm. \f$N \ge 8\f$
+-   @ref FM_LMEDS for the LMedS algorithm. \f$N \ge 8\f$
 @param ransacReprojThreshold Parameter used only for RANSAC. It is the maximum distance from a point to an epipolar
 line in pixels, beyond which the point is considered an outlier and is not used for computing the
 final fundamental matrix. It can be set to something like 1-3, depending on the accuracy of the

modules/calib3d/src/solvepnp.cpp

@@ -47,6 +47,7 @@
 #include "p3p.h"
 #include "ap3p.h"
 #include "ippe.hpp"
+#include "sqpnp.hpp"
 #include "calib3d_c_api.h"
 
 #include "usac.hpp"
@@ -796,7 +797,8 @@ int solvePnPGeneric( InputArray _opoints, InputArray _ipoints,
 
     Mat opoints = _opoints.getMat(), ipoints = _ipoints.getMat();
     int npoints = std::max(opoints.checkVector(3, CV_32F), opoints.checkVector(3, CV_64F));
-    CV_Assert( ( (npoints >= 4) || (npoints == 3 && flags == SOLVEPNP_ITERATIVE && useExtrinsicGuess) )
+    CV_Assert( ( (npoints >= 4) || (npoints == 3 && flags == SOLVEPNP_ITERATIVE && useExtrinsicGuess)
+                || (npoints >= 3 && flags == SOLVEPNP_SQPNP) )
                && npoints == std::max(ipoints.checkVector(2, CV_32F), ipoints.checkVector(2, CV_64F)) );
 
     opoints = opoints.reshape(3, npoints);
@@ -981,6 +983,14 @@ int solvePnPGeneric( InputArray _opoints, InputArray _ipoints,
             }
         } catch (...) { }
     }
+    else if (flags == SOLVEPNP_SQPNP)
+    {
+        Mat undistortedPoints;
+        undistortPoints(ipoints, undistortedPoints, cameraMatrix, distCoeffs);
+
+        sqpnp::PoseSolver solver;
+        solver.solve(opoints, undistortedPoints, vec_rvecs, vec_tvecs);
+    }
     /*else if (flags == SOLVEPNP_DLS)
     {
         Mat undistortedPoints;
@@ -1008,7 +1018,8 @@ int solvePnPGeneric( InputArray _opoints, InputArray _ipoints,
         vec_tvecs.push_back(tvec);
     }*/
     else
-        CV_Error(CV_StsBadArg, "The flags argument must be one of SOLVEPNP_ITERATIVE, SOLVEPNP_P3P, SOLVEPNP_EPNP or SOLVEPNP_DLS");
+        CV_Error(CV_StsBadArg, "The flags argument must be one of SOLVEPNP_ITERATIVE, SOLVEPNP_P3P, "
+            "SOLVEPNP_EPNP, SOLVEPNP_DLS, SOLVEPNP_UPNP, SOLVEPNP_AP3P, SOLVEPNP_IPPE, SOLVEPNP_IPPE_SQUARE or SOLVEPNP_SQPNP");
 
     CV_Assert(vec_rvecs.size() == vec_tvecs.size());