1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
|
/**
\page tutorial-homography Tutorial: Homography estimation from points
\tableofcontents
\section intro Introduction
This tutorial shows how to estimate an homography from points. Two cases are considered in the next sections:
- points are matched without possible mismatches. In that case vpHomography::DLT() or vpHomography::HLM() are used to estimate the homography,
- points are matched with possible mismatches. In that case vpHomography::ransac() or vpHomography::robust() are used.
\section homography Homography estimation
Note that all the material (source code) described in this section is part of ViSP source code and could be downloaded using the following command:
\code
$ svn export https://github.com/lagadic/visp.git/trunk/tutorial/computer-vision
\endcode
Let us consider the following source code also available in tutorial-homography-from-points.cpp. To resume, we do the following:
- define four 3D points \c oP in an object frame,
- define an homogeneous transformation \c aMo from frame \c a to to the object frame \c o,
- define a similar homogeneous transformation \c bMo from frame \c b to to the object frame \c o,
- compute the coordinates of the four 3D points in the image plane \c a and \c b. These are the matched coordinates \c (xa,ya) and \c (xb,yb) that are then used to estimate the homography using either vpHomography::DLT() or vpHomography::HLM().
\include tutorial-homography-from-points.cpp
Now we give a line by line explanation of the code:
First we have to include the header of the vpHomography class.
\snippet tutorial-homography-from-points.cpp Include
In the main() function we first define the 3D coordinates of 4 points that are localized in the plane Z=0:
\snippet tutorial-homography-from-points.cpp Set 3D points
Then we define the homogeneous transformations between frames \c a, \c b and object frame \c o:
\snippet tutorial-homography-from-points.cpp Simulation
From these transformations we compute the coordinates of the points in the image plane \c a and \c b. For each point we have its coordinates \c (xa,ya) in frame \c a and \c (xb,yb) in frame \c b:
\snippet tutorial-homography-from-points.cpp Image plane coordinates
We have now matched couples of coordinates of four points that are used to estimate an homography between image plane \c a and \c b. Two methods are available, either using the DLT (Direct Linear Transform) algorithm or the HLM algorithm.
\snippet tutorial-homography-from-points.cpp Compute homography
\note Note that vpHomography::HLM() allows to consider points that are not coplanar.
Once the homography is estimated, the vpHomography class allows to extract the 3D homogeneous transformation between frames \c a and \c b:
\snippet tutorial-homography-from-points.cpp Get transformation
Just for fun we print the values to this transformation using:
\snippet tutorial-homography-from-points.cpp Print results
This code lead to the following output:
\code
Estimated displacement:
atb: 0.2016519874 -0.1008259937 0.1008259937
athetaub: -3 20 5
n: 0.2588190451 -1.124100812e-14 0.9659258263
\endcode
where we can see that the values for \c atb and \c athetaub are the one specified at the beginning of the source code during \c aMb initialization.
After we show how to retrieve the coordinates in pixels of a point (here point [3]) in the corresponding images using camera parameters:
\snippet tutorial-homography-from-points.cpp Get pixel coordinates
At the end, we show how to project a point with pixel coordinates from image \c b to image \c a using the homography:
\snippet tutorial-homography-from-points.cpp Project
This last part of the code produce the following output:
\code
Ground truth: Point 3 in pixels in frame b: 377.9450564, 193.9928711
Ground truth: Point 3 in pixels in frame a: 353.8501593, 486.1851856
Estimation from homography: Point 3 in pixels in frame a: 353.8501593, 486.1851856
\endcode
\section ransac Ransac or robust homography estimation
Note that all the material (source code) described in this section is part of ViSP source code and could be downloaded using the following command:
\code
$ svn export https://github.com/lagadic/visp.git/trunk/tutorial/detection/matching
\endcode
This section follows the \ref tutorial-matching. It explains how to exploit couples of matched points obtained using SURF detector in order to estimate an homography that allows to reject mismatched couples of points. The homography is then used to track a postcard from its initial position in the reference image.
Let us consider the following source code also available in tutorial-matching-keypoint-homography.cpp.
\include tutorial-matching-keypoint-homography.cpp
The command line allows to use either Ransac algorithm of a robust M-estimator approach:
\code
% ./tutorial-matching-keypoint-homography 0 // to run Ransac
% ./tutorial-matching-keypoint-homography 1 // to run the robust M-estimator
\endcode
Here after is the resulting video. The left image represents the reference image. The right images correspond to the successive images of the input video. All the green lines extremities represent the points that are well matched and used in the homography estimation process. All the red lines represent couple of matched points that are rejected by the robust estimator.
\htmlonly
<iframe width="560" height="315" src="https://www.youtube.com/embed/g-aEH5Chmsg" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
\endhtmlonly
Now, let us explain the new lines that were introduced to estimate the homography.
First we detect the command line arguments to be able later to user either Ransac or the robust M-estimator:
\snippet tutorial-matching-keypoint-homography.cpp Select method
We also initialize the coordinates of the pixels in the reference image that correspond to the postcard corners. These coordinates were obtained after a user initial click. To simplify the code, we set directly the coordinates of the points:
\snippet tutorial-matching-keypoint-homography.cpp Set coordinates
Using these coordinates, we display red lines around the postcard:
\snippet tutorial-matching-keypoint-homography.cpp Display
We need also to define roughly the parameters of our camera:
\snippet tutorial-matching-keypoint-homography.cpp Camera
For each new image, once points are matched using:
\snippet tutorial-matching-keypoint-homography.cpp Matching
We allocate new containers useful for the homography estimation. The coordinates of the points in the reference image will be stored in \c (mPref_x, mPref_y), while the corresponding coordinates in the current image will be stored in \c (mPcur_x, mPcur_y). We also allocate \c inliers a vector of boolean that will indicate if a couple of point is an inlier or an outlier:
\snippet tutorial-matching-keypoint-homography.cpp Allocation
To estimate the homography we need first to convert the points from pixel to meters:
\snippet tutorial-matching-keypoint-homography.cpp Pixel conversion
We can now estimate the homography using either Ransac or the robust M-estimator approach:
\snippet tutorial-matching-keypoint-homography.cpp Homography estimation
For Ransac we consider that at least 50 percent of the points should be inliers (\c mPref_x.size()/2) to reach a consensus and that a couple of point is stated as an inlier if the reprojection error is lower than 2 pixels (\c 2.0/cam.get_px()).
Then using the homography, we project the coordinates of the postcard corners in the current image:
\snippet tutorial-matching-keypoint-homography.cpp Projection
We use these coordinates to draw blue lines around the postcard:
\snippet tutorial-matching-keypoint-homography.cpp Display contour
Since the homography estimation updates the status of the couples of matched points as inliers or outliers, between the matched points we are able to draw green lines when they are inliers, or red lines when they are outliers.
\snippet tutorial-matching-keypoint-homography.cpp Display matches
*/
|
