1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
|
/**
\page tutorial-matching Tutorial: Keypoints matching
\tableofcontents
\section intro_matching Introduction
This tutorial focuses on keypoints detection and matching. You will learn how to detect keypoints on a reference image considered here as the first image of an mpeg video. Then in the next images of the video, keypoints that match those detected in the reference image are displayed. To leverage keypoints detection and matching capabilities ViSP should be build with OpenCV 3rd party.
Note that all the material (source code and video) described in this tutorial is part of ViSP source code and could be downloaded using the following command:
\code
$ svn export https://github.com/lagadic/visp.git/trunk/tutorial/detection/matching
\endcode
\note This tutorial is adapted if your OpenCV version is equal or greater than 2.1.1 version.
\note We assume that you are familiar with video framegrabbing described in \ref tutorial-grabber and with the way to display an image in a window described in \ref tutorial-getting-started.
\section orb_keypoints ORB keypoints detection and matching
Let us consider the following source code also available in tutorial-matching-keypoint.cpp.
\include tutorial-matching-keypoint.cpp
Here after is the resulting video. The left image represents the reference image. The right images correspond to the successive images of the input video. All the green lines extremities represent the points that are matched.
\htmlonly
<iframe width="560" height="315" src="https://www.youtube.com/embed/sMbed_oYJgQ" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
\endhtmlonly
Now, let us explain the lines dedicated to the ORB keypoint detection and matching usage.
First we have to include the header of the vpKeyPoint class that is a wrapper over OpenCV classes.
\snippet tutorial-matching-keypoint.cpp Include
Note that this class is only available if ViSP was build with OpenCV. This is ensured by the check of VISP_HAVE_OPENCV_VERSION macro.
\snippet tutorial-matching-keypoint.cpp Define
Then we open the mpeg video stream and grab the first image of the video that is stored in \c I container. The vpKeyPoint class is instantiated and keypoints are detected on the first image which is considered as the reference image:
\snippet tutorial-matching-keypoint.cpp Construction
The next lines are used to create image \c Idisp to render the matching results; left image for the reference image, right image for the current image that is processed:
\snippet tutorial-matching-keypoint.cpp Create image
Then a display using OpenCV is created and image \c Idisp is rendered:
\snippet tutorial-matching-keypoint.cpp Init display
We enter then in the \c while() loop where a new image is acquired from the video stream and inserted in the right part of image \c Idisp dedicated to rendering of the matching results.
\snippet tutorial-matching-keypoint.cpp Acquisition
We start the rendering by displaying the rendered image and by drawing a white vertical line to separate the reference image from the current one:
\snippet tutorial-matching-keypoint.cpp Display
Keypoint matches between the reference image and the current image \c I are detected using:
\snippet tutorial-matching-keypoint.cpp Matching
Then we parse all the matches to retrieve the coordinates of the points in the reference image (in \c iPref variable) and in the current image (in \c iPcur variable):
\snippet tutorial-matching-keypoint.cpp Get matches
Next we draw green lines between the matched points:
\snippet tutorial-matching-keypoint.cpp Display matches
At the end of the iteration, we flush all the previous display to the render window:
\snippet tutorial-matching-keypoint.cpp Display flush
\section other_keypoints Using others types of keypoints
Using other types of detectors / descriptors (SIFT, SURF, etc.) or matchers (Brute Force, Flann based matcher) is also possible. This can be easily done by using the correct OpenCV identifier name. For example, if you want to use the couple of detector / descriptor SIFT with a matching using FLANN, you only have to change the following lines :
\snippet tutorial-matching-keypoint-SIFT.cpp Construction
A complete example is given in tutorial-matching-keypoint-SIFT.cpp
Available types of detectors, extractors or matchers depend on OpenCV version. Check the OpenCV documentation to know which ones are available.
Due to some patents, SIFT and SURF keypoints are available in a separate module in OpenCV: in nonfree module in OpenCV version before 3.0.0 and in xfeatures2d in OpenCV version from 3.0.0 if OpenCV contrib modules are build. If you want to use them, be sure that you have the nonfree module or xfeatures2d module.
Some usage restrictions could also exist according to the usage you plan (e.g. research or commercial use).
You can now follow \ref tutorial-homography to see how to exploit couple of matched points in order to estimate an homography that allows to track the position of an object.
*/
|
