1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
|
/**
\page tutorial-tracking-keypoint Tutorial: Keypoint tracking
\tableofcontents
\section tracking_keypoint_intro Introduction
With ViSP it is possible to track keypoints using OpenCV KLT tracker, an implementation of the Kanade-Lucas-Tomasi feature tracker.
All the material (source code and video) described in this tutorial is part of ViSP source code and could be downloaded using the following command:
\code
$ svn export https://github.com/lagadic/visp.git/trunk/tutorial/tracking/keypoint
\endcode
\section tracking_keypoint_klt KLT tracker
The following example code available in tutorial-klt-tracker.cpp shows how to use ViSP vpKltOpencv class to track KLT keypoints. This class is a wrapper over the OpenCV KLT tracker implementation.
\include tutorial-klt-tracker.cpp
The video shows the result of the tracking:
\htmlonly
<iframe width="560" height="315" src="https://www.youtube.com/embed/ZYOG4kJPtaM" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
\endhtmlonly
The previous example can be run without command line options. In that case, keypoints are automatically detected before tracking.
\code
$ ./tutorial-klt-tracker
\endcode
It can also be run with [--init-by-click] option. In that case, the user can select a set of keypoints to track with a left mouse click. A right mouse click stops the keypoints selection and allows to start the tracking.
\code
$ ./tutorial-klt-tracker --init-by-click
\endcode
Here is the line by line explanation of the source :
\snippet tutorial-klt-tracker.cpp Include
We include here the headers that define the corresponding classes. vpImageConvert class will be used to convert ViSP images implemented in vpImage class into OpenCV
cv::Mat structures used as an entry by the KLT tracker. Then we include the header of vpKltOpencv class which is the wrapper over OpenCV KLT tracker implementation.
We need also to include a device to display the images. We retain vpDisplayOpenCV that works on Unix and Windows since OpenCV is mandatory by the tracker. Finally we include vpVideoReader header that will be used to read an mpeg input stream.
At the beginning of the main() function, we use the following macro to ensure that OpenCV requested by the tracker is available. Note that OpenCV will also be used to render the images and read the input video stream.
\snippet tutorial-klt-tracker.cpp Check 3rd party
The program starts by the creation of a vpVideoReader instance able to extract all the images of the video file \c video-postcard.mpeg. Here, the video should be in the same folder than the binary.
\snippet tutorial-klt-tracker.cpp Create reader
Then we extract the first image of the video in the gray level ViSP image container \c I.
\snippet tutorial-klt-tracker.cpp Acquire
This image \c I is then converted into \c cvI, an OpenCV image format that will be used by the tracker.
\snippet tutorial-klt-tracker.cpp Convert to OpenCV image
We also create a window associated to \c I, at position (0,0) in the screen, with "Klt tracking" as title, and display image \c I.
\snippet tutorial-klt-tracker.cpp Init display
From now we have to create an instance of the tracker and set the parameters of the Harris keypoint detector.
\snippet tutorial-klt-tracker.cpp Create tracker
The tracker is then initialized on \c cvI image.
\snippet tutorial-klt-tracker.cpp Init tracker
With the next line the user can know how many keypoints were detected automatically or selected by the user during initialization.
\snippet tutorial-klt-tracker.cpp How many features
\note If no keypoints were found, the next call to vpKltTracker::track() will throw an exception.
To detect more keypoints, you may decrease the quality parameter set with the following line:
\snippet tutorial-klt-tracker.cpp Quality
Until the end of the video, we get \c I the next image in ViSP format, display and convert it in OpenCV format. Then we track the Harris keypoints using KLT tracker before displaying the keypoints that are tracked with a red cross.
\snippet tutorial-klt-tracker.cpp While loop
We are waiting for a mouse click event on image \c I to end the program.
\snippet tutorial-klt-tracker.cpp Wait click
\section tracking_keypoint_klt_init KLT tracker with re-initialisation
Once initialized, the number of tracked features decreases over the time. Depending on a criteria, it may sense to detect and track new features online. A possible criteria is for example to compare the number of currently tracked features to the initial number of detected features. If less than a given percentage of features are tracked, you can start a new detection.
To get the number of detected or tracked features just call:
\code
tracker.getNbFeatures();
\endcode
Then the idea is to add the previously tracked features to the list of features that are detected.
The example tutorial-klt-tracker-with-reinit.cpp shows how to do that. In that example we start a new detection on frame 25. Compared to the previous code available in tutorial-klt-tracker.cpp we add the following lines:
\code
if (reader.getFrameIndex() == 25) {
std::cout << "Re initialize the tracker" << std::endl;
#if (VISP_HAVE_OPENCV_VERSION >= 0x020408)
// Save of previous features
std::vector<cv::Point2f> prev_features = tracker.getFeatures();
// Start a new feature detection
tracker.initTracking(cvI);
std::vector<cv::Point2f> new_features = tracker.getFeatures();
// Add previous features if they are not to close to detected one
double distance, minDistance_ = tracker.getMinDistance();
bool is_redundant;
for (size_t i=0; i < prev_features.size(); i++) {
// Test if a previous feature is not redundant with one of the newly detected
is_redundant = false;
for (size_t j=0; j < new_features.size(); j++){
distance = sqrt(vpMath::sqr(new_features[j].x-prev_features[i].x)
+ vpMath::sqr(new_features[j].y-prev_features[i].y));
if(distance < minDistance_){
is_redundant = true;
break;
}
}
if(is_redundant){
continue;
}
//std::cout << "Add previous feature with index " << i << std::endl;
tracker.addFeature(prev_features[i]);
}
#else
...
#endif
}
// Track the features
tracker.track(cvI);
\endcode
In this code we do the following:
- save the features that are tracked until now
- initialize the tracker to detect new features
- parse all the saved features and compare them to the newly detected features. If a previous feature is close in terms of geometric distance to a newly detected one, it is rejected (in our case less than 2 pixels). If not, it is added to the list of detected features.
\section tracking_keypoint_next Next tutorial
You are now ready to see the next \ref tutorial-tracking-me.
*/
|
