1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
|
Harris corner detector {#tutorial_harris_detector}
======================
@tableofcontents
@next_tutorial{tutorial_good_features_to_track}
| | |
| -: | :- |
| Original author | Ana Huamán |
| Compatibility | OpenCV >= 3.0 |
Goal
----
In this tutorial you will learn:
- What features are and why they are important
- Use the function @ref cv::cornerHarris to detect corners using the Harris-Stephens method.
Theory
------
### What is a feature?
- In computer vision, usually we need to find matching points between different frames of an
environment. Why? If we know how two images relate to each other, we can use *both* images to
extract information of them.
- When we say **matching points** we are referring, in a general sense, to *characteristics* in
the scene that we can recognize easily. We call these characteristics **features**.
- **So, what characteristics should a feature have?**
- It must be *uniquely recognizable*
### Types of Image Features
To mention a few:
- Edges
- **Corners** (also known as interest points)
- Blobs (also known as regions of interest )
In this tutorial we will study the *corner* features, specifically.
### Why is a corner so special?
- Because, since it is the intersection of two edges, it represents a point in which the
directions of these two edges *change*. Hence, the gradient of the image (in both directions)
have a high variation, which can be used to detect it.
### How does it work?
- Let's look for corners. Since corners represents a variation in the gradient in the image, we
will look for this "variation".
- Consider a grayscale image \f$I\f$. We are going to sweep a window \f$w(x,y)\f$ (with displacements \f$u\f$
in the x direction and \f$v\f$ in the y direction) \f$I\f$ and will calculate the variation of
intensity.
\f[E(u,v) = \sum _{x,y} w(x,y)[ I(x+u,y+v) - I(x,y)]^{2}\f]
where:
- \f$w(x,y)\f$ is the window at position \f$(x,y)\f$
- \f$I(x,y)\f$ is the intensity at \f$(x,y)\f$
- \f$I(x+u,y+v)\f$ is the intensity at the moved window \f$(x+u,y+v)\f$
- Since we are looking for windows with corners, we are looking for windows with a large variation
in intensity. Hence, we have to maximize the equation above, specifically the term:
\f[\sum _{x,y}[ I(x+u,y+v) - I(x,y)]^{2}\f]
- Using *Taylor expansion*:
\f[E(u,v) \approx \sum _{x,y}[ I(x,y) + u I_{x} + vI_{y} - I(x,y)]^{2}\f]
- Expanding the equation and cancelling properly:
\f[E(u,v) \approx \sum _{x,y} u^{2}I_{x}^{2} + 2uvI_{x}I_{y} + v^{2}I_{y}^{2}\f]
- Which can be expressed in a matrix form as:
\f[E(u,v) \approx \begin{bmatrix}
u & v
\end{bmatrix}
\left (
\displaystyle \sum_{x,y}
w(x,y)
\begin{bmatrix}
I_x^{2} & I_{x}I_{y} \\
I_xI_{y} & I_{y}^{2}
\end{bmatrix}
\right )
\begin{bmatrix}
u \\
v
\end{bmatrix}\f]
- Let's denote:
\f[M = \displaystyle \sum_{x,y}
w(x,y)
\begin{bmatrix}
I_x^{2} & I_{x}I_{y} \\
I_xI_{y} & I_{y}^{2}
\end{bmatrix}\f]
- So, our equation now is:
\f[E(u,v) \approx \begin{bmatrix}
u & v
\end{bmatrix}
M
\begin{bmatrix}
u \\
v
\end{bmatrix}\f]
- A score is calculated for each window, to determine if it can possibly contain a corner:
\f[R = det(M) - k(trace(M))^{2}\f]
where:
- det(M) = \f$\lambda_{1}\lambda_{2}\f$
- trace(M) = \f$\lambda_{1}+\lambda_{2}\f$
a window with a score \f$R\f$ greater than a certain value is considered a "corner"
Code
----
@add_toggle_cpp
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/tree/4.x/samples/cpp/tutorial_code/TrackingMotion/cornerHarris_Demo.cpp)
@include samples/cpp/tutorial_code/TrackingMotion/cornerHarris_Demo.cpp
@end_toggle
@add_toggle_java
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/tree/4.x/samples/java/tutorial_code/TrackingMotion/harris_detector/CornerHarrisDemo.java)
@include samples/java/tutorial_code/TrackingMotion/harris_detector/CornerHarrisDemo.java
@end_toggle
@add_toggle_python
This tutorial code's is shown lines below. You can also download it from
[here](https://github.com/opencv/opencv/tree/4.x/samples/python/tutorial_code/TrackingMotion/harris_detector/cornerHarris_Demo.py)
@include samples/python/tutorial_code/TrackingMotion/harris_detector/cornerHarris_Demo.py
@end_toggle
Explanation
-----------
Result
------
The original image:
The detected corners are surrounded by a small black circle
|
