1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
|
/**
\page tutorial-bridge-opencv Tutorial: Bridge over OpenCV
\tableofcontents
ViSP is interfaced with OpenCV third party. In this tutorial we explain how to convert data such
as camera parameters or images from ViSP to OpenCV or \e vice \e versa.
\section tutorial_bridge_opencv_cam Camera parameters conversions
ViSP camera parameters are implemented in vpCameraParameters class. If you want to calibrate
a camera with ViSP tools follow \ref tutorial-calibration-intrinsic.
Let us recall the pinhole camera model implemented in ViSP. In this model, a scene view is formed
by projecting 3D points into the image plane using a perspective transformation.
\f[
\left[ \begin{array}{c}
u \\
v \\
1
\end{array}\right] =
\left[ \begin{array}{ccc}
p_x & 0 & u_0 \\
0 & p_y & v_0 \\
0 & 0 & 1
\end{array}\right]
\left[ \begin{array}{c}
X_c \\
Y_c \\
Z_c
\end{array}\right]
\f]
where:
- \f$(X_c,Y_c,Z_c)\f$ are the coordinates of a 3D point in the camera frame
- \f$(u,v)\f$ are the coordinates in pixels of the projected 3D point
- \f$(u_0,v_0)\f$ is a principal point that is usually near the image center
- \f$(p_x,p_y)\f$ are the focal lengths expressed in pixel units.
When \f$Z_c \neq 0\f$, the previous equation is equivalent to the following:
\f[
\begin{array}{lcl}
x &=& X_c / Z_c \\
y &=& Y_c / Z_c \\
u &=& u_0 + x \; p_x \\
v &=& v_0 + y \; p_y
\end{array}
\f]
Real lenses usually have some radial distortion. So, the above model is extended as:
\f[
\begin{array}{lcl}
x &=& X_c / Z_c \\
y &=& Y_c / Z_c \\
x^{'} &=& x (1 + k_{ud} r^2) \\
y^{'} &=& y (1 + k_{ud} r^2) \\
r^2 &=& x^2 + y^2 \\
u &=& u_0 + x^{'} \; p_x \\
v &=& v_0 + y^{'} \; p_y
\end{array}
\f]
where \f$k_{ud}\f$ is the first order radial distortion. Higher order distortion coefficients are not considered in ViSP.
\note In ViSP we introduce an extra parameter named \f$k_{du}\f$ which is the radial first order distortion
that allows to transform pixels in meters. If this parameter is unknown as in OpenCV a good
approximation is to consider \f$k_{du} = - k_{ud}\f$
Even if OpenCV notations are different, this model is exactly the same then the one used in
OpenCV and described <a href="http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html">here</a>
where higher order OpenCV distortion parameters are turned to 0.
The following table gives the correspondences between ViSP and OpenCV parameters:
\f[
\begin{array}{l|l}
ViSP & OpenCV \\ \hline
u_0 & c_x\\
v_0 & c_y \\
p_x & f_x \\
p_y & f_y \\
k_{ud} & k_1 \\
k_{du} & -k_1 \\
& k_2 = 0 \\
& k_3 = 0 \\
& k_4 = 0 \\
& k_5 = 0 \\
& k_6 = 0 \\
& p_1 = 0 \\
& p_2 = 0
\end{array}
\f]
From a coding point of view, let us consider the following code also available in
tutorial-bridge-opencv-camera-param.cpp where we initialize camera parameters using ViSP:
\snippet tutorial-bridge-opencv-camera-param.cpp Set ViSP camera parameters
These parameters could be used to initialize OpenCV camera parameters:
\snippet tutorial-bridge-opencv-camera-param.cpp Set OpenCV camera parameters
\section tutorial_bridge_opencv_image Image conversions
ViSP image is implemented in vpImage class, while OpenCV images in `cv::Mat` class. All the functions
that allow image conversion from ViSP to OpenCV or \e vice \e versa are implemented in vpImageConvert.
Some image conversion examples could be found in tutorial-bridge-opencv-camera-param.cpp.
There is also tutorial-bridge-opencv-image.cpp from where the following snippets are extracted.
\subsection tutorial_bridge_opencv_image_vp_to_cv From ViSP to OpenCV image conversion
- **Color image conversion**
The following code allows to read a color image with ViSP:
\snippet tutorial-bridge-opencv-image.cpp Load ViSP color image
and then convert the image in OpenCV using:
\snippet tutorial-bridge-opencv-image.cpp Convert to OpenCV color image
before saving the converted color image
\snippet tutorial-bridge-opencv-image.cpp Save OpenCV color image
- **Grey scale image conversion**
The following code allows to read a grey scale image with ViSP. Note here that if the
input image (ie. `monkey.jpeg`) is a color image, vpImageIo::read() converts implicitly
the color image in a grey scale image:
\snippet tutorial-bridge-opencv-image.cpp Load ViSP grey image
and then convert the image in OpenCV using:
\snippet tutorial-bridge-opencv-image.cpp Convert to OpenCV grey image
before saving the converted grey image
\snippet tutorial-bridge-opencv-image.cpp Save OpenCV grey image
\subsection tutorial_bridge_opencv_image_cv_to_vp From OpenCV to ViSP image conversion
- **Color image conversion**
The following code allows to read a color image with OpenCV:
\snippet tutorial-bridge-opencv-image.cpp Load OpenCV color image
and then convert the image in ViSP using:
\snippet tutorial-bridge-opencv-image.cpp Convert to ViSP color image
before saving the converted color image
\snippet tutorial-bridge-opencv-image.cpp Save ViSP color image
- **Grey scale image conversion**
The following code allows to read a grey scale image with OpenCV:
\snippet tutorial-bridge-opencv-image.cpp Load OpenCV grey image
and then convert the image in ViSP using:
\snippet tutorial-bridge-opencv-image.cpp Convert to ViSP grey image
before saving the converted grey image
\snippet tutorial-bridge-opencv-image.cpp Save ViSP grey image
\section tutorial_bridge_opencv_matrix Matrix conversions
ViSP matrices are implemented in different classes:
- vpMatrix for a non specific matrix
- vpHomogeneousMatrix for a 4-by-4 homogeneous matrix
- vpHomography for a 3-by-3 homography matrix
- vpRotationMatrix for a 3-by-3 rotation matrix
while OpenCV matrices are always implemented as a `cv::Mat`.
Up to now, there is no specific function that does the conversion between ViSP and OpenCv matrices.
Note that each element of a ViSP matrix is a double and that ViSP matrices are row-major.
In the following subsections we illustrate how to convert a vpMatrix to a `cv::Mat`, but the code could be easily
adapted to other ViSP specific matrices as long as the corresponding OpenCV matrix contains double
elements and that it has the same size as the ViSP specific matrix.
\subsection tutorial_bridge_opencv_matrix_cv_to_vp From OpenCV to ViSP matrix conversion
- The following code also available in tutorial-bridge-opencv-matrix.cpp shows how to create
an OpenCV row-major matrix with size 3 rows by 4 cols that contains double elements.
\snippet tutorial-bridge-opencv-matrix.cpp Create OpenCV matrix
- **Deep copy conversion**
To convert the previous matrix in a vpMatrix using a deep copy, you may use:
\snippet tutorial-bridge-opencv-matrix.cpp Convert to ViSP matrix with deep copy
The content of the matrices is now the following:
\verbatim
M_cv:
[1, 2, 3, 4;
5, 6, 7, 8;
9, 10, 11, 12]
M:
1 2 3 4
5 6 7 8
9 10 11 12
\endverbatim
\subsection tutorial_bridge_opencv_matrix_vp_to_cv From ViSP to OpenCV conversion
- The following code also available in tutorial-bridge-opencv-matrix.cpp shows how to create a ViSP
row-major matrix with size 3 rows by 4 cols that contains double elements.
\snippet tutorial-bridge-opencv-matrix.cpp Create ViSP matrix
- **Deep copy conversion**
To convert the previous matrix in a `cv::Mat` using a deep copy, you may use:
\snippet tutorial-bridge-opencv-matrix.cpp Convert to OpenCV matrix with deep copy
The content of the matrices is now the following:
\verbatim
M:
1 2 3 4
5 6 7 8
9 10 11 12
M_cv_deep:
[1, 2, 3, 4;
5, 6, 7, 8;
9, 10, 11, 12]
\endverbatim
- **Pointer assignment, without deep copy conversion**
For performance issues, there is also the possibility to just copy the data pointer. To this end, you may use:
\snippet tutorial-bridge-opencv-matrix.cpp Convert to OpenCV matrix without deep copy
The content of the matrices is now the following:
\verbatim
M:
1 2 3 4
5 6 7 8
9 10 11 12
M_cv:
[1, 2, 3, 4;
5, 6, 7, 8;
9, 10, 11, 12]
\endverbatim
As a side effect, you may know that modifying one of the matrices (ie. `M` like below or `M_cv`)
will affect the content of both matrices. Let us illustrate this behavior, where the diagonal of `M` is set to 1:
\snippet tutorial-bridge-opencv-matrix.cpp Modify ViSP matrix
As expected, the content of `M_cv` matrix is also changed:
\verbatim
Set M = eye
M:
1 0 0 0
0 1 0 0
0 0 1 0
M_cv:
[1, 0, 0, 0;
0, 1, 0, 0;
0, 0, 1, 0]
\endverbatim
**/
|
