File: tutorial-tracking-mb-generic-rgbd.dox

package info (click to toggle)
visp 3.7.0-10
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid
  • size: 166,384 kB
  • sloc: cpp: 392,705; ansic: 224,448; xml: 23,444; python: 13,701; java: 4,792; sh: 207; objc: 145; makefile: 118
file content (631 lines) | stat: -rw-r--r-- 30,698 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
/**

\page tutorial-tracking-mb-generic-rgbd Tutorial: Markerless generic model-based tracking using a RGB-D camera
\tableofcontents

\section mb_rgbd_intro Introduction

This tutorial describes how to use the generic markerless model-based tracker \cite Comport06b, \cite Trinh18a
implemented in vpMbGenericTracker with data acquired by a RGB-D camera like the Intel RealSense devices (SR300
or D400 series) or Occipital Structure devices (Structure Core color or monochrome cameras). You are advised to have
read the \ref tutorial-tracking-mb-generic to have an overview of the generic model-based tracker working on images
acquired by a color camera that uses moving-edges and keypoints as visual features. We suggest also to follow
\ref tutorial-tracking-mb-generic-stereo to learn how to consider stereo cameras, since a RGB-D camera is in a sense
a stereo camera, with the left camera providing the color image and the right the depth image.

Note that all the material (source code, input video, CAD model or XML settings files) described in this tutorial is
part of ViSP source code (in `tracking/model-based/generic-rgbd` folder) and could be found in
https://github.com/lagadic/visp/tree/master/tracking/model-based/generic-rgbd.

\subsection mb_rgbd_intro_features_overview Features overview

Considering the use case of a RGB-D camera, the tracker implemented in vpMbGenericTracker class allows to consider a
combination of the following visual features:
- moving edges: contour points tracked along the normals of the visible edges defined in the CAD model (line, face,
  cylinder and circle primitives) \cite Comport06b
- keypoints: they are tracked on the visible object faces using the KLT tracker (face and cylinder primitives)
  \cite Pressigout:2007
- depth normal: the normal of a face computed from the depth map (face primitives) (since ViSP 3.1.0)
- depth dense: the dense pointcloud obtained from the depth map (face primitives) (since ViSP 3.1.0) \cite Trinh18a.

The moving-edges and KLT features require a RGB camera, more precisely these features operate on grayscale image. The
depth normal and dense depth features require a depth map that can be obtained from a RGB-D camera for example.

If you want to use a depth feature, we advise you to choose the dense depth features that is a much more robust method
compared to the depth normal feature. Keep also in mind that Kinect-like sensors have a limited depth range
(from ~0.8m to ~3.5m).

Note also that combining the visual features can be a good way to improve the tracking robustness (e.g. a stereo
tracker with moving edges + keypoints for the left camera and dense depth visual features for the right camera).

The following video demonstrates the use of the generic tracker with a RGB-D sensor using the following features:
- moving-edges + keypoints features for the color camera
- depth normal features for the depth sensor

\htmlonly
<p align="center"><iframe width="640" height="360" src="https://www.youtube.com/embed/4FARYLYzNL8" frameborder="0" allowfullscreen></iframe></p>
\endhtmlonly

In this video, the same setup is used but with:
- moving-edges features for the color camera
- dense depth features for the depth sensor

\htmlonly
<p align="center"><iframe width="640" height="360" src="https://www.youtube.com/embed/LFej9NF6F1A" frameborder="0" allowfullscreen></iframe></p>
\endhtmlonly

\subsection mb_rgbd_intro_3rd_parties Considered third-parties

We recall that this tutorial is designed to work with either Intel Realsense or Occipital RGB-D cameras.
If you want to run the use cases below, depending on your RGB-D device, you need to install the appropriate SDK as
described in the following tutorials (see section "librealsense 3rd party" or "libStructure 3rd party" respectively):
  - \ref tutorial-install-ubuntu
  - \ref tutorial-install-win10-msvc16
  - \ref tutorial-install-osx-homebrew

Moreover, depending on your use case the following optional third-parties may be used by the tracker. Make sure ViSP
was build with the appropriate 3rd parties:
- \c OpenCV: Essential if you want to use the keypoints as visual features that are detected and tracked thanks to the
  KLT tracker. This 3rd party may be also useful to consider input videos (mpeg, png, jpeg...).
- \c Point Cloud Library: This 3rd party is optional but could be used if your RGB-D sensor provides the depth data as
  a point cloud. Notice that the source code explained in this tutorial doesn't run if you don't have a version of ViSP
  build with PCL as 3rd party. Note also that you don't need to install PCL if you don't want to consider depth as visual
  feature.
- \c Ogre \c 3D: This 3rd party is optional and could be difficult to install on OSX and Windows. To begin with the
  tracker we don't recommend to install it. \c Ogre \c 3D allows to enable \ref mb_generic_settings_visibility_ogre.
- \c Coin \c 3D: This 3rd party is also optional and difficult to install. That's why we don't recommend to install
  \c Coin \c 3D to begin with the tracker. \c Coin \c 3D allows only to consider \ref mb_generic_advanced_wrml
  instead of the home-made \ref mb_generic_advanced_cao.

It is recommended to install an optimized BLAS library (for instance <a href="http://www.openblas.net/">OpenBLAS</a>)
to get better performance with the dense depth feature approach which requires large matrix operations.
On Ubuntu Xenial, it is the <a href="https://packages.ubuntu.com/xenial/libopenblas-dev">libopenblas-dev</a> package
that should be installed.
To select or switch the BLAS library to use, see
<a href="https://wiki.debian.org/DebianScience/LinearAlgebraLibraries">Handle different versions of BLAS and LAPACK</a>:
\code
$ update-alternatives --config libblas.so.3
\endcode
You should have something similar to this:
<blockquote>
There are 3 choices for the alternative libblas.so.3 (providing /usr/lib/libblas.so.3).
<table>
  <tr>
    <th>Selection</th>
    <th>Path</th>
    <th>Priority</th>
    <th>Status</th>
  </tr>
  <tr>
    <td>0</td>
    <td>/usr/lib/openblas-base/libblas.so.3</td>
    <td>40</td>
    <td>auto mode</td>
  </tr>
  <tr>
    <td>1</td>
    <td>/usr/lib/atlas-base/atlas/libblas.so.3</td>
    <td>35</td>
    <td>manual mode</td>
  </tr>
  <tr>
    <td>2</td>
    <td>/usr/lib/libblas/libblas.so.3</td>
    <td>10</td>
    <td>manual mode</td>
  </tr>
  <tr>
    <td>3</td>
    <td>/usr/lib/openblas-base/libblas.so.3</td>
    <td>40</td>
    <td>manual mode</td>
  </tr>
</table>
</blockquote>

\subsection mb_rgbd_input Input data

For classical features working on grayscale image, you have to use the following data type:

\code
vpImage<unsigned char> I;
\endcode

You can convert to a grayscale image if the image acquired is in RGBa data type:

\code
vpImage<vpRGBa> I_color;
// Color image acquisition
vpImage<unsigned char> I;
vpImageConvert::convert(I_color, I);
\endcode

For depth features, you need to supply a pointcloud, that means a 2D array where each element contains the X, Y and Z
coordinate in the depth sensor frame. The following data type are accepted:
- a vector of vpColVector: the column vector being a `3x1` (X, Y, Z) vector
\code
std::vector<vpColVector> pointcloud(640*480);
// Fill the pointcloud
vpColVector coordinate(3);
coordinate[0] = X;
coordinate[1] = Y;
coordinate[2] = Z;
pointcloud[0] = coordinate;
\endcode

- a <a href="http://docs.pointclouds.org/trunk/classpcl_1_1_point_cloud.html">PCL pointcloud</a> smart pointer of
<a href="http://docs.pointclouds.org/trunk/structpcl_1_1_point_x_y_z.html">pcl::PointXYZ</a> data type:
\code
pcl::PointCloud<pcl::PointXYZ>::Ptr pointcloud;
\endcode

If you have only a depth map, a 2D array where each element (u, v) is the distance Z in meter between the depth sensor
frame to the object, you will need to compute the 3D coordinates using the depth sensor intrinsic parameters. We recall
that u relates to the array columns, while v relates to the array rows.
For instance, without taking into account the distortion (see also vpPixelMeterConversion::convertPoint),
the conversion is (u and v are the pixel coordinates, u0, v0, px, py are the intrinsic parameters):

\f[
\left\{\begin{matrix}
X = \frac{u - u_0}{px} \times Z \\
Y = \frac{v - v_0}{px} \times Z
\end{matrix}\right.
\f]

Here an example of a depth map:

\image html tutorial-tracking-mb-generic-depth-map.png

\section mb_rgbd_started Getting started

\subsection mb_rgbd_implementation_detail Implementation detail

\note
This section is similar to the \ref mb_generic_stereo_implementation_detail section in
\ref tutorial-tracking-mb-generic-stereo.

Each tracker is stored in a map, the key corresponding to the name of the camera on which the tracker will process.
By default, the camera names are set to:
-  "Camera" when the tracker is constructed with one camera.
-  "Camera1" to "CameraN" when the tracker is constructed with N cameras.
-  The default reference camera will be "Camera1" in the multiple cameras case.

\image html img-multi-cameras-config.png Default name convention and reference camera ("Camera1").

To deal with multiple cameras, in the virtual visual servoing control law we concatenate all the interaction matrices
and residual vectors and transform them in a single reference camera frame to compute the reference camera velocity.
Thus, we have to know the transformation matrix between each camera and the reference camera.

For example, if the reference camera is "Camera1" (\f$ c_1 \f$), we need the following information:
\f$ _{}^{c_2}\textrm{M}_{c_1}, _{}^{c_3}\textrm{M}_{c_1}, \cdots, _{}^{c_n}\textrm{M}_{c_1} \f$.

\subsection mb_rgbd_choose_features Declare a model-based tracker of the desired feature type

The following enumeration (vpMbGenericTracker::vpTrackerType) values are available to get the desired model-based tracker:
- moving-edges feature: vpMbGenericTracker::EDGE_TRACKER
- KLT feature: vpMbGenericTracker::KLT_TRACKER
- depth normal feature: vpMbGenericTracker::DEPTH_NORMAL_TRACKER
- depth dense feature: vpMbGenericTracker::DEPTH_DENSE_TRACKER

The tracker declaration is then:
\code
vpMbGenericTracker tracker(vpMbGenericTracker::EDGE_TRACKER);
\endcode

To combine the features:
\code
vpMbGenericTracker tracker(vpMbGenericTracker::EDGE_TRACKER | vpMbGenericTracker::KLT_TRACKER);
\endcode

To "fully exploit" the capabilities of a RGB-D sensor, you can use for instance:

\code
std::vector<int> trackerTypes(2);
trackerTypes[0] = vpMbGenericTracker::EDGE_TRACKER | vpMbGenericTracker::KLT_TRACKER;
trackerTypes[1] = vpMbGenericTracker::DEPTH_DENSE_TRACKER;
vpMbGenericTracker tracker(trackerTypes);
\endcode

This will declare a tracker with edge + KLT features for the color camera and dense depth feature for the depth sensor.

\subsection mb_rgbd_interface_with_the_code Interfacing with the code

Each essential method used to initialize the tracker and process the tracking have three signatures in order to ease
the call to the method and according to three working modes:
- tracking using one camera, the signature remains the same.
- tracking using two cameras, all the necessary methods accept directly the corresponding parameters for each camera.
- tracking using multiple cameras, you have to supply the different parameters within a map. The key corresponds to
  the name of the camera and the value is the parameter.

The following table sums up how to call the different methods based on the camera configuration for the main functions.

<table>
<caption id="mb_rgbd_method_example_table">Example of the different method signatures.</caption>
<tr><th>Method calling example:               <th>Monocular case                        <th>Stereo case                                                       <th>Multiple cameras case                                             <th>Remarks
<tr><td>Construct a model-based edge tracker: <td>vpMbGenericTracker tracker            <td>vpMbGenericTracker tracker(2, vpMbGenericTracker::EDGE_TRACKER)   <td>vpMbGenericTracker tracker(5, vpMbGenericTracker::EDGE_TRACKER)   <td>The default constructor corresponds to a monocular edge tracker.
<tr><td>Load a configuration file:            <td>tracker.loadConfigFile("config.xml")  <td>tracker.loadConfigFile("config1.xml", "config2.xml")              <td>tracker.loadConfigFile(mapOfConfigFiles)                          <td>Each tracker can have different parameters (intrinsic parameters, visibility angles, etc.).
<tr><td>Load a model file:                    <td>tracker.loadModel("model.cao")        <td>tracker.loadModel("model1.cao", "model2.cao")                     <td>tracker.loadModel(mapOfModelFiles)                                <td>Different models can be used.
<tr><td>Get the intrinsic camera parameters:  <td>tracker.getCameraParameters(cam)      <td>tracker.getCameraParameters(cam1, cam2)                           <td>tracker.getCameraParameters(mapOfCam)                             <td>
<tr><td>Set the transformation matrix between each camera and the reference one: <td>   <td>tracker.setCameraTransformationMatrix(mapOfCamTrans)              <td>tracker.setCameraTransformationMatrix(mapOfCamTrans)              <td>For the reference camera, the identity homogeneous matrix must be used.
<tr><td>Setting to display the features:      <td>tracker.setDisplayFeatures(true)      <td>tracker.setDisplayFeatures(true)                                  <td>tracker.setDisplayFeatures(true)                                  <td>This is a general parameter.
<tr><td>Initialize the pose by click:         <td>tracker.initClick(I, "f_init.init")   <td>tracker.initClick(I1, I2, "f_init1.init", "f_init2.init")         <td>tracker.initClick(mapOfImg, mapOfInitFiles)                       <td>Assuming the transformation matrices between the cameras have been set, some init files can be omitted as long as the reference camera has an init file. The corresponding images must be supplied.
<tr><td>Track the object:                     <td>tracker.track(I)                      <td>tracker.track(I1, I2)                                             <td>tracker.track(mapOfImg)                                           <td>See the documentation to track with pointcloud.
<tr><td>Get the pose:                         <td>tracker.getPose(cMo)                  <td>tracker.getPose(c1Mo, c2Mo)                                       <td>tracker.getPose(mapOfPoses)                                       <td>tracker.getPose(cMo) will return the pose for the reference camera in the stereo/multiple cameras configurations.
<tr><td>Display the model:                    <td>tracker.display(I, cMo, cam, ...)     <td>tracker.display(I1, I2, c1Mo, c2Mo, cam1, cam2, ...)              <td>tracker.display(mapOfImg, mapOfPoses, mapOfCam, ...)              <td>
</table>

\note As the trackers are stored in an alphabetic order internally, you have to match the method parameters with the correct
tracker position in the map in the stereo cameras case.

\section mb_rgbd_example Dataset use case

\subsection mb_rgbd_example_code Cube tracking on a RGB-D dataset

We provide hereafter an example implemented in tutorial-mb-generic-tracker-rgbd.cpp that shows how to track a cube modeled in cao format using moving-eges, keypoints and dense depth as visual features. The input data (color image, depth map and point clould) were acquired by a RealSense RGB-D camera.

\warning
ViSP must have been built with OpenCV and the KLT module must be enabled to run this tutorial. You need also the Point Cloud Library enabled.

Once built, run the binary:
\code
$ ./tutorial-mb-generic-tracker-rgbd
\endcode

After initialization by 4 user clicks, the tracker is running and produces the following output:
\image html img-mbt-rgbd-cube.png

The source code that is very similar to the one provided in \ref tutorial-tracking-mb-generic-stereo is the following:
\include tutorial-mb-generic-tracker-rgbd.cpp

\subsection mb_rgbd_explanation_of_the_code Source code explained

The previous source code shows how to use the vpMbGenericTracker class using depth as visual feature. The procedure to
configure the tracker remains the same:
- construct the tracker enabling moving-edges, keypoints and depth as visual feature
- configure the tracker by loading a configuration file (`cube.xml`) for the color camera and for the depth camera
  (`cube_depth.xml`). These files contain the camera intrinsic parameters associated to each sensor
- load the transformation matrix (`depth_M_color.txt`) between the depth sensor and the color camera
- load the `cube.init` file used to initialize the tracker by a user interaction that consists in clicking on 4
  cube corners
- load a 3D model for the color and for the depth camera. In our case they are the same (`cube.cao`)
- start a while loop to process all the images from the sequence of data
  - track on the current images (color and depth)
  - get the current pose and display the model in the image

Please refer to the tutorial \ref tutorial-tracking-mb-generic in order to have explanations about the configuration
parameters (\ref mb_generic_settings) and how to model an object in a ViSP compatible format
(\ref mb_generic_advanced_cao).

To use vpMbGenericTracker we include first the corresponding header:

\snippet tutorial-mb-generic-tracker-rgbd.cpp Include

Then we implemented the function `read_data()` to read the color images `color_image_%04d.jpg`, the depth map
`depth_image_%04d.bin` and the point cloud `point_cloud_%04d.bin`.

\snippet tutorial-mb-generic-tracker-rgbd.cpp Read data function

We need grayscale images for the tracking, one corresponding to the color image, the other for the depth image:

\snippet tutorial-mb-generic-tracker-rgbd.cpp Images

We need also a point cloud container that will be updated from the depth map

\snippet tutorial-mb-generic-tracker-rgbd.cpp Point cloud

To set the type of the tracker (the first parameter is the number of cameras, the second parameter is the type of the
tracker(s)):

\snippet tutorial-mb-generic-tracker-rgbd.cpp Constructor

To load the configuration file, we use:

\snippet tutorial-mb-generic-tracker-rgbd.cpp Load config file

To load the 3D object model, we use:

\snippet tutorial-mb-generic-tracker-rgbd.cpp Load cao

We can also use the following setting that enables the display of the features used during the tracking:

\snippet tutorial-mb-generic-tracker-rgbd.cpp Set display features

We then set the map of transformations between our two sensors indicating that the color sensor frame is the reference
frame. `depth_M_color` matrix is read from the input file `depth_M_color.txt`.

\snippet tutorial-mb-generic-tracker-rgbd.cpp Map transformations

We set also the map of images

\snippet tutorial-mb-generic-tracker-rgbd.cpp Map images

The initial pose is set by clicking on specific points in the image after setting a map of `*.init` files
(in our case `cube.init`):

\snippet tutorial-mb-generic-tracker-rgbd.cpp Map init

The tracking is done by:

\snippet tutorial-mb-generic-tracker-rgbd.cpp Track

To get the current camera pose:

\snippet tutorial-mb-generic-tracker-rgbd.cpp Get pose

To display the model with the estimated pose and also the object frame:

\snippet tutorial-mb-generic-tracker-rgbd.cpp Display

\section mb_rgbd_use_case_comparison Comparison between RGB-D cameras

This video shows a comparison between Intel Realsense D435 and Occipital Structure Core. In the left column we show
data acquired using D435, while on the right column they are from the Structure Core color depth camera.

\htmlonly
<p align="center">
<iframe width="560" height="315" src="https://www.youtube.com/embed/ddUNX-hkwqg" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</p>
\endhtmlonly

\section mb_rgbd_use_case Intel Realsense camera use case

Hereafter we provide the information to test the tracker with different objects (cube, teabox, lego-square) with an
Intel Realsense D435 or SR300 RGB-D camera. Here we suppose that librealsense 3rd party is installed, and that ViSP
is build with librealsense support as explained in the following tutorials:
  - \ref tutorial-install-ubuntu
  - \ref tutorial-install-win10-msvc16
  - \ref tutorial-install-osx-homebrew

Enter model-based tracker tutorial build folder:
\code
$ cd $VISP_WS/visp-build/tutorial/tracking/model-based/generic-rgbd
\endcode

There is `tutorial-mb-generic-tracker-rgbd-realsense` binary that corresponds to the build of
tutorial-mb-generic-tracker-rgbd-realsense.cpp example. This example is an extension of
tutorial-mb-generic-tracker-rgbd.cpp that was explained in \ref mb_rgbd_started section.

To see the command line options that are available run:
\code
$ ./tutorial-mb-generic-tracker-rgbd-realsense --help
\endcode

\subsection mb_rgbd_use_case_cube Tracking a 4.2 cm cube

\note Here we consider a 4.2 cm cube. If your cube size differ, modify `model/cube/cube.cao` file replacing `0.042`
with the appropriate value expressed in meters.

To track a 4.2 cm cube with manual initialization, run:
\code
$ ./tutorial-mb-generic-tracker-rgbd-realsense --model_color model/cube/cube.cao
\endcode

You should obtain similar results:

\htmlonly
<p align="center"><iframe width="560" height="315" src="https://www.youtube.com/embed/1pRp-7yO09E" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></p>
\endhtmlonly

Now if the tracker is working, you can learn the object adding `--learn` command line option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-realsense --model_color model/cube/cube.cao --learn
\endcode

Once initialized by 4 user clicks, use the left click to learn on one or two images and then the right click to quit.
In the terminal you should see printings like:
\code
...
Data learned
Data learned
Save learning from 2 images in file: learning/data-learned.bin
\endcode

You can now use this learning to automize tracker initialization with `--auto_init` option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-realsense --model_color model/cube/cube.cao --auto_init
\endcode

\subsection mb_rgbd_use_case_teabox Tracking a teabox

\note Here we consider a 16.5 x 6.8 x 8 cm box [length x width x height]. If your cube size differ, modify
`model/cube/cube.cao` with the appropriate size values expressed in meters.

To track the 16.5 x 6.8 x 8 cm teabox with manual initialization, run:
\code
$ ./tutorial-mb-generic-tracker-rgbd-realsense --model_color model/teabox/teabox.cao
\endcode

You should obtain similar results:

\htmlonly
<p align="center"><iframe width="560" height="315" src="https://www.youtube.com/embed/SwjWFM7srOA" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></p>
\endhtmlonly

Now if the tracker is working, you can learn the object adding `--learn` command line option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-realsense --model_color model/teabox/teabox.cao --learn
\endcode

Once initialized by 4 user clicks, use the left click to learn on one or two images and then the right click to quit.
In the terminal you should see printings like:
\code
...
Data learned
Data learned
Save learning from 2 images in file: learning/data-learned.bin
\endcode

You can now use this learning to automize tracker initialization with `--auto_init` option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-realsense --model_color model/teabox/teabox.cao --auto_init
\endcode

\subsection mb_rgbd_use_case_lego_Square Tracking a lego square

Here we consider an object made with four 2x4 lego bricks assembled as a square. To track this object with manual
initialization, run:
\code
$ ./tutorial-mb-generic-tracker-rgbd-realsense --model_color model/lego-square/lego-square.cao
\endcode

You should obtain similar results:

\htmlonly
<p align="center"><iframe width="560" height="315" src="https://www.youtube.com/embed/zD2nrl5yqxA" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></p>
\endhtmlonly

Now if the tracker is working, you can learn the object adding `--learn` command line option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-realsense --model_color model/lego-square/lego-square.cao --learn
\endcode

Once initialized by 4 user clicks, use the left click to learn on one or two images and then the right click to quit.
In the terminal you should see printings like:
\code
...
Data learned
Data learned
Save learning from 2 images in file: learning/data-learned.bin
\endcode

You can now use this learning to automize tracker initialization with `--auto_init` option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-realsense --model_color model/lego-square/lego-square.cao --auto_init
\endcode

\section mb_rgbd_use_case_occipital Occipital Structure Core camera use case

Hereafter we provide the information to test the tracker with different objects (cube, teabox, lego-square) with an
Occipital Structure Core color or monochrome RGB-D camera. Here we suppose that libStructure 3rd party is installed,
and that ViSP is build with Occiptal Structure support as explained in the following tutorials:
  - \ref tutorial-install-ubuntu
  - \ref tutorial-install-win10-msvc16
  - \ref tutorial-install-osx-homebrew

Enter model-based tracker tutorial build folder:
\code
$ cd $VISP_WS/visp-build/tutorial/tracking/model-based/generic-rgbd
\endcode

There is `tutorial-mb-generic-tracker-rgbd-structure-core` binary that corresponds to the build of
tutorial-mb-generic-tracker-rgbd-structure-core.cpp example. This example is an extension of
tutorial-mb-generic-tracker-rgbd.cpp that was explained in \ref mb_rgbd_started section.

To see the command line options that are available run:
\code
$ ./tutorial-mb-generic-tracker-rgbd-structure-core --help
\endcode

\subsection mb_rgbd_use_case_occipital_cube Tracking a 4.2 cm cube

\note Here we consider a 4.2 cm cube. If your cube size differ, modify `model/cube/cube.cao` file replacing `0.042`
with the appropriate value expressed in meters.

To track a 4.2 cm cube with manual initialization, run:
\code
$ ./tutorial-mb-generic-tracker-rgbd-structure-core --model_color model/cube/cube.cao
\endcode

You should obtain similar results:

\htmlonly
<p align="center"><iframe width="560" height="315" src="https://www.youtube.com/embed/1pRp-7yO09E" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></p>
\endhtmlonly

Now if the tracker is working, you can learn the object adding `--learn` command line option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-strcuture-core --model_color model/cube/cube.cao --learn
\endcode

Once initialized by 4 user clicks, use the left click to learn on one or two images and then the right click to quit.
In the terminal you should see printings like:
\code
...
Data learned
Data learned
Save learning from 2 images in file: learning/data-learned.bin
\endcode

You can now use this learning to automize tracker initialization with `--auto_init` option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-rstructure-core --model_color model/cube/cube.cao --auto_init
\endcode

\subsection mb_rgbd_use_case_occipital_teabox Tracking a teabox

\note Here we consider a 16.5 x 6.8 x 8 cm box [length x width x height]. If your cube size differ, modify
`model/cube/cube.cao` with the appropriate size values expressed in meters.

To track the 16.5 x 6.8 x 8 cm teabox with manual initialization, run:
\code
$ ./tutorial-mb-generic-tracker-rgbd-structure-core --model_color model/teabox/teabox.cao
\endcode

You should obtain similar results:

\htmlonly
<p align="center"><iframe width="560" height="315" src="https://www.youtube.com/embed/SwjWFM7srOA" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></p>
\endhtmlonly

Now if the tracker is working, you can learn the object adding `--learn` command line option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-structure-core --model_color model/teabox/teabox.cao --learn
\endcode

Once initialized by 4 user clicks, use the left click to learn on one or two images and then the right click to quit.
In the terminal you should see printings like:
\code
...
Data learned
Data learned
Save learning from 2 images in file: learning/data-learned.bin
\endcode

You can now use this learning to automize tracker initialization with `--auto_init` option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-structure-core --model_color model/teabox/teabox.cao --auto_init
\endcode

\subsection mb_rgbd_use_case_occipital_lego_Square Tracking a lego square

Here we consider an object made with four 2x4 lego bricks assembled as a square. To track this object with manual
initialization, run:
\code
$ ./tutorial-mb-generic-tracker-rgbd-structure-core --model_color model/lego-square/lego-square.cao
\endcode

You should obtain similar results:

\htmlonly
<p align="center"><iframe width="560" height="315" src="https://www.youtube.com/embed/zD2nrl5yqxA" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></p>
\endhtmlonly

Now if the tracker is working, you can learn the object adding `--learn` command line option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-structure-core --model_color model/lego-square/lego-square.cao --learn
\endcode

Once initialized by 4 user clicks, use the left click to learn on one or two images and then the right click to quit.
In the terminal you should see printings like:
\code
...
Data learned
Data learned
Save learning from 2 images in file: learning/data-learned.bin
\endcode

You can now use this learning to automize tracker initialization with `--auto_init` option:
\code
$ ./tutorial-mb-generic-tracker-rgbd-structure-core --model_color model/lego-square/lego-square.cao --auto_init
\endcode

\section mb_rgbd_next Next tutorial

- In this tutorial, tracker settings were loaded from an XML file. To learn how to use rather a JSON configuration file,
  you can follow \ref tutorial-mb-generic-json.
- If you have an Intel Realsense device, you are also ready to experiment the generic model-based tracker
  on a cube that has an AprilTag on one face following \ref tutorial-tracking-mb-generic-apriltag-live.
- You can also follow \ref tutorial-tracking-mb-CAO-editor.
- Since ViSP 2.7.0 we introduce a new tracker named RBT that allows to track more complex objects.
  To learn more about it you may follow \ref tutorial-tracking-rbt.
*/