1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
|
/**
\page tutorial-franka-ibvs Tutorial: Eye-in-hand IBVS with Panda 7-dof robot from Franka Emika
\tableofcontents
\section franka_ibvs_intro 1. Introduction
This tutorial explains how to implement eye-in-hand image-based visual servoing (IBVS) with Franka Emika's Panda 7-dof
robot equipped with an Intel Realsense camera mounted on its end-effector. It follows \ref tutorial-franka-pbvs that
explains also how to setup the robot.
\section franka_ibvs 2. Eye-in-hand Image-based visual servoing
At this point we suppose that you follow all the \ref franka_prereq given in \ref tutorial-franka-pbvs.
An example of eye-in-hand image-based visual servoing using Panda robot equipped with a Realsense camera is available in
servoFrankaIBVS.cpp.
- Attach your Realsense camera to the robot end-effector
- Put an Apriltag in the camera field of view
- If not already done, follow \ref tutorial-calibration-extrinsic-eye-in-hand to estimate \f$^e{\bf M}_c\f$ the
homogeneous transformation between robot end-effector and camera frame. We suppose here that the file is located in
`apps/calibration/intrinsic/eMc.yaml`.
Now enter in `${VISP_WS}/visp-build/example/servo-franka folder` and run `servoFrankaIBVS` binary using `--eMc` to
locate the file containing the \f$^e{\bf M}_c\f$ transformation. Other options are available. Using `--help` show them:
\code{.sh}
$ cd example/servo-franka
$ ./servoFrankaIBVS --help
\endcode
Run the binary activating the plot and using a constant gain:
\code{.sh}
$ ./servoFrankaIBVS --eMc ../../apps/calibration/intrinsic/eMc.yaml --plot
\endcode
Use the left mouse click to enable the robot controller, and the right click to quit the binary.
At this point the behaviour that you should observe is the following:
\image html img-franka-ibvs-start.png Legend: Example of initial position. The goal is here to center the 4 tag corners in the image.
\image html img-franka-ibvs-converge.png Legend: Example of final position reached after position-based visual servoing. In green, you can see the trajectories in the image of the tag corners. When the camera extrinsic parameters are well estimated these trajectories are straight lines.
\image html img-franka-ibvs-converge-curves.png Legend: Corresponding visual-features (x and y coordinates of the corner points in the image plane) and velocities applied to the robot in the camera frame. You can observe an exponential decrease of the visual features.
You can also activate an adaptive gain that will make the convergence faster:
\code{.sh}
$ ./servoFrankaIBVS --eMc ../../apps/calibration/intrinsic/eMc.yaml --plot --adpative-gain
\endcode
You can also start the robot with a zero velocity at the beginning introducing task sequencing option:
\code{.sh}
$ ./servoFrankaIBVS --eMc ../../apps/calibration/intrinsic/eMc.yaml --plot --task-sequencing
\endcode
And finally you can activate the adaptive gain and task sequencing:
\code{.sh}
$ ./servoFrankaIBVS --eMc ../../apps/calibration/intrinsic/eMc.yaml --plot --adpative-gain --task-sequencing
\endcode
To learn more about adaptive gain and task sequencing see \ref tutorial-boost-vs.
\section franka_ibvs_next 3. Next tutorial
If you want to achieve a physical simulation of a Franka robot, with a model that has been accurately identified
from a real Franka robot, like in the next video, we recommend to make a tour on \ref tutorial-franka-sim that is
available in [visp_ros](http://wiki.ros.org/visp_ros). Here you will find a ROS package that allows to implement
position, velocity and impedance control of a simulated Franka robot using ROS and
[CoppeliaSim](https://www.coppeliarobotics.com/).
\htmlonly
<p align="center"><iframe width="560" height="315" src="https://www.youtube.com/embed/02Bx093Fuak" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></p>
\endhtmlonly
You can also follow \ref tutorial-ibvs that will give some hints on image-based visual servoing in simulation with a
free flying camera.
*/
|
