1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296
|
/**
@page smpl_handtracker_java HandTracker.java - sample program (java)
<b>Source file:</b> Click the following link to view the source code file:
- HandTracker.java\\HandTracker.java
This section describes an OpenNI sample program for tracking a hand. This sample is encapsulated in the org.OpenNI.Samples.HandTracker.jar (java archive).
The documentation describes the sample program's code from the top of the program file(s) to bottom.
Every OpenNI feature is described the first time it appears in this sample program. Further appearances of the same feature are not described again.
@section hnd_trk_class_title HandTracker Class Title
The whole program is defined inside the <code>HandTracker</code> Class.
@section htj_evhndlr_gestrec Handler for 'Gesture Recognized' Event
The @ref xn::GestureGenerator::GestureRecognized "'Gesture Recognized'" event signals that the @ref xn::GestureGenerator "GestureGenerator" node has recognized the named gesture in the scene.
The following code block defines the event handler for the 'Gesture Recognized' event.
@code
class MyGestureRecognized implements IObserver<GestureRecognizedEventArgs>
{
...
}
@endcode
The following is the main code in the body of the event handler.
The event handler calls the @ref xn::HandsGenerator::StartTracking() "StartTracking()" method of the @ref xn::HandsGenerator "HandsGenerator" node. This method causes the HandsGenerator node to start tracking at the specific position where the application expects a hand, which the application supplies as a parameter of the org.OpenNI.Point3D type. The parameter is @ref xn::XnGestureRecognized "getEndPosition()", which is the position of the hand at the end of the gesture.
@code
handsGen.StartTracking(args.getEndPosition());
gestureGen.removeGesture("Click");
@endcode
The getEndPosition() method gets the <code>pEndPosition</code> parameter from, the event handler, which is the position of the hand at the end of the gesture.
Now that the HandsGenerator is tracking a hand, the GestureGenerator does not have to look for the gesture anymore. So it does this by calling the @ref xn::GestureGenerator::RemoveGesture "RemoveGesture()" method to disable the GestureGenerator from looking for the named gesture in the FOV.
@code
gestureGen.removeGesture("Click");
@endcode
@section htj_evhndlr_handcrt Handler for 'Hand Create' Event
Some implementations may want to use the @ref xn::HandsGenerator::HandCreate "'Hand Create'" event to signal that the @ref xn::HandsGenerator "HandsGenerator" node has recognized and started tracking a new hand in response to the application calling the xn::() method. This event returns the ID of the new hand, the time of its recognition, and its position on recognition in the current frame.
The following code block defines the event handler for the 'Hand Create' event.
@code
class MyHandCreateEvent implements IObserver<ActiveHandEventArgs>
{
...
}
@endcode
The following is the main code in the body of the event handler.
The event handler calls a new list of @ref XnPoint3D "org.OpenNI.Point3D" type structs. The <code>getPosition()</code> method returns the position at which the hand was created.
@code
ArrayList<Point3D> newList = new ArrayList<Point3D>();
newList.add(args.getPosition());
@endcode
Now that the HandsGenerator is tracking the hand, it adds it to a history list created by this application. The history list is described later in @ref rec_raw_mainprg_dcl_blk_hist "declaration block".
@code
history.put(new Integer(args.getId()), newList);
@endcode
@section htj_evhndlr_handupd Handler for 'Hand Update' Event
The @ref xn::HandsGenerator::HandUpdate "'Hand Update'" event signals that the @ref xn::HandsGenerator "HandsGenerator" node signals that a currently tracked hand was recognized at a specific position in the new frame. OpenNI continues to send this event at each further frame that the hand is still present. This event returns the ID of the hand, which is the same ID returned by the HandCreate event, the hand's new position, and the time of the update.
The following code block defines the event handler for the 'Hand Update' event.
@code
class MyHandUpdateEvent implements IObserver<ActiveHandEventArgs>
{
...
}
@endcode
The following is the main code in the body of the event handler.
The assignment in the first statement adds the new position of the hand to its own history buffer. The 'history' is a hash from an integer (the hand's unique, persistent ID) and a list of its last locations.
When adding a new location of the hand, we want to add them to the correct list.
@code
ArrayList<Point3D> historyList = history.get(args.getId());
historyList.add(args.getPosition());
while (historyList.size() > historySize)
{
historyList.remove(0);
}
@endcode
The above while loop provides a maximum history size, giving a 'moving tail' effect. This means that if there are too many points in the buffer, the oldest one is removed.
@section htj_evhndlr_handdestr Handler for 'Hand Destroy' Event
The @ref xn::HandsGenerator::HandUpdate "'Hand Destroy'" event signals that an existing hand has disappeared from the frame for any reason. This event returns the user ID – still the same user ID as before – and the time that it disappeared.
The following code block defines the event handler for the 'Hand Destroy' event.
@code
MyHandDestroyEvent implements IObserver<InactiveHandEventArgs>
{
...
}
@endcode
The following code is the main code in the body of the event handler.
Now that the hand has been destroyed, in this example, the application wants to remove the user's ID from the history list. So the program gets the user's ID from the handler argument <code>args.getId</code>.
The code block below first removes the specific user's history buffer by calling <code>history.remove()</code>, and then we check if the general history is left empty, which means all hands were removed. The code then calls <code>history.isEmpty()</code> to check if the <code>history</code> listof hands has now become empty as a result of this hand having been destroyed. If it is empty, the program will then want to again start looking for the "Click" gesture. Thus, it then calls the @ref xn::GestureGenerator::AddGesture() "AddGesture()" method.
@code
history.remove(args.getId());
if (history.isEmpty())
{
try
{
gestureGen.addGesture("Click");
} catch (StatusException e)
{
e.printStackTrace();
}
}
@endcode
@section htj_mainprg_dcl_blk Main Program Declaration Block: OpenNI Nodes
The declaration block immediately above the main program (try) declares the OpenNI objects required for building the OpenNI production graph. The production graph is the main object model in OpenNI.
@code
private static final long serialVersionUID = 1L;
private OutArg<ScriptNode> scriptNode;
private Context context;
private DepthGenerator depthGen;
private GestureGenerator gestureGen;
private HandsGenerator handsGen;
@endcode
Each of these concepts is described separately in the following paragraphs.
The @ref prod_graph "production graph" is a network of software objects - called production nodes - that can identify blobs as hands or human users. In this sample program the production graph identifies blobs as human users, and tracks them as they move.
the @ref xn::ScriptNode "ScriptNode" object loads an XML script from a file or string, and then runs the XML script to build a production graph.
a @ref xn::Context "Context" object is a workspace in which the application builds an OpenNI production graph.
a @ref xn::DepthGenerator "DepthGenerator" node generates a depth map. Each map pixel value represents a distance from the sensor.
A @ref xn::GestureGenerator "GestureGenerator" node recognizes certain hand gestures and raise corresponding events accordingly. A gesture is a specific hand movement. The GestureGenerator node scans the FOV to detect gestures and generates the gesture data.
A @ref xn::HandsGenerator "HandsGenerator" node generates tracking data about a single hand or multiple hands with persistent IDs.
@section rec_raw_mainprg_dcl_blk_hist Main Program Declaration Block: Application Histogram
The following declaration block defines data structures for the application histogram. These are not OpenNI-specific declarations.
@code
private HashMap<Integer, ArrayList<Point3D>> history;
private byte[] imgbytes;
private float histogram[];
private BufferedImage bimg;
int width, height;
@endcode
<code>history</code> is a history list of @ref XnPoint3D " org.OpenNI.Point3D " objects, which are the 3D (x,y,z) co-ordinates of each depth pixel in the depth map generated by a DepthGenerator node.
<code>histogram</code> is an array of floats to build a histogram of distribution of depth values. It will be used to represent different depth values with different colors..
The following declaration for <code>SAMPLE_XML_FILE</code> is the path to an OpenNI XML script file for inputting and building a stored production graph.
private final String SAMPLE_XML_FILE = "../../../../Data/SamplesConfig.xml";
@section setup_pg HandTracker() method: Setting up the Production Graph
The public HandTracker() routine sets up the Production Graph.
The following code initialization initializes the production graph from an OpenNI XML script file The call to @ref xn::Context::InitFromXmlFile "Context.createFromXmlFile()" combines the effects of two other initialization methods – @ref xn::Context::Init() and then @ref xn::Context::RunXmlScriptFromFile() – to initialize the context object and then create a production graph. @code
scriptNode = new OutArg<ScriptNode>();
context = Context.createFromXmlFile(SAMPLE_XML_FILE, scriptNode);
@endcode
The following statements create and set up a GestureGenerator node. The addGesture() method activates the GestureGenerator node to start looking for the named gesture ("Click") in the FOV by adding the gesture's name to the node's list of gestures it actively scans for in the FOV.
@code
gestureGen = GestureGenerator.create(context);
gestureGen.addGesture("Click"); {Not exactly the same as in the API}
@endcode
The following statement sets up an event handler for the 'Gesture Recognized' event. The handler is the MyGestureRecognized() function declared elsewhere in the program sample.
@code
gestureGen.getGestureRecognizedEvent().addObserver(new MyGestureRecognized() );
@endcode
The following code block sets up event handlers for a number of hand recognition events. For details about these events, see
<a href="#xncpp_wrpr_handgen_events">Hand Event Sequence Diagram</a>, and
<a href="#xncpp_wrpr_handcreate_ev">'Hand Create'</a>,
<a href="#xncpp_wrpr_handupdate_ev">'Hand Update'</a>, and
<a href="#xncpp_wrpr_handdestroy_ev">'Hand Destroy'</a>.
@code
handsGen = HandsGenerator.create(context);
handsGen.getHandCreateEvent().addObserver(new MyHandCreateEvent());
handsGen.getHandUpdateEvent().addObserver(new MyHandUpdateEvent());
handsGen.getHandDestroyEvent().addObserver(new MyHandDestroyEvent());
@endcode
The following statement creates a @ref xn::DepthGenerator "DepthGenerator".
@code
depthGen = DepthGenerator.create(context);
@endcode
In the following statement, the latest data generated is copied to an easy-to-access @ref glos_frame_object "<i>frame object</i>". In OpenNI terminology: the node has data that is designated as 'metadata to be placed in the node's metadata object'. The node's getMetaData() method gets this data and copies it to a metadata object, <code>depthMD</code>. The data includes all configuration information associated with the data itself. This metadata object is then termed the @ref glos_frame_object "<i>frame object</i>". <br>
For more explanation on this, see @ref conc_meta_data, @ref glos_frame_object, and @ref frame_data.
@code
DepthMetaData depthMD = depthGen.getMetaData();
@endcode
The following statement ensures all created @ref dict_gen_node "generator nodes" are generating data.
@code
context.startGeneratingAll();
@endcode
The following sets up the histogram (see @ref rec_raw_mainprg_dcl_blk_hist). See
<code>getFullXRes</code> and <code>getFullYRes</code> get the number of rows and columns in the full frame (that is, the entire field-of-view), ignoring cropping. This enables the program set up the right size buffer.
@code
history = new HashMap<Integer, ArrayList<Point3D>>();
histogram = new float[10000];
width = depthMD.getFullXRes();
height = depthMD.getFullYRes();
imgbytes = new byte[width*height];
DataBufferByte dataBuffer = new DataBufferByte(imgbytes, width*height);
Raster raster = Raster.createPackedRaster(dataBuffer, width, height, 8, null);
bimg = new BufferedImage(width, height, BufferedImage.TYPE_BYTE_GRAY);
bimg.setData(raster);
@endcode
@section htj_calcHist CalcHist() - Using the Depth Values to Build an Accumulative Histogram
There are no OpenNI-specific declarations in this routine.
@section htj_update_depth_fn updateDepth() method: Updating the Depth Map
The following statement sets up the frame object. It is described in the <a href="#getMetaData">HandTracker() method above</a>. For more explanation on this, see @ref conc_meta_data, @ref glos_frame_object, and @ref frame_data.
@code
DepthMetaData depthMD = depthGen.getMetaData();
@endcode
The following method call waits for any node to have generated new data. This method then 'updates' each and every node in the entire production graph. For an overview to reading data and the <b>WaitXUpdateAll</b> methods, see <i>@ref conc_updating_data__summary_of_wait_fns</i>.
The @ref xn::Context::WaitAnyUpdateAll() "waitAnyUpdateAll()" method in the following statement waits for any node to have generated a new data frame. The method then makes the data frames of all nodes in the entire production graph available for getting. The application can then get the data (for example, using a metadata GetData() method). This method has a timeout.
@code
context.waitAnyUpdateAll();
@endcode
The following code block creates a convenient buffer for the depth map and then calls the calcHist() method to calculate the histogram. There are no OpenNI-specific operations in this code block.
@code
ShortBuffer depth = depthMD.getData().createShortBuffer();
calcHist(depth);
depth.rewind();
@endcode
This code block copies the depth into a local buffer, so that it can be drawn on the screen.
@code
while(depth.remaining() > 0)
{
int pos = depth.position();
short pixel = depth.get();
imgbytes[pos] = (byte)histogram[pixel];
}
@endcode
@section htj_getPreferredSize getPreferredSize() method
There are no OpenNI-specific operations in this routine.
@section htj_paint paint() method
This function draws the current location of each known hand, with a tail consisting of its previous points. It uses the history list populated in the HandGenerator callbacks.
*/
|