File: SampleProgs_HandTracker_java.txt

package info (click to toggle)
openni 1.5.4.0%2Bdfsg-8
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid, trixie
  • size: 45,580 kB
  • sloc: cpp: 116,706; ansic: 58,807; sh: 10,287; cs: 7,698; java: 7,402; python: 1,547; makefile: 492; xml: 167
file content (296 lines) | stat: -rw-r--r-- 15,629 bytes parent folder | download | duplicates (7)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
/**
@page smpl_handtracker_java HandTracker.java - sample program (java)

	<b>Source file:</b> Click the following link to view the source code file:
		-  HandTracker.java\\HandTracker.java  

	This section describes an OpenNI sample program for tracking a hand. This sample is encapsulated in the org.OpenNI.Samples.HandTracker.jar (java archive).
		
	The documentation describes the sample program's code from the top of the program file(s) to bottom.
	
	Every OpenNI feature is described the first time it appears in this sample program. Further appearances of the same feature are not described again.	
	
	@section hnd_trk_class_title HandTracker Class Title
	
		The whole program is defined inside the <code>HandTracker</code> Class. 
		
		
	@section htj_evhndlr_gestrec  Handler for 'Gesture Recognized' Event
	
		The @ref xn::GestureGenerator::GestureRecognized "'Gesture Recognized'" event signals that the @ref xn::GestureGenerator "GestureGenerator" node has recognized the named gesture in the scene. 
		
		The following code block defines the event handler for the 'Gesture Recognized' event.
		@code	
			class MyGestureRecognized implements IObserver<GestureRecognizedEventArgs>
			{
				...
			}
		@endcode	
		
		The following is the main code in the body of the event handler.
		
			The event handler calls the @ref xn::HandsGenerator::StartTracking()  "StartTracking()" method of the @ref xn::HandsGenerator "HandsGenerator" node. This method causes the HandsGenerator node to start tracking at the specific position where the application expects a hand, which the application supplies as a parameter of the org.OpenNI.Point3D type. The parameter is @ref xn::XnGestureRecognized "getEndPosition()", which is the position of the hand at the end of the gesture.
			@code	
				handsGen.StartTracking(args.getEndPosition());
				gestureGen.removeGesture("Click");
			@endcode	

			The getEndPosition() method gets the <code>pEndPosition</code> parameter from, the event handler, which is the position of the hand at the end of the gesture.
			
			Now that the HandsGenerator is tracking a hand, the GestureGenerator does not have to look for the gesture anymore. So it does this by calling the @ref  xn::GestureGenerator::RemoveGesture "RemoveGesture()" method to disable the GestureGenerator from looking for the named gesture in the FOV.
			@code	
				gestureGen.removeGesture("Click");
			@endcode			
			
			
	@section htj_evhndlr_handcrt Handler for 'Hand Create' Event
	
		Some implementations may want to use the @ref xn::HandsGenerator::HandCreate "'Hand Create'" event to signal that the @ref xn::HandsGenerator "HandsGenerator" node has recognized and started tracking a new hand in response to the application calling the xn::() method. This event returns the ID of the new hand, the time of its recognition, and its position on recognition in the current frame.	
		
		The following code block defines the event handler for the 'Hand Create' event.
		@code	
			class MyHandCreateEvent implements IObserver<ActiveHandEventArgs>
			{
				...
			}
		@endcode	
		
		The following is the main code in the body of the event handler.
		
			The event handler calls a new list of @ref XnPoint3D "org.OpenNI.Point3D" type structs. The <code>getPosition()</code> method returns the position at which the hand was created.
			@code	
				ArrayList<Point3D> newList = new ArrayList<Point3D>();
				newList.add(args.getPosition());
			@endcode	
			
			Now that the HandsGenerator is tracking the hand, it adds it to a history list created by this application. The history list is described later in @ref rec_raw_mainprg_dcl_blk_hist "declaration block".
			@code	
				history.put(new Integer(args.getId()), newList);
			@endcode						
			
			
	@section htj_evhndlr_handupd Handler for 'Hand Update' Event
	
		The @ref xn::HandsGenerator::HandUpdate "'Hand Update'" event signals that the @ref xn::HandsGenerator "HandsGenerator" node signals that a currently tracked hand was recognized at a specific position in the new frame. OpenNI continues to send this event at each further frame that the hand is still present. This event returns the ID of the hand, which is the same ID returned by the HandCreate event, the hand's new position, and the time of the update.
		
		The following code block defines the event handler for the 'Hand Update' event.
		@code	
			class MyHandUpdateEvent implements IObserver<ActiveHandEventArgs>
			{
				...
			}
		@endcode	
		
		The following is the main code in the body of the event handler.
		The assignment in the first statement adds the new position of the hand to its own history buffer. The 'history' is a hash from an integer (the hand's unique, persistent ID) and a list of its last locations.
		
		When adding a new location of the hand, we want to add them to the correct list.
		@code	
			ArrayList<Point3D> historyList = history.get(args.getId());
			
			historyList.add(args.getPosition());
			
			while (historyList.size() > historySize)
			{
				historyList.remove(0);
			}
		@endcode	

		The above while loop provides a maximum history size,  giving a 'moving tail' effect. This means that if there are too many points in the buffer, the oldest one is removed.
		
		
@section htj_evhndlr_handdestr Handler for 'Hand Destroy' Event
	
		The @ref xn::HandsGenerator::HandUpdate "'Hand Destroy'" event signals that an existing hand has disappeared from the frame for any reason. This event returns the user ID &ndash; still the same user ID as before &ndash; and the time that it disappeared.
		
		The following code block defines the event handler for the 'Hand Destroy' event.
		@code	
			MyHandDestroyEvent implements IObserver<InactiveHandEventArgs>
			{
				...
			}
		@endcode	
		
		The following code is the main code in the body of the event handler.
		
		Now that the hand has been destroyed, in this example, the application wants to remove the user's ID from the history list. So the program gets the user's ID from the handler argument <code>args.getId</code>.


		The code block below first removes the specific user's history buffer by calling <code>history.remove()</code>, and then we check if the general history is left empty, which means all hands were removed. The code then calls <code>history.isEmpty()</code> to check if the <code>history</code> listof hands has now become empty as a result of this hand having been destroyed. If it is empty, the program will then want to again start looking for the "Click" gesture. Thus, it then calls the @ref xn::GestureGenerator::AddGesture() "AddGesture()" method.
		@code	
			history.remove(args.getId());
			if (history.isEmpty())
			{
				try
				{
					gestureGen.addGesture("Click");
				} catch (StatusException e)
				{
					e.printStackTrace();
				}
			}
		@endcode			


	@section htj_mainprg_dcl_blk Main Program Declaration Block: OpenNI Nodes
	
		The declaration block immediately above the main program (try) declares the OpenNI objects required for building the OpenNI production graph. The production graph is the main object model in OpenNI. 
		@code	
			private static final long serialVersionUID = 1L;
			private OutArg<ScriptNode> scriptNode;
			private Context context;
			private DepthGenerator depthGen;
			private GestureGenerator gestureGen;
			private HandsGenerator handsGen;
		@endcode
		
		Each of these concepts is described separately in the following paragraphs.	
	
		The @ref prod_graph "production graph" is a network of software objects - called production 	nodes - that can identify blobs as hands or human users. In this sample program the production graph identifies blobs as human users, and tracks them as they move. 
		
		the @ref xn::ScriptNode "ScriptNode" object loads an XML script from a file or string, and then runs the XML script to build a production graph.		

		a @ref xn::Context "Context" object is a workspace in which the application builds an OpenNI production graph. 	

		a @ref xn::DepthGenerator "DepthGenerator" node generates a depth map. Each map pixel value represents a distance from the sensor. 				

		A @ref xn::GestureGenerator "GestureGenerator" node recognizes certain hand gestures and raise corresponding events accordingly. A gesture is a specific hand movement. The GestureGenerator node scans the FOV to detect gestures and generates the gesture data. 			
		
		A @ref xn::HandsGenerator "HandsGenerator" node generates tracking data about a single hand or multiple hands with persistent IDs. 	
		
		
		@section rec_raw_mainprg_dcl_blk_hist Main Program Declaration Block: Application Histogram
		
		The following declaration block defines data structures for the application histogram. These are not OpenNI-specific declarations.
		@code
			private HashMap<Integer, ArrayList<Point3D>> history;
			private byte[] imgbytes;
			private float histogram[];

			private BufferedImage bimg;
			int width, height;
		@endcode
		<code>history</code> is a history list of @ref XnPoint3D " org.OpenNI.Point3D " objects, which are the 3D (x,y,z) co-ordinates of each depth pixel in the depth map generated by a DepthGenerator node. 
		
		<code>histogram</code> is an array of floats to build a histogram of distribution of depth values. It will be used to represent different depth values with different colors..
		
		The following declaration for <code>SAMPLE_XML_FILE</code> is the path to an OpenNI XML script file for inputting and building a stored production graph.
			private final String SAMPLE_XML_FILE = "../../../../Data/SamplesConfig.xml";		
		
		
	@section setup_pg HandTracker() method: Setting up the Production Graph
	
		The public HandTracker() routine sets up the Production Graph.
		
		The following code initialization initializes the production graph from an OpenNI XML script file The call to @ref xn::Context::InitFromXmlFile "Context.createFromXmlFile()" combines the effects of two other initialization methods &ndash; @ref xn::Context::Init() and then @ref xn::Context::RunXmlScriptFromFile() &ndash; to initialize the context object and then create a production graph.		@code			
			scriptNode = new OutArg<ScriptNode>();
            context = Context.createFromXmlFile(SAMPLE_XML_FILE, scriptNode);
		@endcode	
		
		The following statements create and set up a GestureGenerator node. The addGesture() method activates the GestureGenerator node to start looking for the named gesture ("Click") in the FOV by adding the gesture's name to the node's list of gestures it actively scans for in the FOV.
		@code
			gestureGen = GestureGenerator.create(context);
            gestureGen.addGesture("Click"); {Not exactly the same as in the API}
		@endcode	
		
		The following statement sets up an event handler for the 'Gesture Recognized' event. The handler is the MyGestureRecognized() function declared elsewhere in the program sample.
		@code
            gestureGen.getGestureRecognizedEvent().addObserver(new MyGestureRecognized() ); 
		@endcode	
		
		The following code block sets up event handlers for a number of hand recognition events. For details about these events, see 
		<a href="#xncpp_wrpr_handgen_events">Hand Event Sequence Diagram</a>, and 
		<a href="#xncpp_wrpr_handcreate_ev">'Hand Create'</a>,
		<a href="#xncpp_wrpr_handupdate_ev">'Hand Update'</a>, and 
		<a href="#xncpp_wrpr_handdestroy_ev">'Hand Destroy'</a>.		
		@code
			handsGen = HandsGenerator.create(context);
			handsGen.getHandCreateEvent().addObserver(new MyHandCreateEvent());
			handsGen.getHandUpdateEvent().addObserver(new MyHandUpdateEvent());
			handsGen.getHandDestroyEvent().addObserver(new MyHandDestroyEvent());
		@endcode
		
		The following statement creates a @ref xn::DepthGenerator "DepthGenerator".
		@code
			depthGen = DepthGenerator.create(context);
		@endcode	
		
		In the following statement, the latest data generated is copied to an easy-to-access @ref glos_frame_object "<i>frame object</i>". In OpenNI terminology: the node has data that is designated as 'metadata to be placed in the node's metadata object'. The node's getMetaData() method gets this data and copies it to a metadata object, <code>depthMD</code>. The data includes all configuration information associated with the data itself. This metadata object is then termed the @ref glos_frame_object "<i>frame object</i>". <br>
		For more explanation on this, see @ref conc_meta_data, @ref glos_frame_object, and @ref frame_data.		
		@code
			DepthMetaData depthMD = depthGen.getMetaData();
		@endcode	
		
		The following statement ensures all created @ref dict_gen_node "generator nodes" are generating data.
		@code
			context.startGeneratingAll();
		@endcode	

		The following sets up the histogram (see @ref rec_raw_mainprg_dcl_blk_hist). See 
		<code>getFullXRes</code> and <code>getFullYRes</code> get the number of rows and columns in the full frame (that is, the entire field-of-view), ignoring cropping. This enables the program set up the right size buffer.
		@code
			history = new HashMap<Integer, ArrayList<Point3D>>(); 

			histogram = new float[10000];
			width = depthMD.getFullXRes();
			height = depthMD.getFullYRes();

			imgbytes = new byte[width*height];

			DataBufferByte dataBuffer = new DataBufferByte(imgbytes, width*height);
			Raster raster = Raster.createPackedRaster(dataBuffer, width, height, 8, null);
			bimg = new BufferedImage(width, height, BufferedImage.TYPE_BYTE_GRAY);
			bimg.setData(raster);
		@endcode	
		
		
	@section htj_calcHist CalcHist() - Using the Depth Values to Build an Accumulative Histogram		
	
		There are no OpenNI-specific declarations in this routine.
		
		
	@section htj_update_depth_fn updateDepth() method: Updating the Depth Map
	
		The following statement sets up the frame object. It is described in the <a href="#getMetaData">HandTracker() method above</a>. For more explanation on this, see @ref conc_meta_data, @ref glos_frame_object, and @ref frame_data.
		@code
			DepthMetaData depthMD = depthGen.getMetaData();
		@endcode	
		
		The following method call waits for any node to have generated new data. This method then 'updates' each and every node in the entire production graph. For an overview to reading data and the <b>WaitXUpdateAll</b> methods, see <i>@ref conc_updating_data__summary_of_wait_fns</i>.	

		The @ref xn::Context::WaitAnyUpdateAll() "waitAnyUpdateAll()" method in the following statement waits for any node to have generated a new data frame. The method then makes the data frames of all nodes in the entire production graph available for getting. The application can then get the data (for example, using a metadata GetData() method). This method has a timeout.		
		@code
			context.waitAnyUpdateAll();
		@endcode
		
		The following code block creates a convenient buffer for the depth map and then calls the calcHist() method to calculate the histogram. There are no OpenNI-specific operations in this code block.
		@code
			ShortBuffer depth = depthMD.getData().createShortBuffer();
			calcHist(depth);
			depth.rewind();		
		@endcode
		
		
		This code block copies the depth into a local buffer, so that it can be drawn on the screen.
		@code        
			while(depth.remaining() > 0)
			{
				int pos = depth.position();
				short pixel = depth.get();
				imgbytes[pos] = (byte)histogram[pixel];
			}		
		@endcode
		
		
	@section htj_getPreferredSize getPreferredSize() method

		There are no OpenNI-specific operations in this routine.		
	
	
	@section htj_paint paint() method
	
		
		This function draws the current location of each known hand, with a tail consisting of its previous points. It uses the history list populated in the HandGenerator callbacks.

*/