![]() |
OpenNI 1.5.4
|
<b>Source files:</b> Click the following link to view the source code file: - NiUserTracker\main.cpp - opengles.cpp - SceneDrawer.cpp This section describes the NiUserTracker sample program written in C++. The executable program for Windows is NiUserTracker.exe. The documentation describes the sample program's code from the top of the program file to bottom. Every OpenNI feature is described the first time it appears in this sample program. Further appearances of the same feature are not described again. FILE NAME: main.cpp
The following declarations define the OpenNI objects required for building the OpenNI production graph. The production graph is the main object model in OpenNI.
Each of these concepts is described separately in the following paragraphs.
The production graph is a network of software objects - called production nodes - that can identify blobs as hands or human users. In this sample program the production graph identifies blobs as human users, and tracks them as they move. See The Production Graph for more about the production graph.
A xn::Context object is a workspace in which the application builds an OpenNI production graph.
The xn::ScriptNode object loads an XML script from a file or string, and then runs the XML script to build a production graph. The ScriptNode object must be kept alive as long as the other nodes are needed.
A xn::DepthGenerator node generates a depth map. Each map pixel value represents a distance from the sensor.
A xn::UserGenerator node generates data describing users that it recognizes in the scene, identifying each user individually and thus allowing actions to be done on specific users. The single UserGenerator node gets data for all users appearing in the scene.
A xn::Player node plays a saved recording of an OpenNI data generation session.
This function releases the OpenNI nodes. Releasing the nodes unreferences them, decreasing their reference counts by 1. If a node's reference count reaches zero, it will be destroyed. In this sample program the result of this function should be the destruction of all the nodes. void CleanupExit() { g_scriptNode.Release(); g_DepthGenerator.Release(); g_UserGenerator.Release(); g_Player.Release(); g_Context.Release(); exit (1); }
This section describes the event handlers this sample program requires, describing the nature of the events themselves and what is done inside the handlers. A typical order of invocation of the events in the default configuration, where online-calibration is enabled, would be: 1. 'New User' event 2. 'Calibration Complete' event 3. 'Lost User' event Online-calibration enables the acquisition of a skeleton without the need for poses. The events are described below in order of their declaration in the source code. Note: When online-calibration is turned off ( which is <i>not </i> the default configuration) a 'Pose Detected' event would typically occur after the 'New User' event and before the Calibration Complete' event.
The <b>'New User' event</b> signals that a new user has now been recognized in the scene. A new user is a user that was not previously recognized in the scene, and is now recognized in the scene. The user is identified by a persistent ID. Below is a typical implementation of the event handler. It's processing is as follows. Now that a new user has been detected, the handler calls @ref xn::PoseDetectionCapability::StartPoseDetection() "StartPoseDetection()" to start pose detection.
The <b>'Lost User' event</b> signals that a user has been lost from the list of previously recognized users in the scene. The exact meaning of a 'lost user' is decided by the developer of the @ref xn::UserGenerator. However, a typical implementation would define that a lost user is a previously recognized user that then exits the scene and does not return, even after a 'Lost User' timeout has elapsed. Thus this event might be raised only after some delay after the user actually exited the scene. Below is a typical implementation of the event handler. It's processing is as follows. Now that an existing user has been lost, the handler deletes the user's entry from the @ref utcs_init_joints_array <code>joints</code> array.
The <b>'Pose Detected' event</b> signals that a human user made the pose named in the call to the StartPoseDetection() method. The user is designated with the ID given by the <code>nID</code> parameter. Below is a typical implementation of the event handler. It's processing is as follows. Now that a pose has been detected, the handler calls @ref xn::PoseDetectionCapability::StopPoseDetection() "StopPoseDetection()" to stop pose detection. The handler then calls @ref xn::SkeletonCapability::RequestCalibration() "requestSkeletonCalibration()" to start calibration. The <code>true</code> disregards any previous calibration and forces a new calibration.
The <b>'Calibration Start' event</b> signals that Signals that a specific user's SkeletonCapability object is now starting the calibration process. Below is a typical implementation of the event handler. It has no OpenNI specific code. It just records the time the handler was called and then prints it out.
The <b>'Calibration Complete' event</b> signals that a specific user's skeleton has now completed the calibration process, and provides a result status. The user is identified by the ID given by the <code>nId</code> parameter. Below is a typical implementation of the event handler. It's processing is as follows. The handler tests whether the calibration process was completed successfully. If successful, that means that a user has been detected and calibrated, and enough information has been obtained to create a skeleton to represent the user. The handler then advances the processing to the next stage, i.e., to call @ref xn::HandsGenerator::StartTracking() "StartTracking()" to start tracking the skeleton, which represents a human user body, within a real-life (3D) scene for analysis, interpretation, and use by the application. (Description continued after the code.)
In the above handler, if the calibration process failed, the handler restarts the whole calibration sequence. The way the handler restarts the calibration sequence depends on whether the specific generator demands detecting a pose before starting calibration . If a pose is required, the program calls StartPoseDetection() to start attempting to detect a pose for a specific user.
If a pose is not required, the program calls GetSkeletonCap().xn::SkeletonCapability::RequestCalibration() "RequestCalibration()". GetSkeletonCap() gets a SkeletonCapability object for accessing Skeleton functionality. the RequestCalibration() method starts the calibration process to calibrate a user. The TRUE parameter means to disregard previous calibration to force a further calibration.
The following definition is for the path to an OpenNI XML script file. This file is for inputting and building a stored production graph.
This routine saves to a file the skeleton calibration data of the first user that it finds is calibrated. . This is a very useful tool for developers. They can save their own calibration, and test their application again without calibrating each time (going into pose, spend time on calibration).
From the code above:
The GetUsers()
gets user skeleton calibration data and places it in the aUserIDs
array, with one entry per user. Then in the 'save' loop, later on, the code loops through each user in turn testing if it has been calibrated, and when it finds the first calibrated user it saves its calibration data to a file, XN_CALIBRATION_FILE_NAME defined as "UserCalibration.bin" above.
The following routine loads the user skeleton calibration data from a file. This is a very useful tool for developers. They can save their own calibration, and test their application again without calibrating each time (going into pose, spend time on calibration). The code loads data only for the first found user that is not yet calibrated or in the middle of being calibrated.
This function is called each frame. There are no OpenNI-specific declarations in this function.
There are no OpenNI-specific declarations in this function.
There are no OpenNI-specific declarations in this function.
There are no OpenNI-specific declarations in this function. The CHECK_RC() macro checks whether the most recent OpenNI operation was successful or returned an error result. On error, the @ref xn::xnGetStatusString "xnGetStatusString()" method converts the OpenNI error return code to the corresponding error string for printing. For the sake of conciseness, the rest of this documentation skips calls to this macro.
This routine graphically displays the data on a screen. The following declare metadata objects to provide frame objects for the @ref xn::DepthGenerator node and for the @ref xn::UserGenerator node. A @ref dict_gen_node "generator node's" @ref glos_frame_object "frame object" stores a generated data frame and all its associated properties. This data frame and its properties are accessible through the node's metadata object.
In the following statements, the DepthGenerator frame object is used to access the XRes() and YRes() methods. These methods return the X and Y dimensions of the depth buffer. These values are used for stepping through the depth map buffer to get the individual pixel values.
the WaitOneUpdateAll() method in the following statement updates the application buffer of each and every node in the entire production graph, but first waiting for a specified node to have generated a new data frame. The application can then get the new data (for example, using a metadata GetData()
method). The WaitOneUpdateAll() method has a timeout. In this sample program, the following statement updates the production graph only if the UserGenerator node has new data.
The following code block gets the frame objects to use them to draw the depth map, users, and skeletons. Frame objects are a snapshot of the generated map data and its associated configuration information at a certain point in time. Frame objects provide fast and easy access to the DepthGenerator node's data and configuration information.
In the above, The GetMetaData() gets the DepthGenerator node's frame object, saving it in the xn::DepthMetaData object.
GetUserPixels() gets the pixel map of the specified user. This is a pixel map of the entire scene saved as a frame object, where the pixels that represent the body are labeled with user IDs. Each pixel is labeled with the ID of the user that contains that pixel.
The main program starts by initializing an OpenNI status flag and then initializes the production graph (see the following code). If the program is invoked with a parameter containing a recording name, the program initializes the production graph from the recording file. Otherwise, it initializes the production graph from the standard OpenNI XML file.
Production graph initialized from recording: In the following code block, g_Context.Init() initializes the context. The call to g_Context.xn::Context::OpenFileRecording() "OpenFileRecording()" then opens a recording file. The argv[1] parameter supplies the name of the recording file. The g_Player parameter returns a xn::Player node through which playback can be controlled, e.g., seeking and setting playback speed.
Production graph initialized from standard XML file: In the following code block, g_Context.InitFromXmlFile() initializes the context and loads the script file to build a production graph. SAMPLE_XML_PATH is the path to the XML file, g_scriptNode
is the xn::ScriptNode object as described earlier, and the errors
object returns a list of any errors that occurred.
In the following code, the @ref xn::Context::FindExistingNode() "FindExistingNode()" call gets a reference to production nodes in the production graph. In this example, the application passes the g_depth parameter to get a reference to a @ref xn::DepthGenerator "DepthGenerator node" so that it can work with it. Then the same for a @ref xn::UserGenerator "UserGenerator node".
The following code blocks initialize and register event handlers for the UserGenerator node and its xn::SkeletonCapability "skeleton capability". A skeleton capability provides <b>Skeleton</b> functionality to a @ref xn::UserGenerator node. First the application checks that the node supports skeleton capability.
To be able to track a user's skeleton, the SkeletonCapability can execute a calibration process to measure and record the lengths of the human user's limbs. This would make it easier for OpenNI to then successfully track the human user. The calibration process can be initiated by the human user performing an agreed calibration pose.
Here is the code for registering the event handlers. The xn::UserGenerator accesses its skeleton capability by calling the xn::UserGenerator::GetSkeletonCap() method.
The application then checks if the skeleton capability requires a pose detection in order to execute a calibration. If so, the application will have to get a xn::PoseDetectionCapability object. The code then registers to a 'Pose Detected' event.
The following statement sets the skeleton profile. The skeleton profile specifies which joints are to be active, and which to be inactive. XN_SKEL_PROFILE_ALL means all the joints. The xn::UserGenerator node generates output data for the active joints only. This profile applies to all skeletons that the xn::UserGenerator node generates.
The following statements register to event handlers that report on the progress of detecting a pose and the whole calibration process.
The following statement enters all nodes in the production graph into 'Generating state'. (In this sample application, this includes at least DepthGenerator and UserGenerator.)In this state the node generates new frames. After the application has called this method it calls one of the WaitXUpdateAll methods, e.g., WaitAnyUpdateAll(), to update all generator nodes in the context to their latest available data, first waiting for any of the nodes to have new data available. The application can then get the data (for example, using a metadata GetData() method).
Here there is a block of statements that are not OpenNI specific.
The following statement destroys all the nodes, releasing their memory.
FILE NAME: SceneDrawer.cpp
This event handler - shown below - stores the most recent state of calibration progress, in order to show it as a label later on.
This event handler - shown below - stores the most recent state of pose progress, in order to show it as a label later on.
<code> g_pDepthHist[]</code> is an array with MAX_DEPTH entries (10,000 at the time of writing), one entry for each depth value that the sensor can output. This array is used for the histogram feature in the <code>DrawDepthMap()</code> function later in this file. Each entry of the array is a counter for the corresponding depth value. <code>Histogram[] </code> is used in the DrawDepthMap() function later in this file. As the first stage of processing, DrawDepthMap() builds the histogram by scanning the depth map. For each depth pixel, DrawDepthMap() inspects the depth value, and for that value's entry in the array, it increments its counter by 1. The DrawDepthMap() function then performs further processing, as described later in the description for that function.
There are no OpenNI-specific declarations in this routine.
There are no OpenNI-specific declarations in this routine.
There are no OpenNI-specific declarations in this routine.
There are no OpenNI-specific declarations in this routine.
There are no OpenNI-specific declarations in this routine.
This function draws a limb of the avatar representation of a human user by drawing a line between two OpenNI joints, of type @ref xn::XnSkeletonJoint, passed as parameters to this function. The two joints are meaningful points that represent human's body joints.
The human user, player
, is specified by an integer XnUserID parameter. The two OpenNI joints xn::XnSkeletonJoint points are enum indicators, e.g., XN_SKEL_HEAD.
The statements of this function are explained below.
The function verifies that the user is being tracked by calling the IsTracking() method.
The following code block obtains the X-Y-Z locations of the two joints.
The following code block draws the avatar's limb by drawing a line between the two adjacent points. It uses the locations joint1
and joint 2
obtained above.
The xn::XnSkeletonJointPosition coordinates are real-world coordinates, so the convertRealWorldToProjective() is used to convert the real world coordinates to projective coordinates for the purpose of drawing them on a 2D texture.
The rest of the code in this function draws the line on the graphic display. This code is not OpenNI specific.
This function converts an @ref xn::XnCalibrationStatus type to a string. This is shown in the code block below, with example cases.
This function converts an @ref xn::XnPoseDetectionStatus type to a string. This is shown in the code block below, with example cases.
The DrawDepthMap() function is located on the <code>SceneDrawer.cpp</code> file. In this function, both the frame objects -- <code>dmd</code> and <code>smd</code> -- are accessed to get their data. The same method is used, <code>Data()</code>, which is the standard metadata method for returning a pointer to a frame object's data.
The main user processing loop of this function gets each user in turn and displays it. The following declarations support this processing:
The following initializations are for calculating the accumulative histogram. The following statement accesses the Map Output mode to get the DepthGenerator's map dimensions and pixel color format. @ref xn::DepthMap::XRes "XRes" and @ref xn::DepthMap::YRes "YRes" get the frame X an Y resolutions of the most recently generated data. X and is the number of columns and rows, respectively, in the frame after any required cropping has been applied. See @ref conc_map_wrapper_classes "Map Wrapper Classes" for more information.<br>
The following code block calculates the accumulative histogram.
The following statement initializes the histogram array. This array is a key part of this sample program. (This code is not OpenNI specific.) The histogram feature of this sample program creates a gradient of the scene's depth scene, from dark (far away) to light (close), regardless of the color.
The first loop, a nested for- loop just counts the frequency of each depth value.
The following loop converts the frequency count into an accumulative count. Starting from the first entry this loop calculates a new value for each entry's counter as the sum of itself ([n]
) and the value of the previous counter ([n-1]
).
The following code block completes the histogram.
This code block loops over all the depth values, checking to which user each pixel belongs, and sets the color in the texture according to the user (white for background, others for specific users) and the distance (hue).
The main loop of this function for processing users gets each user in turn and displays it.
The following code block gets a user's center of mass (CoM). This is a single point for representing the user. The CoM is a useful point to represent the user. When you don't have any other reference point (e.g., you don't have the position of a specific joint, or of the head, or any other such point), this is an adequate point with which to start to represent the user. This application uses the CoM as the position at which it writes the user's label. The label comprises its user ID and its current state.
The following statements access the status of each user to display it above each corresponding user image that is displayed on the output display device.
The following statement adds the user's ID to the label, to be displayed on the com of the user the name of the user.
The following statement gets whether the user's skeleton is being tracked.
The following statement gets whether the user's skeleton is still in the middle of being being calibrated. This means that tracking has not yet started.
The following 'else-other' statement displays that the application is still looking for the user to start a pose in order to start calibration and the current status of the pose detection. Values are: OK, NO_USER, TOP_FOV, SIDE_FOV, ERROR
Finally, this application demonstrates an example method, DrawLimb()
, for drawing limbs of all tracked users on a graphical display. A limb is the graphical representation of the human user's arm or leg for example. This method works by taking two parameters that specify a start joint and an end joint for drawing a vector that represents the limb. For example, a 'head' start joint to a 'neck' end joint draws the neck; 'neck' to 'left shoulder' draws the 'left shoulder bridge'.
The call looks something like this: