Best approach for external data stream integration

[cKleinhuis]

I don't know what do you mean by "joint informations", because I don't have a Kinect device, but I'll see what I can do with Kinect libraries.

kinect driver not only delivers the depth image stream and color image stream, but also concrete scanned humans body joints, e.g. the hand position in form of a 3d coordinate, all body joints are included and up to 3 players can be scanned with one kinectv2 device

i think either as large input matrix or as input texture these joint informations can be passed through to a shader, a utility method with enumerated body joins which returnbs the vec3 position of that joint could be helpful as well

[Han Maokun]

I see, how about the joint informations ? Perhaps encode joint positions i to a bitmap?

Libfreenect doesn't support skeleton tracking, according to 'http://stackoverflow.com/questions/19181332/libfreenect-vs-openni', seems OpenNI library has an advantage on this.
I use libfreenect originally in my implementation since I was on a project depends on OpenFrameworks, in which libfreenect is used to interact with Kinect device.
I don't have experience on OpenNI, next I am gonna have a test with Kinect model 1473, and see if it works. Looks to me OpenNI is more promising, I hope it also supports the coming next-gen Intel Realsense devices.

[Han Maokun]

kinect driver not only delivers the depth image stream and color image stream, but also concrete scanned humans body joints, e.g. the hand position in form of a 3d coordinate, all body joints are included and up to 3 players can be scanned with one kinectv2 device

i think either as large input matrix or as input texture these joint informations can be passed through to a shader, a utility method with enumerated body joins which returnbs the vec3 position of that joint could be helpful as well

Currently in my implementation for Kinect device support, the depthmap data is stored in R channel of a texture, I don't know the exact data format of the joints information, hope they can fit in the left 'G' 'B' 'A' channel of the same  texture.

[cKleinhuis]

the joint data would could fit into there, but it is far too short a full teture