Logo by MandelBRO - Contribute your own Logo!

END OF AN ERA, FRACTALFORUMS.COM IS CONTINUED ON FRACTALFORUMS.ORG

it was a great time but no longer maintainable by c.Kleinhuis contact him for any data retrieval,
thanks and see you perhaps in 10 years again

this forum will stay online for reference
News: Follow us on Twitter
 
*
Welcome, Guest. Please login or register. January 16, 2019, 04:00:19 AM


Login with username, password and session length


The All New FractalForums is now in Public Beta Testing! Visit FractalForums.org and check it out!


Pages: [1] 2   Go Down
  Print  
Share this topic on DiggShare this topic on FacebookShare this topic on GoogleShare this topic on RedditShare this topic on StumbleUponShare this topic on Twitter
Author Topic: Best approach for external data stream integration  (Read 1043 times)
0 Members and 1 Guest are viewing this topic.
Patryk Kizny
Global Moderator
Fractal Fertilizer
******
Posts: 372



kizny
WWW
« on: October 21, 2015, 07:56:10 PM »

Hey,

So the next idea I have in my mind is to use external data to dynamically modify a fractal.
The data is coming from external sensors over serial port. It may be arduino-based stuff, MIDI or OSC.
I wonder what approach would you recommend to input this data in real-time to Synthclipse rendering a fractal.

Let's assume that the data input frequency would be higher than typical frame rate. So let's say I'm targeting 25FPS realtime performance and I'd need either to sample inputs at the framerate frequency or more often and only feed it to Synthclipse renderer every 40ms.

Is it doable only with Javascript stuff? Or I'd need to code something in C++ and feed it into?
What would be the best approach?

Thanks!
Logged

Visual Artist, Director & Cinematographer specialized in emerging imaging techniques.
cKleinhuis
Administrator
Fractal Senior
*******
Posts: 7044


formerly known as 'Trifox'


WWW
« Reply #1 on: October 21, 2015, 09:46:11 PM »

i find similar to shadertoy the option cool to have the webcam input as texture channel cheesy
Logged

---

divide and conquer - iterate and rule - chaos is No random!
_revers_
Conqueror
*******
Posts: 138



« Reply #2 on: October 22, 2015, 08:56:13 AM »

@Patryk
You obviously need to pack data into texture. (Like in audio texture [music UC], where 512 samples are packed for one frame). Creating and filling a texture in JavaScript is doable but gathering an input from external devices is not. The other concern about JavaScript generated textures in every frame would be performance, which might be slow.
Please point me some C++ library (preferably cross platform) for reading input you want and I see how much effort would it take me to add this to Synthclipse.
My friend Han Maokun has done Kinect integration for Synthclipse which maybe I include in the next release.

@Chris
I will also consider capturing camera input.
« Last Edit: October 22, 2015, 09:59:18 AM by _revers_ » Logged
cKleinhuis
Administrator
Fractal Senior
*******
Posts: 7044


formerly known as 'Trifox'


WWW
« Reply #3 on: October 22, 2015, 09:15:21 AM »

That is col to have kinect data inside synthclipse out of the box, i hopeit supports the new kinectv2
Logged

---

divide and conquer - iterate and rule - chaos is No random!
Patryk Kizny
Global Moderator
Fractal Fertilizer
******
Posts: 372



kizny
WWW
« Reply #4 on: October 22, 2015, 07:39:47 PM »

Please point me some C++ library (preferably cross platform) for reading input you want and I see how much effort would it take me to add this to Synthclipse.

Essentially what would be sufficient for me on the basic level is an access to a serial port communication.

What the library/module should do is:
- make it possible to configure serial port settings from Synthglipse GUI and set baud rate, polling rate, open and close port, listen to a stream
- parse the data
- allow to create simple bindings/mappings: table of incoming variables, dropdown or textfield to enter corresponding Uniform on the right
- listen to the communications on the oper port, parse the data and set the uniforms accordingly during the runtime

Let's say in simplest scenario the communication on serial port would have a form of simple ASCII variables sent in separate lines over the serial port.
Say:
"VarName1 VarValue1; VarName2 VarValue2; VarName3 VarValue3; etc;"
And the lib would assign these values to selected uniforms in the runtime.

I have no practical knowledge of C++, but I bet there's a dozens of libs for facilitating serial communication.
Therere's a bunch of tools that are available for PD/MAX MSP including simple communications interfaces and templates for Arduino - say Firmata (https://www.arduino.cc/en/Reference/Firmata) for example or Simple Message System: http://playground.arduino.cc/Interfacing/PD

Here's some (old lib) for Arduino and C++ with LibSerial:
http://sglez.org/2008/08/05/interfacing-arduino-with-c-and-libserial/

And for Windows:
http://playground.arduino.cc/Interfacing/CPPWindows

Here's more details on available libs for other software:
http://playground.arduino.cc/Main/InterfacingWithSoftware

My friend Han Maokun has done Kinect integration for Synthclipse which maybe I include in the next release.

Is there any way to get in touch with him and get access to it? Kinect is my 2nd most important area of interest after serial for arduino.
Logged

Visual Artist, Director & Cinematographer specialized in emerging imaging techniques.
Patryk Kizny
Global Moderator
Fractal Fertilizer
******
Posts: 372



kizny
WWW
« Reply #5 on: October 22, 2015, 07:41:09 PM »

You obviously need to pack data into texture. (Like in audio texture [music UC], where 512 samples are packed for one frame).

Not so sure. Why not read the data on C++ end, parse it and use to set uniforms at each frame?
Logged

Visual Artist, Director & Cinematographer specialized in emerging imaging techniques.
Han Maokun
Forums Newbie
*
Posts: 4



« Reply #6 on: October 23, 2015, 02:46:02 AM »

That is col to have kinect data inside synthclipse out of the box, i hopeit supports the new kinectv2

Hello everyone, I am Han Maokun:)
I've been on the work of Kinect sensor feature of Synthclipse these days, I have a Kinect module 1473 in my hand, the library used is libfreenect(https://github.com/OpenKinect/libfreenect), to support Kinect V2, libfreenect2(https://github.com/OpenKinect/libfreenect2) is needed, I think they can both exists in Synthclipse.
Logged
_revers_
Conqueror
*******
Posts: 138



« Reply #7 on: October 23, 2015, 08:56:50 AM »

@Patryk
Thanks for the links. I will take a look at those libs.
I've thought about packing data into texture since you said that the frequency of the data is higher than the framerate.

@Han
I will include your Kinect 'module' in the next Synthclipse release.

@Chris
I will consider supporting Kinect v2, but without a test device it might be hard. You would need to help me testing it before release.
Logged
Patryk Kizny
Global Moderator
Fractal Fertilizer
******
Posts: 372



kizny
WWW
« Reply #8 on: October 23, 2015, 09:37:25 AM »

I got a bunch of kinects here so I'm also keen to help testing.
Logged

Visual Artist, Director & Cinematographer specialized in emerging imaging techniques.
_revers_
Conqueror
*******
Posts: 138



« Reply #9 on: October 24, 2015, 12:38:28 AM »

Thanks, that would be very helpful. I let you know when I start implementation.
Logged
cKleinhuis
Administrator
Fractal Senior
*******
Posts: 7044


formerly known as 'Trifox'


WWW
« Reply #10 on: October 24, 2015, 09:34:21 AM »

i have windows10 with kinect2 readdy for testing!
Logged

---

divide and conquer - iterate and rule - chaos is No random!
cKleinhuis
Administrator
Fractal Senior
*******
Posts: 7044


formerly known as 'Trifox'


WWW
« Reply #11 on: October 24, 2015, 09:35:29 AM »

how is kinect implementation planned? as uniform input for variables? or using jsx scripting ?

Logged

---

divide and conquer - iterate and rule - chaos is No random!
_revers_
Conqueror
*******
Posts: 138



« Reply #12 on: October 24, 2015, 11:02:41 AM »

Cool, I give you both app for testing when it will be ready.

As of the implementation, depth map created by Kinect is provided as a texture (uniform sampler2D) with Uniform Control type "depthmap".
Logged
cKleinhuis
Administrator
Fractal Senior
*******
Posts: 7044


formerly known as 'Trifox'


WWW
« Reply #13 on: October 24, 2015, 11:36:07 AM »

I see, how about the joint informations ? Perhaps encode joint positions i to a bitmap?
Logged

---

divide and conquer - iterate and rule - chaos is No random!
_revers_
Conqueror
*******
Posts: 138



« Reply #14 on: October 24, 2015, 10:19:33 PM »

I don't know what do you mean by "joint informations", because I don't have a Kinect device, but I'll see what I can do with Kinect libraries.
Logged
Pages: [1] 2   Go Down
  Print  
 
Jump to:  


Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines

Valid XHTML 1.0! Valid CSS! Dilber MC Theme by HarzeM
Page created in 0.109 seconds with 28 queries. (Pretty URLs adds 0.008s, 2q)