END OF AN ERA, FRACTALFORUMS.COM IS CONTINUED ON FRACTALFORUMS.ORG

Hey,

So the next idea I have in my mind is to use external data to dynamically modify a fractal.
The data is coming from external sensors over serial port. It may be arduino-based stuff, MIDI or OSC.
I wonder what approach would you recommend to input this data in real-time to Synthclipse rendering a fractal.

Let's assume that the data input frequency would be higher than typical frame rate. So let's say I'm targeting 25FPS realtime performance and I'd need either to sample inputs at the framerate frequency or more often and only feed it to Synthclipse renderer every 40ms.

Is it doable only with Javascript stuff? Or I'd need to code something in C++ and feed it into?
What would be the best approach?

Thanks!

@Patryk
You obviously need to pack data into texture. (Like in audio texture [music UC], where 512 samples are packed for one frame). Creating and filling a texture in JavaScript is doable but gathering an input from external devices is not. The other concern about JavaScript generated textures in every frame would be performance, which might be slow.
Please point me some C++ library (preferably cross platform) for reading input you want and I see how much effort would it take me to add this to Synthclipse.
My friend Han Maokun has done Kinect integration for Synthclipse which maybe I include in the next release.

@Chris
I will also consider capturing camera input.

Essentially what would be sufficient for me on the basic level is an access to a serial port communication.

What the library/module should do is:
- make it possible to configure serial port settings from Synthglipse GUI and set baud rate, polling rate, open and close port, listen to a stream
- parse the data
- allow to create simple bindings/mappings: table of incoming variables, dropdown or textfield to enter corresponding Uniform on the right
- listen to the communications on the oper port, parse the data and set the uniforms accordingly during the runtime

Let's say in simplest scenario the communication on serial port would have a form of simple ASCII variables sent in separate lines over the serial port.
Say:
"VarName1 VarValue1; VarName2 VarValue2; VarName3 VarValue3; etc;"
And the lib would assign these values to selected uniforms in the runtime.

I have no practical knowledge of C++, but I bet there's a dozens of libs for facilitating serial communication.
Therere's a bunch of tools that are available for PD/MAX MSP including simple communications interfaces and templates for Arduino - say Firmata (https://www.arduino.cc/en/Reference/Firmata) for example or Simple Message System: http://playground.arduino.cc/Interfacing/PD

Here's some (old lib) for Arduino and C++ with LibSerial:
http://sglez.org/2008/08/05/interfacing-arduino-with-c-and-libserial/

And for Windows:
http://playground.arduino.cc/Interfacing/CPPWindows

Here's more details on available libs for other software:
http://playground.arduino.cc/Main/InterfacingWithSoftware


Is there any way to get in touch with him and get access to it? Kinect is my 2nd most important area of interest after serial for arduino.

Hello everyone, I am Han Maokun:)
I've been on the work of Kinect sensor feature of Synthclipse these days, I have a Kinect module 1473 in my hand, the library used is libfreenect(https://github.com/OpenKinect/libfreenect), to support Kinect V2, libfreenect2(https://github.com/OpenKinect/libfreenect2) is needed, I think they can both exists in Synthclipse.

I got a bunch of kinects here so I'm also keen to help testing.

i have windows10 with kinect2 readdy for testing!

Cool, I give you both app for testing when it will be ready.

As of the implementation, depth map created by Kinect is provided as a texture (uniform sampler2D) with Uniform Control type "depthmap".

I don't know what do you mean by "joint informations", because I don't have a Kinect device, but I'll see what I can do with Kinect libraries.