Synchronizing Fractal Transformations to Music

[pinballpsycho]

This is a video I created syncing 2D fractal transformations with Dvorak's 'American Quartet'.  I chose this piece because I found a copy of it that is pubic domain.

https://player.vimeo.com/video/137190237

[Michael Scharrer]

Did you manually synchronize it or did you use some algorithm? I'd be very interested in the latter.

[freakiebeat]

I am working on a video which synchronizes fractal movement with music. The video is called Trance Mission and named after the original title of the music which was commissioned for another animation I was doing, but never got used. I am still rendering the video, but previewing it in minute segments as the rendering progresses.

These are flame fractals which I submitted to the electric sheep project during flock #244 around 2011. I have gone back and added motion elements to get the various transforms to move in conjunction with the music in addition to their normal animation. All this had to be done on the command line since there are no flame editors which handle flam3 motion elements.

link to the preview: https://vimeo.com/ondemand/trancemission/142289148

[cKleinhuis]

i do likewise i have written a ffgl resolume plugin

http://spack-o-mat.digitalgott.com/index.php?/archives/14-sOm-Mandelbrot-Twitch-released.html

to be used with
http://resolume.com

[pinballpsycho]

@Michael,  I personally have not been able to read directly from an audio file and generate timing and wave envelopes for individual instruments to trigger these effects, primarily because everything has been mixed into a single waveform (actually two - one for each channel).  Once they're mixed like that, I know of no way to reliably (and cheaply) "un-mix" it into separate instruments.  Perhaps others might - I'd love to know the process myself.  My approach is quite manual; I break each instrument out, store the notes into a heap, then use the data from the heap to drive the effects.  There are mistakes.

If you commission work like freakiebeat did, perhaps you could ask them to record the instruments into separate tracks and then use that to automate the timing and create envelopes.

[cKleinhuis]

for finding tools for create MIDI out of audio, i found several tools that seem to be able to do such
just google "audio to midi"

here on this page is a plugin for ableton (i think) which does it
http://www.widisoft.com/english/mp3-midi-products.html

[pinballpsycho]

Excellent!  Thanks Claus!