END OF AN ERA, FRACTALFORUMS.COM IS CONTINUED ON FRACTALFORUMS.ORG

the only way to insert own formulas is by writing them hard coded in c++ or straight assembler, and include them in the plugins folder,
for experimenting i would suggest you try out "fragmentarium"  although it might seems hard to include own formulas, you find the developer
of the program here, and with the right scripts you can create own formulas without fiddling too much with distance estimates ... hope
it helps it is using GLSL to define the images, and comes with a loadful of example scripts

you need a pretty capable GPU though:

http://syntopia.github.io/Fragmentarium/

There's also the brute force raytracer (so no distance estimation), which works good enough to get a rough idea of how the surface looks even on my old NV 9400M.  Note that the current binary release has a couple of slight bugs in the shaders, but you can fix that with any old text editor.  Listed here: http://www.fractalforums.com/fragmentarium/fragmentarium-1-0-beta-1/msg64200/#msg64200

These issues were fixed in the 'beta 3' linked to in the first post in that thread.