Welcome to Fractal Forums

Hey all,

Apologies if I missed if it was discussed anywhere earlier.
I wonder if and how could it be done to export transparency and Z-Buffer?

Help well appreciated.
Thanks!

on the C++ side this can be used
GL_COLOR_BUFFER_BIT is the default but GL_DEPTH_BUFFER_BIT is also valid

Many thanks.
Would you be so kind to point me to an example of how this can be added?
I have basic scripting skills and quite new to C++ and fragmentarium.

Would I need to render twice same anim or an extra output can be added?
Perhaps you would be so kind to add it somewhere to a wishlist?

google "qt5 opengl tutorial" for code or "glsl tutorial" for shader scripts and Fragmentarium comes with a LOT of shader code, tutorials and examples, a good idea would be to track down some shader or effect that you like or find interesting and disect it until you understand it, then try writing your own, there is a lot of reference material hidden away in threads here on FF ;)

Hi Patryk,

The alpha and depth values are not written by the shaders at all, so you cannot retrieve them on the C++ side.

But you could modify for instance 'DE-raytracer.frag' to optionally output depth (and perhaps alpha), by adding code such as:

This will only give you 256 z-buffer levels, though.

I am using this bit to populate the depth buffer, as the last line in trace() before returning hitColor, for the spline path occlusion, this does make it available on the C++ side, look in Examples/Include/DE-Raytracer.frag
I think the depth buffer can be set at 24 or 32 bits depending on your hardware too.

I haven't tried saving it or using it for anything other than spline paths , I was thinking point cloud sent to geometry shader? but it would be full of holes as anything not seen would not exist in the depth buffer, unless the point cloud is constructed from several passes viewing from different angles or maybe slices.

Ah - I should probably say that I meant the 'classic' Fragmentarium. I forgot you implemented some OpenGL integration.

Still, I think it would be demanding - even being able to save a 24-bit or 32-bit channel image is difficult. (I managed to get HDR import into Fragmentarium, but I never got HDR export to work).

Thank you guys. And huge thanks 3dickulus for plans on integrating Z-depth you mentioned in the main thread.
Let me elaborate more on what we need in the ideal world. Again, this is all coming from the desire of integrating fractals in the proper VFX pipelines.

Z-buffer
- The Z-buffer is important for us because it allows us to play with the images in post production with quite a lot of flexibility. Not having Z-depth is working with tied hands.
- Same goes for transparency, but if there is a good Z-depth saved along with the images, it's easy to subtract the 'background' based on Z-depth in post. So Z-depth is crucial

Now we hit another point.

color depth for both main images and z-depth
- I did not investigate that yet with Fragmentarium, but looking at what you said I recon currently there's only 8bpp output.
- the standard for VFX and postproduction pipelines require at least 16bits per channel. 8bpp limits image manipulation significantly.
- for demanding applications we go for 32bpp.

File formats
- Generally we can work with anything - be it .png or .tiff or other formats. Crucial is to have sufficient color depth.
- Often .dpx files are being used. I might only assume these are not 'by default' supported by system, so adding may be a bit more time consuming. It is not a priority.

Saving Z-depth along with main image
Looking at the perspective of saving Z-Depth along with the images, I can suggest the following options to make it convenient:
→ (A) Worst case scenario, but still better than no Z-Depth: rendering both Z-depth and main to one image, one above the other. I could work with it, but it would add processing time to split this in post to make it usable.
→ (B) Simple and easy - outputting main and z-depth in 2 separate files, named commonly, only with "Z-buffer" suffix for ZD, followed by frame number
→ (C) Pro solution - I would suggest looking into OpenEXR format (http://www.openexr.com/) developed by Industrial Light & Magic. It allows for various color depths, has compression algorithms, but what's most important here, it allows to save multiple layers into the same file and is being often used in post-productions pipelines. Today we speak about saving only Z-buffer along and we can live with separate file outputs. But looking towards a future-proof solution, OpenEXR will be a great choice allowing to save many different channels/layers into same file efficiently.

At what depth you are handling the buffer? Are we limited on the buffer level or that's just a matter of saving to file bottleneck?

Just my suggestions for the future development.
Hope this is anyhow helpful.

Hi Patryk, first let me say that I like this sw, but working with GPU floating point precision you will not be able to enlarge objects such as when using a software based on CPU.
Whereas you probably need double precision support, it is mandatory to find a suitable GPU or you can better use directly Mandelbulber, sw that is able to create all those layers that you're specifically asking for.
But you are free to use another sw, obviously.  :)

Internally, the accumulation buffer in Fragmentarium can be 8, 16, or 32 bit (only 32 bit is float - and it is also the default. 8 and 16 is integer). But these cannot be exported: in Qt the QImage image class only supports 8bpp, which means we can only save PNG in 8-bit (at least using Qt). Also some of the code, like putting the tiles together when doing tiled rendering uses the QImage class and is thus 8bpp only. QGLFramebufferObject.toImage() is 8bpp too. There are workarounds for all this, and I guess OpenEXR could be supported but it is quite some work.

thankyou Syntopia, I was just crafting a similar reply... the ILM EXR format does look interesting but would require a rewrite of all of the image handling.

edit: could one develop a shader in Fragmentarium or shadertoy and then use something like http://www.fractalforums.com/index.php?topic=21755.msg84293#msg84293 (with some special coding) to generate image/buffer files at the bitdepth required, I have seen glsl to pascal and glsl to c converters so the higher cpu precision could be exploited this way.

just looking at EXR stuff...
you could render images with 3 or 4 different exposure settings and combine them for a final image but this has the obvious effect of tripling or quadrupling the render time,
also one has control of camera features like brightness contrast exposure etc so you could also just set it up right in the first place instead of rendering extra data (a feature of EXR is like having many exposures embedded in the image file, I know it's more complicated than that though)
that gets excluded in the end anyways (in post I imagine you look at a scene, adjust the exposure to bring out/in details, and say "that's the one, print")
by the very nature of the display medium/device ie:the colour resolution of your monitor or film printer or eyes.
In Fragmentarium you can morph all camera settings over any range of frames in a number of ways so if an exposure or other cam param transition is required to bring out /in detail it can be done pre in one render instead of post after 3 or 4, but of course we end up with an RGBA8 on our monitors.

Difficulty in programming aside, I ask myself "how useful is this for shader development" or more simply "on my desktop".

Isn't it this discussion you were thinking of? http://www.fractalforums.com/programming/compilerun-glsl-shader-as-cplusplus/


There is no need for storing multiple exposures when using floats as these hdr-formats do. The images should be grabbed from the Fragmentarium 32-bit float buffers which has an extremely high dynamic range.

yes, that one and https://code.google.com/p/delphi-shader/
yes that is kind of my point, internally it is about as good as we can get on consumer hardware, developing a Fragmentarium-like app with the intention of going to film is a different animal compared to the exploration of fractals on my desktop.

Just tried the delphi-shader - which is nice, but also very slow.

I tried changed a Fragmentarium shader to use doubles (just insert #version400 at the top, and change vec3 -> dvec3, float -> double), which made it run 36.6x slower. (Rather close to the expected theoretical factor 32x for the Nvidia Maxwell GPU I use). Still it was faster than the Delphi-shader (based on the roughly similar Mandelbulb example)! And Maxwell is really bad at double precision: in particular some AMD GPU's are running 1/4th in DP: http://www.geeks3d.com/20140305/amd-radeon-and-nvidia-geforce-fp32-fp64-gflops-table-computing/ (The pro cards, FirePro and Tesla, run double precision at half speed).

So the question is whether running on the CPU is worth it.

so from that I gather the 2 minute 1080p render @ 3000 frames that took over night will take over a few months @ film quality resolutions

OpenEXR might be handy for when you find that insanely awesome location and want to get really high quality output for reproductions

aside: I thought delphi-shader might be helpful for MB3D

Emulated double precision is a lot faster- I get about double the speed on my GTX 880M. Unfortunately, the double precision functions included in Fragmentarium haven't worked for me since I upgraded my GPU- it seems that Nvidia's compilers no longer recognize the fastprecision and fastmath pragmas. It can be fixed by using "precise" qualifiers for all variables, but that either requires OpenGL 4.0 or the extension ARB_gpu_shader5, which is only supported from GL 3.2 and later.

By the way, I made an optimized set of double precision functions that run somewhat faster than the old ones (they require ARB_gpu_shader5, though, because they rely on the single-precision fma operation). I also added fairly fast sin, cos, and exp functions. I didn't make a division one yet- once I add that one, I'll post the whole set.

I only think the OP asked for 32-bit output export (which will not lower rendering speed). I don't think internal calculation 64-bit resolution was needed?


You are quite right. The EmulatedDouble.frag does not work on my machine either. It has been reported here, but no answer:
https://devtalk.nvidia.com/default/topic/820790/opengl/bug-compiler-no-longer-respects-pragma-optionnv-fastmath-off-is-there-an-alternative-/

Having a set of trigonometric 64-bit functions would be cool. Actually, there are not 64-bit trigonometric functions in GLSL either, so these would be useful.

Hi there! Looks like I opened a pandora box :)

I think there's some confusion coming from visual.bermarte post.
Calculation precision on GPU is what it is and I did not have in mind messing with that part.
What I meant is only being able to export images in either 16bpp int (which is good) or 32fp (which is fantastic, but in most cases 16bpp will do).
Since the internal buffer can already handle 8/16/32 accumulation that looks doable.

And again to explaining why it is important - yes you can tweak exposure and everything in Frag.
But even with a lot of additional frags bells and whistles Frag falls short in terms of image postproduction workflow.
To put that straight, If I want to use fractals in a shortfilm and in a regular VFX workflow that assumes several steps:
- rendering good looking images in fractal software
- exporting that + ZD + Transparency
- compositing in postproduction by adding the layers and stuff I want
- finally color grading the final piece

You can check films by Julius Hortshuis which follow the same workflow.
Conclusion is that the image coming from frag is not a final, it's just an input for further artistic interpretation.
And yes, screens are all 8 bit at the end of the pipe. That's exactly why rendering 8bpp from fract sw is not sufficient - you need headroom for all image manipulation. If you input 8bit, manipulate it slightly you start seeing banding, posterization on gradients etc… All bad things affecting negatively the quality of the image we strive for.

Exporting multiple subexposures from Frag does not make any sense. If the buffer is 16 or 32bpp we only need a way to grab it and save to any usable format.
Now the final word on EXR - I suggested it for 16/32bpp precision, yes, byt also because it handles multiple image layers so you could save normal image RGB on one layer, transparency on the other, Z-depth on the other. I saw some people including multiple independent fractal objects within same space. So the next step could be color-tagging these and adding this render element to the stack for future separation in post.

I understand the reasons that make EXR desirable, I did commercial, in-house graphics, animating and video production in the late 80s and early 90s, lots of story boarding and preproduction stuff, so I really do get it. I watched some of your vids on your site, very nice work btw.
I do think HDR/EXR is worth looking at but it's no small task to implement it in a prog like Fragmentarium and we here are mostly pursuing our interests for fun not profit. To that extent, I have downloaded the source code, because I am interested, but it won't compile and they (Ilm) don't provide binaries. If anyone else has compiled it on linux please let me know the trick and I will take a serious look at it.

Thanks for clearing that up  ;D my main point was to help you suggesting a different sw. Sorry but if you can't get what you want (for free), then you should do your homework with what you have. Still I wish you good luck with your job, i had the chance to see your productions and I find them well done too.  :)

Check this plugin:
http://freeimage.sourceforge.net/

Looks like it is avail for Linux and supports many formats including OpenEXR.

very cool, source code is available ;)

Ok, I got a few more ideas overnight.
So QT gives us 2 bottlenecks:
- 8 bits limit
- no transparency.

I think it could be pretty easy and feasible to introduce on a shader level an 'extra pass' that would render the same scene, but introducing color-tagging of various layers.
So lets say we can encode alpha channel on one channel, fractal body on the other and floor on another one. We could also add another pass rendering just AO, shadows or whatever you like.
This gives a huge amount of flexibility for further postproduction.

Now the easiest case is that we render an extra pass of the animation.
But I am thinking along the lines:
- can we extend fragmentarium on a CPP side that way, that in GLSL we could define few tags and outputs (buffers) render them in the same batch and save to separate files?
Not sure how easy it is, but looks doable and making sense.

So we would have a raytracer with predefined set of elements, checkbox enabled in settings - like: fractal body, floor, alpha, etc…
The we would have some flags in the GLSL that would tell Frag to output these, probably defining also filename suffix.
During animation or still rendering, for each of frames Frag would render a few buffers and save to a few files.

Does that make sense?

I have successfully updated the ray tracer to allow for color tagging various layers. So the only remaining question is how tough would it be to add multiple files per frame support on the cpp side.

I think this approach could be used to work around the 8bpp limit. We could tweak the shader to do 2 passes and combine results outside of frag. Time consuming but unless export via 3rd party lines is added this could be a way to go.

Small update on the depth.
Even if we're limited by 8bpp, I think there's a simple way of overcoming this limitation for Z depth.
Clip Depth to 24bits size, then split into 3 channels and write to 8bpp RGB. It's easy to recombine it in post (since there's no better option).

Same trick could be used for having better output for color - if we had multiple files per frame output in Frag, we could easily clip the color from the buffer to 16bits, then split into 2 8bpp images (simple per-channel division should work). Then just multiply 8bpp subframe pairs in post.

While Z-Depth can be added as a separate pass, exporting color passes would already require multiple exports per buffer implemented on a Frag side.

Dick, what is the reason for calculating it that way? i.e what's the reasoning behind "1000.0 - 0.00001" part?


How can this be done? Is that avail on the GLSL side?

Mikael, can you let me know where and how it is decided how Frag buffer is handled?
Do I have control over it on the GLSL side?

Here we go. Looks like we're somewhere. Sharing frags.
I have successfully implemented Z-Depth pass and color-tagging of layers.
In addition, I have implemented experimental 24bit Z-Depth buffer, which would require a bit of processing to decode.
32bpp buffer is scaled down do 24bit and split into 3 8b channels.

To recombine use:
where r,g,b are pixel values decoded from

Trippy 24bpp Depth

And frags - apologies for multiple posts, but there's a 256KB limit on the attachments. At least for newbies ;)

can you encode to RGBA for 32 bit data so...

The internal depth buffer use in GL is an odd creature, if you google "working with GL depth buffers", or the like, you will find the same infos I did when hacking up spline path occlusion, I don't know the details of why it works the way it does, I just know that it works, and had no thought of saving the depth buffer to a file while I developed the spline path occlusion, those numbers are the ones that seemed to get the desired results.

nice to see you are making some progress  :beer:

Cool, but what would I do with it if I cant export it thanks to QT limitations?

encoded as RGBA? = standard qt image, which could be exported?

you can find all Qt image formats here http://doc.qt.io/qt-5/qimage.html#Format-enum

the data structure is directly accessible on a per byte or per scanline.byte basis so if you create an image of the desired size and format you can read/write directly to the storage buffers whatever values you want :D

I'm not sure what you are asking here. But you can control the bit depth of the buffer Fragmentarium is rendering to using the
'#buffer RGBA32F' directive (see e.g. 3D.frag). Notice that this is not related at all to the OpenGL depth buffer resolution (which I never used in the original Fragmentarium).

By far the best solution to all this would be to integrate a library capable of exporting HDR data, such as the OpenEXR library you suggested yourself.


I have written a bit on how to calculate gl_FragDepth here: http://blog.hvidtfeldts.net/index.php/2014/01/combining-ray-tracing-and-polygons/ - notice, that this is ONLY needed if you want to integrate standard OpenGL vector graphics with GLSL pixel graphics (which I don't think you want)

Many thanks Mikael.

The QT 32bit is 32bit total, so it is still 8bits per channel.
I was thinking about 32bit per channel. Or at least 16.
So yes, OpenEXR in the future.

I added RGBA16F too, it renders clean pictures so I'm assuming it works, still saves QImage RGB8
edit: ( in latest source code only)
I have been studying the EXR code and I might be able to manage it, seems relatively simple :hmh: , the plan: save 16/32 float rgbz directly from the GL buffers when using filename extension ".exr" for output images. It will require a lot of ram me thinks...
edit: (not quite there yet ;) )

here is the first (ever?) OpenEXR image from Fragmentarium :D

exrstats

Very nice! The image opens nicely in EasyHDR

Yeah! The dream came true!
Made my day and you rock!

On my end I've implemented simple version of materials with slightly improved and optimized blinn-phong based shading and I'm updating the renderer to deal with transparency and matte objects through the pipeline. It's gonna take a bit more time, but we already see the light!

@Dick - are you able to think about capturing or intercepting multiple layers from fragmentarium to write multiple layers into EXR?
I guess there's only one buffer (hopefully i'm wrong), but perhaps we could make use of 2Dsampler/texture objects to funnel data to EXR?

@Syntopia ty :)  on my system it doesn't look quite right, could be the display/conversion prog but it looks a bit too bright when converted to png and too dark, low contrast, in the desktop display app. (OpenEXR is included as part of the SuSE linux dist). It should look exactly like the default bulb image at startup.

@PK it is only at a very rudimentary stage, saving one tile, they will still need to be stitched together for larger tiled images, I am looking at layers but again only in the simplest of forms. The image I attached previously should be RGBAF so 4 float layers. I have had no luck with depth buffer, I think it's because rendering is done with 2 accumulation buffers then to a screen buffer, the depth buffer stuff that I did for spline path occlusion is only active when in Non-Tiled + Non-Accumulation mode, however the fragment code should still be populating the depth buffer (maybe that needs to accumulate too), but I think it gets destroyed when we render our texture to a quad for display.
Perhaps Syntopia can weigh in with some of the finer points of Fragmentarium's internal rendering flow :) can we use GL_Accum and Qts default double buffers (created for QGLWidgets) instead of the current setup?

My plan is for better colour resolution for printing fractals so I may not take it any farther than that but the end result will be the ability to save EXR images for processing in other software. ;)

EDIT: exr image from GL float buffers, first pic shows Fragmentarium render in background, Okular viewer and png converted from exr
EDIT: second pic is displayed with the OpenEXR viewer :D

Again, easier than anticipated :D
I have set it up so that all the user has to do is to make the output base filename end with .exr and you get RGBA16F images, but they are upside down and don't seem to work with odd tile size, must be even :hmh: so a little tinkering over this weekend and hopefully a fresh release.

Yeah! That's great! Is there a build for that already?
That puts some pressure on me to add alpha channel to the raytracer and I'm almost there.

I inspected the EXR file and it looks as follows:
https://www.dropbox.com/s/6f6niyfjl4b9cdp/Screenshot%202015-08-01%2000.38.04.png?dl=0
https://www.dropbox.com/s/71vmztyq0lzt5li/Screenshot%202015-08-01%2000.37.57.png?dl=0

There are RGB channels and alpha.
Alpha is all 1.0 values.

Does it mean alpha would take other values if present in buffer?

yes, I know what the RGBA file contains, exactly what the GL is using to create the raster image on the screen, it is directly from the rendered texture data, the second pic looks very rich, not washed out up or down, leading me to believe that the viewer that comes with the EXR installation shows a better representation than other progs, the conversion to RGBA8, done GL to Qt to screen, is bound to be a little different than an optimized 16/32 bit viewer.

yes ;)

I looked at writing separate layers but there is a builtin facility for writing RGBA and the internal GL format is RGBA so this seems like a logical first step, as I learn more about the EXR code it will probably be easy to write depth stored in alpha channel as a layer named "Z", so RGBZ, one of the EXR examples demonstrates this, does your VFX software allow for loading/accessing named layers? or "knows" that a layer named "Z" is depth?

I think I've got it! 3ridiculously simple, 2 lines in 3D.frag,  1 line in DE-Raytracer.frag and a few lines added to the EXR tile routine.
is this what a png with zbuffer transparency should look like ???

and the alpha from depth looks like this...

How to connect to EXR layers if I wanted to output from my raytracer?

I am only hooking up the depth as alpha for now...
these adjustments are in reference to the default files 3D.frag and DE-Raytracer.frag in the files distributed with Fragmentarium...

in file 3D.frag...
line 87
line 129 (adds a checkbox in the "Post" tab so we can turn it on and off)
line 185

in file DE-Raytracer.frag...
line 397

in C++ file Fragmentarium/GUI/DisplayWidget.cpp there are a bunch of lines commented out of the getRGBAFtile routine, the whole thing is like this now...

and here are the results... EDIT: should add a float slider to replace 1.0 with user offset in the "depth =" calculation

@PK the next patch release 1.0.12 ? will have this enabled so that you can assign any value to depth and it will be in the EXR file alpha channel.
If 3D and DE frags don't have these changes the alpha channel will be blank so old frags will still run :)

edit: this value may or may not be the real scene depth, I did it this way so I can see it, what sort of values does your VFX software expect ?

ok so it still needs a little work, you will have to tinker with your favourite DE.frag a little but it does work.

Answering your previous question - As long as output is 32bpp floating point, Z-depth can be output w/ out any manupulations.
For my purposes i scale the image so that it reaches 0-1 scale (where white is 1 and it's the closest point to the camera).

hm.. scaling, would be nice to have auto scaling, I have added a depth magnitude slider, but would like to do this on C++ side so no changes will be needed in GLSL frags. I have adjusted so that reflections don't interfere with spline occlusion.

depth to alpha is now handled with a single define
in user fragment, text editor, all that is needed is to add "#define DEPTH_TO_ALPHA" line before the raytracer include, this will add a checkbox to the "Post" tab, this is only useful if you intend to post process images with software that knows what EXR format is before assembling the animation.
This does not work for any other format, EXR only.

Cool! I think there's still something not exactly right with the EXR output. Looks like a massive gamma shift to me.

0> screen shot from Fragmentarium window
1> screen shot from iv program showing exr file "as loaded"
2> screen shot from iv program ALPHA channel only from exr

viewed with "iv" program from OpenImageIO package
edit: the image(2) is off register because I did a sloppy clip from screen, the image(1) looks a bit lighter, but again, clipped from screen from "iv" program so it is probably calculating the display from RGB+A channels, when flipped to show only RGB it is the same as the screen shot. Still not a huge difference.

3> this is displayed with the desktop viewer which I don't think is accurate
4> "iv" program with Exposure = -2.0 and Gamma = +2.0 with Exposure = 0.0 and Gamma = 1.0 (default in iv?) the image displayed looks the same as Fragmentarium view
5> saved as png from "iv", viewed with desktop viewer and screengrabbed