Help required with antialiasing and the median filter

[zenzero-2001]

Hi

I have been experimenting with antialiasing in my forthcoming fractal explorer. I have been using HPDZ's median filter, as described here: http://www.hpdz.net/TechInfo/AntiAliasing.htm and as discussed in this thread: http://www.fractalforums.com/programming/antialiasing-fractals-how-best-to-do-it/

I have got some very good results, especially with the Burning Ship fractal where it has been quite a revelation.

Unfortunately antialiasing of pixels near the border does not seem to work. This is very obvious for example, when looking at the whole Mandelbrot set. I assume that it doesn't work in these situations because many of the sub-sampled pixels will be at max iterations (or marked as periodic - I use INT_MAX) and this will affect the filter result. If the number of samples is odd I take the middle sample (after sorting). If the number of samples is even I take the mean of the middle two samples. With even samples, if the second middle sample is outside the set, I take the first middle sample as my median and do not calculate the mean.

I'm not sure if this is the best way to handle the problem. But the antialiasing doesn't seem to work near the border with either an even number or an odd number of samples. Should I be using a different strategy in these situations or maybe even a different filter? Or am I doing something wrong?

Thanks,

zenzero-2001

[Adam Majewski]

 "antialiasing of pixels near the border"
Near the border of image or border of the set ?

[zenzero-2001]

Hi,

Near the border of the set.

I think the problem could be because of the wider variation in the iteration counts near the border of the set. Although reducing the colour density does not seem to help.

I haven't really looked at it since I posted, have been a bit busy and under the weather.

[hobold]

I strongly suspect that for pixels overlapping a high contrast border, the median filter will not be effective for anti-aliasing - except in the isolated case that there is an even number of samples, exactly half of which are "inside" and the other half "outside".

The exceptional case does what is needed here: average the colour contributions from inside and outside the border, according to the number of samples that land on either side.

[zenzero-2001]

Thanks for the reply hobold.

I'm not sure if I understand your second paragraph. Are you suggesting I should use the mean filter for pixels that contain subpixels that are both inside and outside the set?

[hobold]

I'm not sure if I understand your second paragraph. Are you suggesting I should use the mean filter for pixels that contain subpixels that are both inside and outside the set?

Short answer: yes.

Long answer: anti aliasing is supposed to remove/conceal/suppress artifacts that arise from the discrete sampling of a continuous signal. If the contrast between the inside and the outside of the fractal is high, then those artifacts are more pronounced as well.

There may be cases when a particular filter is theoretically the correct way of removing artifacts, but another filter results in a nicer looking image. You will then have to make a choice between correctness and appearance.

[Syntopia]

I must admit that I don't get the median-filtered AA approach.

Take the case where you try to depict the a Mandelbrot set without colors, just black for inside and white for outside.

So for each pixel you collect a set of samples in the the neighboor of the center of the pixel (often choosen in the square pixel area).

If you try to find a single representative sample (the median) for your set of samples for the pixel, naturally all resulting pixels in your output will be either pure black or pure white. This means borders will be pixelated (alised). In this case, it doesn't matter if the median is choosen in parameter space (e.g. smoothened iteration count) or color space. I did this in the upper image, where I took 300 samples for each pixel, and chose the median (which happens to be the most frequent color in in this particular case).

On the other hand, if you calculate the average of the colors for your subsamples, you get a grey scale image with much more detail. Take a look at the bottom picture. Again 300 samples: samples were averaged with a triangle filter weight based on their distance from the center (but using an unweighted average won't change much). Gamma correction should be taken into account (since light intensities are not linear when sent to the frame-buffer), but this will only change the contrast (this is the case when the input colors are pure black and white).

[zenzero-2001]

Thanks Syntopia and honold, you gave me some ideas to think about.

I did some experimenting with the mean filter. The output does not look as good as the median filter and it doesn't seem to help too much near the border of the set.

Earlier I said that reducing the colour density didn't help, but on second thoughts I think that it does. Unfortunately, this has the side effect of desaturating the colours that are further away from the set. I think that using a coloring strategy similar to HPDZ's histogram coloring or rank order coloring could help here. I will have to do some investigating. Presently, I am using the same coloring strategy that Ultra Fractal uses and I obtain very similar outputs.