Welcome to Fractal Forums

Fractal Software => Mandel Machine => Topic started by: Botond Kósa on October 15, 2014, 09:31:37 AM




Title: Version 1.3.3 is available
Post by: Botond Kósa on October 15, 2014, 09:31:37 AM
A small update is available containing a new feature: iteration data can be saved in an uncompressed format for further processing.

The saved .mmi files have the following structure:
<image width>: 4-byte integer
<image height>: 4-byte integer
<array of iteration values>: 8-byte double precision floating point values, in top->down, left->right order

Some of the double precision iteration values have special meaning:
  • Negative values: the value was guessed based on the nearest non-negative neighbors using interpolation.
  • Values exceeding the iteration limit: these pixels were identified as glitches by Pauldelbrot's method. They can also be negative, meaning that the pixel is in the middle of a glitch and was guessed.
These values serve primarily debugging purposes. For an image that was created with auto glitch correction on, it is simply enough to take the absolute value of the raw iteration values.


Title: Re: Version 1.3.3 is available
Post by: Dinkydau on October 16, 2014, 01:20:09 PM
It sounds useful. I wonder how this works. What exactly do the double precision iteration values represent? I would expect iteration data to consist of integers, being the numbers of iterations at which the pixels exceeded the bailout-value.


Title: Re: Version 1.3.3 is available
Post by: Botond Kósa on October 16, 2014, 02:29:44 PM
What exactly do the double precision iteration values represent? I would expect iteration data to consist of integers, being the numbers of iterations at which the pixels exceeded the bailout-value.

The double precision values contain the normalized iteration counts used for smooth coloring, as described here:
http://en.wikipedia.org/wiki/Mandelbrot_set#Continuous_.28smooth.29_coloring (http://en.wikipedia.org/wiki/Mandelbrot_set#Continuous_.28smooth.29_coloring)

If you want to know the number of iterations at which the pixel exceeded the bailout-value, just truncate the double value.


Title: Re: Version 1.3.3 is available
Post by: SeryZone on October 23, 2014, 07:48:48 AM
duh... When I load palette, MM automatically closes. + my primitive SFT Map Visualizer won't load iteration data :-}


Title: Re: Version 1.3.3 is available
Post by: Botond Kósa on October 23, 2014, 09:08:29 AM
duh... When I load palette, MM automatically closes.
Could you send me the file that causes MM to crash?

+ my primitive SFT Map Visualizer won't load iteration data :-}
Did you try to load an .mmit or an .mmi file? The .mmit is heavily compressed, and it is only used to load into MM later. The .mmi is the public uncompressed format. The exact specification can be found in the first post of this topic. Keep in mind that the format is different from the .kfb files.


Title: Re: Version 1.3.3 is available
Post by: istinn on November 03, 2014, 01:26:25 PM
duh... When I load palette, MM automatically closes. + my primitive SFT Map Visualizer won't load iteration data :-}

same here, java7, java8 x64, when loading image, image shows under but previous palette does not get replaced or change, by clicking inside it just crash.
crash log attached


Title: Re: Version 1.3.3 is available
Post by: claude on December 15, 2014, 12:05:42 AM
I guess all the data is little-endian?

top->down, left->right order

Is this the right order (same as PPM)?  Natural language is so ambiguous... (or maybe it's just me)

Code:
for (int j = 0; j < height; ++j) {
  for (int i = 0; i < width; ++i) {
     int k = j * width + i;
     // array[k] contains pixel at (i,j) with origin (0,0) at top left
  }
}

Kalles Fraktaler seems to use k = i * height + j, which makes swapping the order of the loops more cache-friendly...


Title: Re: Version 1.3.3 is available
Post by: Botond Kósa on December 15, 2014, 12:29:53 AM
I guess all the data is little-endian?
Yes.

Is this the right order (same as PPM)?  Natural language is so ambiguous... (or maybe it's just me)
I am not familiar with PPM format. The data in .mmit files is stored in the usual order, in rows from top to bottom, and from left to right inside a row.


Code:
for (int j = 0; j < height; ++j) {
  for (int i = 0; i < width; ++i) {
     int k = j * width + i;
     // array[k] contains pixel at (i,j) with origin (0,0) at top left
  }
}

Kalles Fraktaler seems to use k = i * height + j, which makes swapping the order of the loops more cache-friendly...
Is that code from Kalles Fraktaler? It iterates through the pixels in the same order as MM. Array index k can be calculated either as j * width + i or i * height + j, it makes no difference regarding cache-friendliness as long as i is incremented in the inner loop.


Title: Re: Version 1.3.3 is available
Post by: claude on December 15, 2014, 12:46:56 AM
I am not familiar with PPM format. The data in .mmit files is stored in the usual order, in rows from top to bottom, and from left to right inside a row.

That's the same as PPM.  Kalles Fraktaler has the data transposed (columns from left to right, and from top to bottom inside a column) so it can use C nested array syntax in the intuitive order array[ i ][ j ].

Quote
Is that code from Kalles Fraktaler?

No, I just wrote it to show what I understood you to mean by top to bottom, left to right.

Quote
Array index k can be calculated either as j * width + i or i * height + j, it makes no difference regarding cache-friendliness as long as i is incremented in the inner loop.

j * width + i isn't equal to i * height + j, so using the wrong one garbles the image.  cache-friendliness is improved when the non-multiplied variable is incremented in the inner loop.


Title: Re: Version 1.3.3 is available
Post by: Botond Kósa on December 15, 2014, 07:19:22 AM
j * width + i isn't equal to i * height + j, so using the wrong one garbles the image.
It really does, it must have been too late at night when I wrote that :crazyeyes:

cache-friendliness is improved when the non-multiplied variable is incremented in the inner loop.
Unless the compiler is smart enough to apply strength reduction (http://en.wikipedia.org/wiki/Strength_reduction) and eliminate the multiplication by width or height entirely.


Title: Re: Version 1.3.3 is available
Post by: claude on December 16, 2014, 01:00:36 PM
The saved .mmi files have the following structure:
<image width>: 4-byte integer
<image height>: 4-byte integer
<array of iteration values>: 8-byte double precision floating point values, in top->down, left->right order
...
Values exceeding the iteration limit

This makes it impossible to find out the iteration limit from the file.  I need it to write the corresponding mmittokfb, here's the commit with kfbtommit:
https://gitorious.org/maximus/kf-extras/commit/fd9d1e4071df4f510d2d7c9e88ccbe3d8e1fe935
http://code.mathr.co.uk/kf-extras/commitdiff/fd9d1e4071df4f510d2d7c9e88ccbe3d8e1fe935

Could this be added in a future version?  It's quite crucial to know for colouring images with interior regions...

If you do change the file format, please make it extensible - something like this perhaps:

file:
<magic number>: 4-byte integer (perhaps "MMIT")
<header length>: 4-byte integer
<header> header-length bytes, defined below
<array of iteration values>: as before

header:
<version>: 4-byte integer - bump the number if the header changes incompatibly (no need to bump if appending new fields while keeping the meaning of the existing fields)
<width>: 4-byte integer
<height>: 4-byte integer
<iteration-limit>: 8-byte double (or something else sufficient for what mandel machine can render)


Title: Re: Version 1.3.3 is available
Post by: Alef on December 19, 2014, 05:39:49 PM
Since only 2 2D are being developed right now...
Maybe test this:
http://www.fractalforums.com/new-theories-and-research/colours-gradients-and-their-interpolation/msg79162/#new (http://www.fractalforums.com/new-theories-and-research/colours-gradients-and-their-interpolation/msg79162/#new)
Maybe this would work nicely in aplication. Maybe not;)


Title: Re: Version 1.3.3 is available
Post by: Botond Kósa on December 19, 2014, 09:24:10 PM
Since only 2 2D are being developed right now...
Maybe test this:
http://www.fractalforums.com/new-theories-and-research/colours-gradients-and-their-interpolation/msg79162/#new (http://www.fractalforums.com/new-theories-and-research/colours-gradients-and-their-interpolation/msg79162/#new)
Maybe this would work nicely in aplication. Maybe not;)
Do you mean the color palette? Or the function used to interpolate the gradients?


Title: Re: Version 1.3.3 is available
Post by: Botond Kósa on December 19, 2014, 09:57:22 PM
This makes it impossible to find out the iteration limit from the file.  I need it to write the corresponding mmittokfb, here's the commit with kfbtommit:
https://gitorious.org/maximus/kf-extras/commit/fd9d1e4071df4f510d2d7c9e88ccbe3d8e1fe935

Could this be added in a future version?  It's quite crucial to know for colouring images with interior regions...
True, somehow I forgot to include the iteration limit. Anyway, you could find it out by searching for the largest value in the file:
  • If it is an integer and divisible by 4, it's safe to assume that it is the iteration limit and the image contains interior pixels.
  • If it is 4k+1 (where k is an integer) the iteration limit is 4k and you've found a glitch pixel (the image was calculated using perturbation algorithm with glitch detection turned off).
  • It it has a fractional part, the image contains exterior pixels only.


If you do change the file format, please make it extensible - something like this perhaps:

file:
<magic number>: 4-byte integer (perhaps "MMIT")
<header length>: 4-byte integer
<header> header-length bytes, defined below
<array of iteration values>: as before

header:
<version>: 4-byte integer - bump the number if the header changes incompatibly (no need to bump if appending new fields while keeping the meaning of the existing fields)
<width>: 4-byte integer
<height>: 4-byte integer
<iteration-limit>: 8-byte double (or something else sufficient for what mandel machine can render)
Thanks, I will implement this format in a future version.