00019 1.14e382 WIP (now with glitches)

[stardust4ever]

Full view:
http://stardust4ever.deviantart.com/art/00019-1-14e382-WIP-490703526

Still frame from near the end of an upcoming video, "Simply Awesome Revisited." This is a fresh take on my first ever deep zoom Mandelbrot video, dating back to 2009 on my 32-bit laptop. The updated version will be complete and zoom twice as deep.
<a href="https://www.youtube.com/v/JlCupXjP4vY&rel=1&fs=1&hd=1" target="_blank">https://www.youtube.com/v/JlCupXjP4vY&rel=1&fs=1&hd=1</a>

Frames will be rendered at 3840x2160 and scaled to 1080p Full HD Anti-Aliased. Final depth is 3e387 magnification or approximately 1287 zooms. Rendering is in progress and being done in Kalles Fraktaler, currently 68 hours in.

[Kalles Fraktaler]

Yeah, I just finished rendering the frames to your Quasi perpendicular location in 1920x1080 on my 7-year old 32-bit dual core laptop. It took a couple of days and there were no issues...

You bet I wish I knew about perturbation decades ago...

[stardust4ever]

Yeah, I just finished rendering the frames to your Quasi perpendicular location in 1920x1080 on my 7-year old 32-bit dual core laptop. It took a couple of days and there were no issues...

You bet I wish I knew about perturbation decades ago...

You rendered the whole thing? You really didn't have to do that. I just wanted to see if you could reproduce the glitch. The fact you rendered it on a 32-bit machine meant you didn't use the same 64-bit build I did (2.7.2). It may well have gotten fixed, I dunno. I ended up rendering the Quasi perpendicular zoom at 360p native but still produced a passable 720p video.

Currently my render progress on the "Simply Awesome" render is taking longer glitch correcting than actual rendering. One wierd thing I noticed was the first frame took over 12 hours (not surprizing) then about twenty or so frames later the time interval was getting closer to 30 minutes per frame, and then one frame took nearly 12 hours, and subsequent frames were ~30 minutes. They were pushing ~10 minutes per frame last night before I went to bed, but yesterday evening another frame was 3-4 hours. Why do some frames take so much longer to render than others adjacent to it? No matter really as once I hit below E300 or so I'm going to walk in and check my PC and it will all be done.

I remember a similar problem with Mandel Machine (which is fraster than KF btw). I was zooming to a region at thousands of zoom levels. The images were appearing within seconds as I was trekking through the spaggetti trying to reach my destination. As I approached the destination pattern which was a highly sophisticated Juila, rendering slowed down by about an order of magnitude. Once I zoomed past the pattern, progress sped back up. Weird.

Truth be told, had I been using Fractal Extreme, every pixels would have taken the same time interval as the orbit calculation. I have spent a month rendering a 3200x3200 image in Fractal Extreme (Magnum Opus Ex), which took a mere 4 minutes to do the same location at 12800x12800 in Mandel Machine. So perturbation rendering is truly an awesome breakthrough. I'd say just as significant as the development of 64-bit processors. It would be nice if you could get KF a little more optimized though, but it's just amazing you were able to include all my formulas.

[Kalles Fraktaler]

Yes, in very deep locations the orbit takes a sinificant amount of time to be calculated, and if many additional references needs to be added to solve many small scattered glitches, it will take time.
I had a look at your original movie and it is typically producing many small 'unconnected' glitches, i.e. the distance is more than some e18 and aren't solved from ref.points outside them. Since it has many embedded distant Julias.

Another thing is that the more dense pattern, the less effective the series approximation is.
If, for example, the lowest iteration value is one million and 2, and the highest one million and 10, and if approximation yields one million to be able to be skipped, the view will be rendered much faster than if the difference between these values are thousands or millions.

Also, KF's ap library (mine) is slower than FX's - MM's is faster.
MM is utilizing SIMD, so it can calculate multiple double operations in parallel - in each parallel thread.
MM have also further optimizations, which I'm not sure can render all locations (at least it has been so)
It takes unfortunately too much time to do these optimizations in KF so I'm not going to do it. Others may use KF as a starting point, that's why I'm also sharing the source.

[stardust4ever]

Yes, in very deep locations the orbit takes a sinificant amount of time to be calculated, and if many additional references needs to be added to solve many small scattered glitches, it will take time.
I had a look at your original movie and it is typically producing many small 'unconnected' glitches, i.e. the distance is more than some e18 and aren't solved from ref.points outside them. Since it has many embedded distant Julias.

Yeah it's not just the orbits but sometimes calculating the pixels themselves (with perturbation) takes longer too.

BTW, my new zoom video zooms right up to the final minibrot around 2^640 / X E192, and then into the seahorse valley of the mini, continuing onwards until the final minibrot at 3E387.  The idea behind it is giving the viewer the illusion that the zoom video is about to end, then continuing onwards and finishing at a point roughly twice as deep. That's why I said a small amount of horizontal panning at this location would be ideal if I could make it work seamlessly. Other than that, the zoom path is unchanged, just a continuation of the first.

What I've seen so far in the reverse order frames Kalles Fraktaler is generating, is the density gets lower, lower, lower, then hits the insanely dense Julia formations created by the original minibrot. Zooming in, the iteration depth literally gains approximately 100,000-150,000 iterations in a couple frames time span, then the density drops quite dramatically. Going backwards, frames are rendered faster and faster as the density drops, then suddenly hit a brick wall in terms of speed, and slowly creep up again.

I'm not sure what the ideal upper limit setting for orbit calculations is on KF. I do fell, however, that 69 orbits is generous. Even at 3840x2160, I'm not seeing any visible holes in the completed images even if the software isn't completely finished every pixels. After being anti-aliased to 1920x1080p frames, I seriously doubt any unfixed glitch pixels will be detectable in the final video presentation.

[Kalles Fraktaler]

You can change the value of 69 additional references to 0-200.
And if you open the iteration dialog while rendering, the max/min/approx are updated in the dialog.
Then you can see that not only does the difference between min and max increase in dense areas, also the approx value decrease which makes approximation far less effective. And the approximation in KF is actually better than in MM (perhaps the only thing (currently) more effective in KF than MM )
I usually have this dialog visible while rendering, so that I'm not accidentally click in the window and break the rendering...

[stardust4ever]

You can change the value of 69 additional references to 0-200.
And if you open the iteration dialog while rendering, the max/min/approx are updated in the dialog.
Then you can see that not only does the difference between min and max increase in dense areas, also the approx value decrease which makes approximation far less effective. And the approximation in KF is actually better than in MM (perhaps the only thing (currently) more effective in KF than MM )

All of the images so far look fine even zoomed in. I haven't been able to find any visible glitches in the Jpeg images even when zoomed in with Windows photo preview. A couple of pixels with the wrong color in the gray rats nest areas ain't gonna matter after the image is scaled down to 1080p and placed into a zoom video.

I usually have this dialog visible while rendering, so that I'm not accidentally click in the window and break the rendering...

Good idea. If I happen to screw something up by accident (stray click, computer glitches/crashes, power failure, etc), how should I go about resuming the process without redoing the previously rendered frames? Just asking for future reference.

[Kalles Fraktaler]

Good idea. If I happen to screw something up by accident (stray click, computer glitches/crashes, power failure, etc), how should I go about resuming the process without redoing the previously rendered frames? Just asking for future reference.

File->Resume Zoom sequence

Not only does it take the previous rendered kfb and place it in the center of the current frame.
There is also a recovery.kfb constantly being updated and removed when the render is finished, if this file exists it opens it and continue with it instead.

This is because we have fractal friends in countries with unreliable power supply (Ukraine)...
We are few, but spread all over the world! (Sweden (me) - New Zealand (panzerboy?): opposite spots on the planet!)

[stardust4ever]

That's good to know. I've got an uninterruptible power supply (~350w rated) attached to my desktop. Due to the high load (~250w typical), there's only about 10 minutes of reserve power with the computer running at max capacity. So I configured the computer to hibernate immediately when the supply drops to 80% battery reserve instead of default 10% like on laptops. If the power only cuts off for a couple minutes, it will keep running, but if the outage is longer, it goes night night. I tested this once pulling the plug and it successfully went into low power state. My Windows 7 supports hybrid sleep, so it always backs the 16 Gb RAM to disk when sleeping. It just takes a few moments longer to resume after a total power failure.

Also with the internet router hooked up to it, I should get a few hours to browse the web on my laptop also. It doesn't happen often, but nobody wants to lose work.

[stardust4ever]

Hmm, the render job finally finished yesterday afternoon after running 133 hours or so. I found a couple of glitches in the video frames. Is there anyway to repair these?





The first image with the minibrot is kinda knarly, but the second actually looks pretty cool, if technically incorrect. Weird how the glitch features are a bit fuzzy, as if some sort of Gaussian noise distribution was introduced to the iteration values.

[Kalles Fraktaler]

There is an 'Examine Zoom sequence' function under the 'File' menu.

Did you uncheck the checkbox 'Approximation low tolerance' in the Iteration dialog?
Otherwise you may have encountered a new problem, and if you don't mind the location would be valuable to me.

You need to find the frame, by its number, unfortunately backward counted, and re-render the whole frame (Action->Refresh) and then click to add references.

[stardust4ever]

There is an 'Examine Zoom sequence' function under the 'File' menu.

Did you uncheck the checkbox 'Approximation low tolerance' in the Iteration dialog?
Otherwise you may have encountered a new problem, and if you don't mind the location would be valuable to me.

You need to find the frame, by its number, unfortunately backward counted, and re-render the whole frame (Action->Refresh) and then click to add references.

As for the checkbox, I have no idea but it was probably left at it's default value.

Code:

Real=
-1.999774068234095245827897264052411635101178617311214911823844762794970374009932407802133036916959822776036761564910985129461406226597915032748262794453209798251808826849795275766849806523622180300960808177976027146569242775483384790833054502799415817959872387439457290590555783095517149084698645864978742571808705946974528133610159111119093196209760035126186963769605887973284860131245303250300195634072815058342792745
Imag=
0.000000017118280986851973264814230660100962431024078197327092249448448716558397421331807246460464451548687976062474389662420867675913639632356385505294432056044492647145894341948023900899252276937730358154392137754078328630545287259510549665484244322466598661312899479156647349903899143642747729310081316661944823432036661726198488364197513069782083377067938616973354251492293076920231591994281366750321615428491013165

Frame size = 3840x2160

Affected frames:
Glitch 1 = 00957_4.92e99, 00956_9.85r99, 00955_1.97e100, 00954_3.94e100
Glitch 2 = 00809_1.75e144
Final frame = 00001_3.00e387

If you want I can upload the affected kfb files to MEGA in a .7z archive.
EDIT: PM sent!

On a side note, had I not saved the Jpeg sequence, I wouldn't have seen this until after I rendered the video. I'm currently rendering the video with color cycling set to .125 (the palette has 1024 colors using wave coloring and bands are fairly dense) and the rendering process is far slower than normal, taking several hours, even though I'm only rendering to 360p low quality to test the sequence. I plan on adding variable speed later and the final zoom will be full 1080p.

[Kalles Fraktaler]

You changed the value of additional references right
To what value, did you set it to zero?

[stardust4ever]

You changed the value of additional references right
To what value, did you set it to zero?

Um, I think it was 69.

[Kalles Fraktaler]

Um, I think it was 69.

Yes it probably was.

I tested your location at 1.75e144 and it worked at 640 but at 3840 I get the same.
And it is because approximation.
If you turn approximation off (Actions->Special->No approximation) it work, but of course takes longer time.

I should lower the tolerance even further, at least on higher resolutions.