Welcome to Fractal Forums

Hi Kalles

There is a bug in the version 2.9.7 of kalles fraktaler 2.
When I click on "auto solve glitch" with "examine zoom sequence", the problem below appear.

most/all authors of perturbation programs i've seen are using default settings which favor speed over accuracy, probably even more so for zooms.  if you want more accuracy there is probably an option somewhere where you can raise the max number of reference points and such?

Interesting.
Can you send me the location?

Re: -1.296355138173036216808304477845282266265735748447889106496038985761209917380841177445968518215701758678084105701798781194955286791090159889479830250582998616337343672293770610499414537932319513273091525519480957820496859278286357694764845398638622284835930478246909426978498903001292186349332559002364119692765842997347411680020529930794430297889475200868675134070916915029352018955671209679653588455096044586235738194335823043789630623088453676657949865089362217637125034542974484819892882518624751323660120500136532115644867205611992160754276675349948316142781488215964382264146492785725093186522485081128928485363264599339734251203063880701539610046638303626795735718966970641220962923079472132879218084171429125502639157779993043432103141243971131090219953825914905004024872001745764432019897994637060100741704114438981666498439414773127273768682855615943049460249321102817802315841571856105883956688597465218149939352580623597585689908462847383648235816382580470517645800446361062024466323256389226230312616351187419538060225167736803525145232562199571699713437958091003679794587041876469868465856426563926212975296854780647056242705957785494023937014971169623443261919352966951619778850300586466326970568706005442490399114414710530104463493125608296913819151789916280880724413432577930767002163323557996698828262572794499999194869018065415227752694458414200815408162608128472937332894712532633236438374064168748491036172167649190381488090168987338631867626142176553092393627587023711821699750838956897945647744877403406483850925618315252953328987436142781151229876098476397533207189895153754771856587049648790114547938829260964489740063313214749090993166989912043877739160463422052860270972469115318915701047171917880990618003083436720660633950048613362270852364674467190551511460960744443762947808490866715105893507673957973164746095058432375554387135706903601439292278884911936421548663220799245064998695604164093487261179349347853800540569072644950925270588311758385070983686862244455927350856123325710859816597565337333091569397234250743592576434389631290690505531799777246836999955179257209426476184953422200993585637112614339598607545
Im: -0.441851605735196601212848580174986948840096866702890580517816641664137309219631137124274552377733978128686419957669157494088039035922351119632811221778713954178086193458861028284209543020285018565769515482463245803379644163301622266580558324633885400684383926664648595707250845941186242632516809899396314190876372172913204206324576438201859918119458401143921495705079306772286572229419308265992148297977456492889986116205742645363993821848697882610709906258793336239004842943906986210736820978592084207290666236555957482963656063549010444105123098070134336471987534785202927629973161950511592581427203797488045441440026238035076135625264861339753279100714891372407515853098284627171802695919672630737106960344603522635273210944830171135346252462537703505997818763885988466869217773916646359441140614304590841814645837510230426286057813658976772992338117014640045672814945221795097188356817324852849480363813371689059659232967175321254910109114494920069754633824103731891628134989201361537759392931257569337517543481361841545238865829088485855850404668357967323401050343944511551794864978777250158803946591708326816355481561539615842405016841201060211482818918114785749499657615064702167274001250041088644742208013944449418954651669543291476924409842045542521333491935816940422301647922356137533118613929599056439398381818098344974479177393005542332755082083163150701450215288991169921779016395219072435561522108487241990391382526569922293685620041733671097121849429464487621794520020167605266610614577515828657827352106545315040578420525657755525972854022796637807349147592268281748367517361119005099898668920219185722517400901384616477647235736032139186840880432015693914842914632869607777328404595380956495243408171754158478726042799312293183947848336579629235798589776678289925280075378662568941327347464781202650604139064414786204594121158855777979536943325628171806748815008807348214362171459702970449699336203605053062288484913989563224164570214316235959408550401925546639489047738690869625062858315750966130296354830068847624374499751573173110055245096965702203820751345879478548223084264302424284854828105269816428496557088832634046948605
Zoom: 1.49575125084E2126
Iterations: 40,000,000 (minibrot not enough defined), the video will have 100,000,000 iterations.

All keyframes are calculated without auto solve glitches. I prefer render keyframes before, and auto solve glitches after.
picture above: Zoom = 8.38E1028

However, the bug is the same for any location with glitch zone.

Here is an example of the worst glitch of this type.
the 2 references + 1 added is a simulation of keyframes after autosolve glitches (always 10 references).

Crap happens...

I have some glitched frames in existing video (simply awesome II HD):
http://sta.sh/01aqqf7kg83r
http://sta.sh/017skb4ouloa

I gave up trying to fix frames and simply upped it directly to YT as is. :hurt:
https://www.youtube.com/watch?v=vnUjXAIIcq0

The auto correct glitches in the examine function is only adding a limited number of additional references, 5 or 10 I think?
I can see that the areas are filled with previous view, i.e. not updated, so they are at least identified as glitches.
One issue that stardust and some others have encountered is that glitches are not identified because of Series Approximation skipping the point where the glitch occurs.

I rendered the location at 8.38E1028 in 640x360 and it needed 10 references...
I guess you rendered in much higher resolution, so you would need more references than the perhaps 10 that the examine function adds.
Also, these types of references, the spirals from passing the elephant valley, are requiring many additional references in KF, which probably could be improved a lot if only the center of these spirals could be found in advance.

Can I, semi-on-topic, ask whether there's a way to set the settings in such a way that glitches are highly improbable for even lengthy zooms?  Even if at the expense of the rendering speed?  Like maybe tying the number of references to the depth, and having it overcompensate by some multiple?

The example you gave, a 640x360 picture needing 10 references... that's 1 reference per 23,040 pixels.  Would the rendering speed not stay way ahead of most of your competition even if for the given depth and image size you used 50 or 100 references?  And with settings like that or maybe even higher--but still nowhere near calculating even a 10th of the image fully in arbitrary precision--would the likelihood of glitches get low enough to no longer worry about?

Or am I oversimplifying the issues?

As long as high bailout is used, I haven't encountered one location where pauldelbrot's glitch finding method doesn't work. The issues we rarely find are due to series approximation, which could be checked against some pixels inside the view and not only on the edges, but that would be slower.
So, the glitch finding method is reliable enough I think

Actually, I compute my keyframes with only 1 reference and the "reuse reference" button locked. at this stage, the glitch areas are normal. I auto solve glitches after (without "reuse reference").
I am noted empirically that areas with the previous view are the max iterations value from 1 to 9 references, but after the last reference, areas are sometime preserved the value, or sometimes a glitched values.
Therefore the bug is at the last reference.
When it stays a visible glitch areas, I begin a second passage of auto solve glitches. So the conservation of the iterations value inside areas is necessary.

With the 2.6.2 version, there is another problem. Auto solve glitches is working normally. But sometimes, one reference (for example the 4th) solve no pixel, the program add the next reference at the same pixel, solve no pixel, and stuck on the same location until the last reference.
With a reference added manually, the problem is temporarily solved until another reference that solve no pixel.

I think I set the last reference without identifying glitches, because I thought the glitches would be so small they wouldn't be noticed on the last reference...  :embarrass:
Maybe I should change that, and remove the limited number of additional references?
Yes, since 2.6.2 I added a list of added references, so that the program doesn't try to add it on the same pixel again :)

You can leave the number of references (10 is a good number), but with the identifying glitches for the last if it's necessary to make a 2nd (and 3rd) passage.
And continue to reuse center if no glitch is found.

Edit : I tested another auto solve glitches with 2.9.7 version. Sometimes, the glitched iteration values appear in one part of glitch areas before the last reference.

I tried the automatic glitch solver in the latest version, 2.10, and it is able to add more references in the second round...

So, still working reliably after a lot of battle testing? That's good to know.

P.S. How does KF do as much with series approximation as it does? It often seems to get 60-70% of the iterations, vs. nanoscope's 20. Does it use more terms? (How many?) Or is it because it's checking against the image edges rather than just checking for the error estimate to get to a certain threshold and quitting there? (That might allow being less conservative.)

Yes, more terms are used, starting with 10 for 640x360, more for larger views e.g. 60 for 3860x2160, down to 5 for small glitches.
And it is checking against 8 points on the edges (centers and corners), these points are calculated with perturbation from iteration 0 while the approximation is calculated and each iteration is compared for a threshold of 0.00001 for high tolerance and 0.001 for default low tolerance.
Also, my own class floatexp is used for the approximation algorithm instead of standard double datatype.
That is because most of the terms easily reach out of bounds for the double datatype, but later in the process these terms may grow and influence the result (chaotically)

I'll note that for later.

Meanwhile, on the topic of redoing the same point over and over in an endless loop, Nanoscope's solution to this is very simple and doesn't clutter up memory with a list of pixels previously used as references: when it picks a glitched pixel and calculates a reference there, it also uses that iteration to color in that particular pixel. Combined with it specifically selecting a glitched pixel (even if the glitch has a hole, and the barycenter is a nonglitched pixel), this guarantees it won't try the same pixel twice, and that it will eventually finish the image, by hook or by crook.

Hello, after a long time without Kalles Fraktaler, I downloaded Kalles Fraktaler 2.11.1 GMP. I test to render a zoom sequence with many parturbation glitches. With "Auto solve glitches" checked while rendering, there isn't bug, but with the function "Examine zoom sequence → Autosolve glitches", the same bug since 2.9.7 version appear. I tried to recompute entirely a frame with this function, I noticed that Paudelbrot method doesn't work when the bug appear.

try to render this location with "reuse reference" checked, and try to solve glitches with "Examine zoom sequence" and "reuse reference" unchecked. It take about 5-10 minutes.

Yep, seems that pauldelbrot's method of detecting glitches is not bullet proof after all.
Also this move is based on a location where the glitch is not detected. The movie begins with the image rendered in mandel macine and there are some small pattern in the middle of the four structures that is lost when rendered in kf. However on the large image on his deviant-art page shows that also mm renders this part distorted.
https://olbaid-st.deviantart.com/art/Deep-Mandelbrot-Set-023-10-1086-695492737

Seems that when the main reference has "contact" with a deeper minibrot parts from another minibrot can be distorted. This phenomen gets more common now this the extremely effective newton-raphson method of zooming.
I know how to solve this but it could make kf slower:
* have a lower threshold for glitch detection.
* not add the main or any reference in the center of the image, only in centers of glitches.
* move the result after a newton-raphson zooming by a fraction of a pixel - so that the "contact" with the minibrot is broken.

I think the last suggestion would be most effective, so if claude reads this, this is my best suggestion.

http://www.youtube.com/v/2tojn_ax_-8&rel=1&fs=1&hd=1

I just tested to render the same location (Store Zoom-out images) with auto solve glitch checked. The bug never appears.
When I render without auto solve glitch, The bug appears only when I solve glitch with "examine zoom sequence".

I have a deep zoom project to 10^2126 with all frames rendered without solve glitch (many glitches), and I don't want to solve by hand.

I can confirm the bug in 2.12.2+git (unreleased) with examine zoom sequence auto solve glitch.

Steps to reproduce:
* start kf.exe
* load the settings file posted by Fractal universe
* select "reuse reference"
* save zoom out sequence (doesn't take long at default resolution 640x360)
* deselect "reuse reference" or restart kf.exe
* select "examine zoom sequence" and load the previously saved sequence
* select "save and previous" to go to last (deepest) frame
* select "autosolve glitches"

What should happen:
* glitches are solved in all frames from the current until the first (least deep)

What actually happens:
* glitches are solved in the deepest frame only, nothing happens on other frames (including when advancing automatically after solving glitches on the last frame). the timer is running and it looks like kf should be working, but CPU usage is close to 0

Possible causes (speculation, there may be others too):
* some kind of timing race condition or logic error between the glitch solving code and the KFB loading code, so nothing happens when not on the deepest frame
* some kind of logic error in the "zoom size" calculation from the KFB filenames (which I changed recently to avoid a crash, maybe I introduced a bug by accident).  EDIT this was the cause, fix will be in 2.12.3 later today.

The root cause is still unknown, pending further investigation.

I noticed that the auto solve glitches dialog adds at most 10 references to each frame.  So to solve all glitches you have to run it over and over, manually.  I will try to add a setting where you can change how many references can be added per frame, so you can increase it if necessary.  I also noticed that the reference count limit for the main image is 199 or so, which isn't enough for some very large images.  I'll try to lift this limit too.

In my tests "auto solve glitch" enabled, and "reuse reference" disabled, results in a zoom out sequence without glitches much more quickly over all than trying to "examine zoom sequence auto solve glitches" on a glitchy zoom out sequence rendered with "reuse reference" enabled (I stil didn't manage to get a glitch free sequence yet, after adding 100s of references).  I don't know why exactly, I suspect some precision loss or misalignment between pixels and their true parameter plane locations.  This is hard to debug, so for now I recommend not using the "reuse reference" method.  This doesn't help you much, sorry :(

There's something very weird going on with "reuse reference" that I reported before. Try having it enabled and working seemingly fine at some location that doesn't have to be very deep, then change the coordinates drastically and let it render again. It will look like you only moved a tiny bit. Could it be that upon changing the coordinates, the location of the reused reference is automatically used as the center?

So, false alarm w/r/t my glitch-detection method? It's a KF bug with reuse reference?

(OT) PING: Dinkydau -- do you ever check your facebook messages?

No, I never used facebook. Originally I signed up only to follow my student association's facebook page. I will answer your message.

Unfortunately, I think the problem is real.  The glitches at the center of the 4 large structures are not detected, there should be more detail there instead of a flat "hole".  I tried with series approximation disabled (render time ~9mins at 512x512, with approximation was ~90s) and the problem remained. :( :( :(


Location taken from https://olbaid-st.deviantart.com/art/Deep-Mandelbrot-Set-023-10-1086-695492737 , I tried to recreate the palette but I didn't quite get it right, anyway attached for convenience, also the image showing the problem.

update the problem occured with: glitch_threshold = 0.0000001 (this threshold is multiplied by squared magnitude for the glitch test).  Using Pauldelbrot's original 0.001 threshold (again multiplied by squared magnitude) the image renders correctly.  Sorry for the scare...

update 2 added another image with comparison of 4 threshold levels - even when the holes don't appear, the outer layers are much grainier with smaller thresholds, the 1e-3 (0.001) threshold looks much better.  Here's a little table of render times:

I think this is the best way, 1e-3 = 0.001 should be good for the squared magnitude test, as Pauldelbrot suggested (IIRC he originally said 1e-3 for non-squared magnitude which is 1e-6 for squared magnitude but later corrected himself).  Maybe I make this settable in the GUI (done, will be in the next release - just a checkbox that when enabled, uses the square root of the formula's glitch threshold instead. Default is disabled, because it slows down rendering a lot).

Here's a comparison with the Evolution of Trees location:
(https://mathr.co.uk/kf/evolution-of-trees-glitch-tolerance.gif)
(gif colour limit loses some quality, but the difference is still clear)
high took 2m38.5 with 14 refs; low took 7m15.7 with 46 refs


How to add the main reference in a glitch automatically, without having rendered the image already?

Anyway I tried adding references with the 0.0000001 threshold, both with add reference and with set main reference, there was still some bad appearance in the 4 structures (no longer a hole, but the details weren't quite correct).


I may try this at some point, but I doubt it will work...

Thanks claude.
I think your solution is the best, to have it optional to lower the threashold to a value that guarantees that all glitches will be found!

Now with the extremely effective Newton-Raphson, it is very easy to reach e1000 depths, which seems to be where this situation occurs.
I don't remember encountering this on shallower depths.

And I am very glad to find that Pauldelbrot's glitch detection method is still rock solid, which is an essential part of this kind of rendering :)

I discovered that it is not "reuse reference" that is to blame - just disabling "auto solve glitches" when rendering the zoom out sequence is enough to cause problems.  Something is broken with "examine zoom sequence auto solve glitches" itself.  Example attached, trying to "auto solve glitches" of a zoom out sequence (Fractal universe's location) rendered without "auto solve glitches" and without "reuse reference".  I'll try to find out what is wrong, and fix it for 2.12.4 (but if I don't manage it by the end of this week, I'll release 2.12.4 without a fix - there are plenty of other bug fixes that could be useful to have).


Yes I can reproduce this behaviour.  Added to known bugs list, so I don't forget about it.

Is there even any use for the ability to disable "auto solve glitches" anymore? I'm wondering if non-auto-solve-glitches patterns of usage here are basically obsolete. If it's both buggy and obsolete, maybe the auto solve glitches option should go (forced to always-on) and the alternative (buggy) solve-glitches methods should be removed?

I can see it being useful to render a whole video without glitch correction, so you can see roughly what it will look like before the whole thing is properly done (and perhaps work on things like zoom speed transitions or a soundtrack while glitch correction is proceeding).

Also, "reuse reference" provides a nice speedup for very deep zoom sequences, but "reuse reference" breaks "auto solve glitches", at least with the way kf2 is currently implemented.