Performance for FPS-style mandelbox exploration

[2DArray]

Hey guys - I'm a game developer, and I'm pretty new to rendering 3D fractals, but I've been prototyping a game focused on them.  Basically, it gives you first-person-shooter-style controls and lets you walk around the interior and explore.

<a href="https://www.youtube.com/v/MDDGOCzYk48&rel=1&fs=1&hd=1" target="_blank">https://www.youtube.com/v/MDDGOCzYk48&rel=1&fs=1&hd=1</a>

At the moment, Mandelbox is the only supported shape, but I'm interested in adding more later on.  It'll take some more study to produce other surfaces that work well with the gameplay, though...

Anyway, the real point of my post is that I'm trying to make my raymarch shader run faster.  I'm developing this on a pretty modest laptop, but right now it's only pushing 6 fractal iterations and 40 raymarch steps...and I feel like it could do better.  Luckily, because of the way the game works, it doesn't need to support any crazy huge zooms, so I'd be happy to leave the box iterations pretty low, but I really want to increase that draw distance without requiring that people have crazy machines.

To be honest, I'm...also relatively new to writing shaders, and I don't have a totally solid grasp of how the loop unrolling and branch compilation works, so I assume that I'm doing some silly things that are wasting precious GPU time.

This is CG for SM3.0, running in Unity3D, by the way.

What can I improve, and why is it wrong?

Code:

float mandelboxDF(float4 z, inout float orbit) {
	
	//distance field formula was completely script-kiddied off of fractalforums
	//(knighty and Rrrola are some radical dudes)
	
	float4 offset=z;
	
	//in this case, this z0 value could be replaced with offset...
	//...but i might play with non-default offsets later
	float3 z0=z.xyz;
	
	//orbit trap:
	//square distance from current position to negative starting position,
	//added to current position's square length
	orbit=dot(z.xyz+z0,z.xyz+z0)+dot(z.xyz,z.xyz);
	
	for (int i=0;i<boxIters;i++) {
		//boxfold
		z.xyz=clamp(z.xyz,-1.0,1.0)*2.0-z.xyz;
		//spherefold
		z*=clamp(max(mr2/dot(z.xyz,z.xyz),mr2),0.0,1.0);
		
		//orbit trap
		//performed before scale/offset because...
		//...i dunno, it's prettier
		orbit=min(orbit,dot(z.xyz+z0,z.xyz+z0)+dot(z.xyz,z.xyz));
		
		//scale+offset
		z=z*scalevec+offset;
	}
	return (length(z.xyz)-C1)/z.w - C2;
}

fixed4 frag(vertO input):COLOR {
	int i;
	
	float3 rayDir=normalize(input.viewDir);
	fixed4 output=fixed4(0.0,0.0,0.0,0.0);
	float3 normal;
	float3 lightVector;
	
	float4 pos=float4(_WorldSpaceCameraPos,1.0);
	
	//compute a few things outside of the big loops
	C1=abs(_Scale-1.0);
	C2=pow(abs(_Scale),float(1.0-boxIters));
	mr2=dot(_SFMinRadius,_SFMinRadius);
	scalevec=float4(_Scale,_Scale,_Scale,abs(_Scale));
	
	float dist=1.0;
	float orbit=10.0;
	
	for (i=0;i<rayIters&&dist>_HitDist;i++) {
		dist=mandelboxDF(pos,orbit);
		pos.xyz+=rayDir*dist*_StepMultiplier;
	}
	if (dist<_HitDist) {
		//starter shading from raymarch iteration count
		output=(1.0-i/float(rayIters));
		
		//apply some colors based on orbit trap
		float ct=(abs(frac(orbit*1.0)-0.5)*2.0)*0.35+0.65;
		float ct2=abs(frac(orbit*.071)-0.5)*2.0;
		output.xyz*=lerp(fixed3(0.8,0.7,0.4)*ct,fixed3(0.7,0.15,0.2)*ct,ct2);
		
		//z-buffer for mixing with polygonal geometry
		fixed rawDepth=tex2Dproj(_CameraDepthTexture,UNITY_PROJ_COORD(input.screenPos)).r;
		if (rawDepth!=1.0) {
			float depth=LinearEyeDepth(rawDepth)*.01;
			float3 finalRay=_WorldSpaceCameraPos-pos.xyz;
			if (depth*depth<dot(finalRay,finalRay)) {
				discard;
			}
		}
		
	}
	if (i>=rayIters) {
		discard;
	} else {
		//apply some simple lighting
		normal=0.0;
		normal.x=mandelboxDF(pos+float4(0.001,0.0,0.0,0.0),orbit)-mandelboxDF(pos-float4(0.001,0.0,0.0,0.0),orbit);
		normal.y=mandelboxDF(pos+float4(0.0,0.001,0.0,0.0),orbit)-mandelboxDF(pos-float4(0.0,0.001,0.0,0.0),orbit);
		normal.z=mandelboxDF(pos+float4(0.0,0.0,0.001,0.0),orbit)-mandelboxDF(pos-float4(0.0,0.0,0.001,0.0),orbit);
		normal=normalize(normal);
		
		lightVector=normalize(float3(1.0,1.0,1.0));
		output.xyz*=clamp(dot(normal,lightVector),0.0,1.0)*0.5+0.5;
	}
	return output;
}

[cbuchner1]

oh, this is really excellent work. While I cannot really help you with making raymarching faster, I will be suggesting
a bit of game mechanics.

Have you considered doing some kind of 3D shooter that occurs inside the vast caverns found in 3D Mandelboxes?
These things sometimes look like giant spaceships or alien worlds, and one could place turrets and force fields all
over the place and fight an armada of small spaceships or sentry bots inside these futuristic environments.

Christian

[cKleinhuis]

wow, in fact i have the same idea of navigating in the fractal with that spider web

speaking about performance, as far as i know, having a distance estimation is the most efficient way to do it, you should make it so that it is controlled by the global "detail" setting, with is configured by "low,high,crazy" or something, that adjusts these values in the shader (maxdist, mindist, raysteps)

i think you have 2 implementations ? one in the shader and one in c# code that performs the collision detection ?

[2DArray]

Have you considered doing some kind of 3D shooter that occurs inside the vast caverns found in 3D Mandelboxes?

This would definitely be sweet, but I've been focusing on platforming because it seems a little farther removed from the "flythroughs" that a lot of people are already pretty accustomed to.  I also just really like the idea of interacting directly with the surface instead of mostly floating past it - makes it feel more "real."

i think you have 2 implementations ? one in the shader and one in c# code that performs the collision detection ?

That is correct!  And yeah, there'll definitely be a quality setting with a few different pre-compiled versions of the shader.

As an alternative question, does anybody know of other 3D fractals that would work well with this type of gameplay?  Mandelbox has some surprisingly good qualities for it - lots of flat surfaces to walk on, really nice interior spaces, etc.  I'd like to try and cook up a custom one, but that'll require some more study...I understand how fractals work in general, but I don't quite get the math behind the distance field part (specifically that "running derivative", which I haven't seen used in any other context).

[Mrz00m]

Hi man, i can collaborate on this i have some ideas about physics and unity mechanics... Sorry i didn't see your project until today, i would have been helping previously... dam. and at the moment i am juggling various projects...

Anyways... HOW DID YOU DO THE PHYSICS??? That's awesome?!?! i was thinking of generating physics inside a compute shader, by making an invisible minecraft world of cubes that you can walk on as colliders, where only cubes that contain the fractal are kept and the others are void spaces...

Here is a video i filmed today anyways... the commentary sent me to sleep so i did a multilingual conference on it to stop people falling asleep. bizarly i didn't see your vid on YT and i must have searched for unity fractals quite alot since you posted that video.

<a href="https://www.youtube.com/v/ugqWxabffpQ&rel=1&fs=1&hd=1" target="_blank">https://www.youtube.com/v/ugqWxabffpQ&rel=1&fs=1&hd=1</a>

here is a demo base unity3d mandelbox unityproject fileyou can import into a unity3d 5.3 free versiony, assets>import>locate the file m then if you can't load the .unity scene file, search for "learn" in the file browser to find it.https://dl.dropboxusercontent.com/u/114667999/learnignMandelbulbsInUnity.zip

original larger and more confusing project in the video is here:  https://github.com/i-saint/Unity5Effects

[Mrz00m]

Hehe, sorry for the funny accents on that video... was a bit ad-lib.

I just found where the formulas are kept in that code!!!! it's an actual CGinc file... as in a set of functions accessible from any C code...

distance_functions.cginc

Code:

#ifndef distance_functions_h
#define distance_functions_h

#include "foundation.cginc"

float kaleidoscopic_IFS(float3 z)
{
    int FRACT_ITER      = 20;
    float FRACT_SCALE   = 1.8;
    float FRACT_OFFSET  = 1.0;

    float c = 2.0;
    z.y = modc(z.y, c)-c/2.0;
    z = RotateZ(z, PI/2.0);
    float r;
    int n1 = 0;
    for (int n = 0; n < FRACT_ITER; n++) {
        float rotate = PI*0.5;
        z = RotateX(z, rotate);
        z = RotateY(z, rotate);
        z = RotateZ(z, rotate);

        z.xy = abs(z.xy);
        if (z.x+z.y<0.0) z.xy = -z.yx; // fold 1
        if (z.x+z.z<0.0) z.xz = -z.zx; // fold 2
        if (z.y+z.z<0.0) z.zy = -z.yz; // fold 3
        z = z*FRACT_SCALE - FRACT_OFFSET*(FRACT_SCALE-1.0);
    }
    return (length(z) ) * pow(FRACT_SCALE, -float(FRACT_ITER));
}


float tglad_formula(float3 z0)
{
  ....

etc there are 5 formulas for the moment

so it's possible to add entire libraries of M3D algo's.


I just checked if it runs with two shaders in the same game, it's fine, 75fps... If it's possible to get colors in there it would be awesome...

Because the formulas are in a CGinc, i think it's possible to have a central formula code, and use it from two rendering graphcs card codes, one of the graphics, and one to send collider information back to the games central physics engine.

it's amazing well programmed code. here are some pics:

[Mrz00m]

OK so i should complete this theoretical page on implementation of a fractal world where you can walk up to a fractal, and even chop a cubic portion out of it, move it to another place, and have cubes containing fractiworlds that you can throw at each other and make your SuperLsdezioBrothers (inc) massively multiplayer haven for LSD crazies to hang about on online and throw spells fractals at each other:

Can you take out a cube of a 3d fractal, a frizzely spongey lacey iso surface, carry it somewhere, and make a stack of rolley about miny fractal cubes and then push them over? Managing localized cubes of fractal is similar to zooming in using a distance exclusion, theoretically you can color the flat spaces of the cube where they intersect fractals so you have solid cubes of 3d fractals...

if you want to dig into fractals like in minecraft, you have to use something not based on raymarching for the camera which means the processing power becomes insane the more you dig complex surfaces the camera must see as void.

so you can design the game to work with cubeitals wich can individually be moved, resized and filled with a formula using individual formulas and variables that are controlled by a powerful graphics-c++ interfacing logic subsystem, an code-bus, the " kernal/system files of the game" which is an advanced topic of sending and recieving simple data from the graphics processor, and knowing what the limits are of the graphics processor for loading dozens of formulas with actually effective control over some/many variables of every formula controlleable from the load memory/control processor.


You can either use an already existant physics engine or you can write your own. If you use a standard physics API, you can adjust the bouncyness and iceyness of objects and you have more controls to describe physics in advanced terms without writing the advanced terms in the first place.

The boundary mesh of an splatter shape fractal, a city, architecture or a cave type isosurface has to be built from the base up using raycasting or a fast marching cubes equivalent, octree (fractal physics sub cubes of subcubes, awesome!). If you raycast physics you only get it's shape from ont point at a time and if you measure it in a grid, some spheres, or in volumes, you get a 3d physical boundary approximation of the mesh.

if you raycast from multiple directoins and if you subdivide space into cubes or tetrahedrons it's an fascinating topic of which is more efficient because it is super subjective topic.

You measure a distance around the place of the player on which it's useful to make a mesh of a 1000 physics cubes if you want simple physics environment that you can design in 100 lines in a week's time, and optimized advanced routines if you want 10 000 physics cubes which you can spend months to write.

So in terms of programming... To write an working basic physics engine for walking climbing on, it takes a week, only if you understand very clearly where to write the visual shader formula and where to write the physics compute shader physics array of cubes to represent the climbing frame the player walks on.

Objects in 3d space will be occluded by the fractal if they travel through a wall which is cool, and it's very confusing to resolve logically because of the illusion that they share the same code, when in fact the graphics card is passed Z-Distance information from the camera only at rendering level and so there needs to be a completely non-isosurface physics engine which pathetic compared to the power of the visual processing which is only lines from one point in space. The occlusion is only apparent and it's completely immaterial and etherial.

  

if you want to design 5-10 standard variables to change every formula, then you need 5-10 variables going into the graphics shader and the graphics physics card and controlling the formula, and accessing the arrays built in graphics cginc file into the processor, if dx11 shaders can even read a cginc file, if that's even possible...

can someone code that plz ty  

NB you can even cast rays up down forwards backwards left right from the players last frame position, and that would let the player land on stuff and can be faster than building cubes in the graphics program.in that sense you can also send 6 rays out from every cube in a cube lattics and send a the least recently used physics cube into the next available space in a high efficiency code trick.