Welcome to Fractal Forums

Fractal Software => Fragmentarium => Topic started by: Patryk Kizny on November 11, 2015, 02:36:31 PM




Title: Advanced Terrain Generator
Post by: Patryk Kizny on November 11, 2015, 02:36:31 PM
I've been playing a bit for the last few days with creating a more realistic and flexible terrain generator.
I am using also the latest version of my raytracer that includes nicely integrated Sky model.
Part of R&D for an upcoming short.

The default terrain in fragmentarium is based on the cnoise implementation which is very slow for the task.
Recently I posted a bunch of other noise functions and the IQ noise is way faster.
I also developed a nice compound sinus noise mode and pulse gizmos to control all this in larger scale.

This is all still in works, but I'll be sharing the bits and bites as soon as it gets more polished form.




Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 11, 2015, 04:45:09 PM
Erosion does not look bad!


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 11, 2015, 08:14:17 PM
very very nice, I can't wait to test it out :spgloomy:

I think we might need a "PatrykKizny" folder in the examples, if that's ok with you to distribute

the sky does look really good.

can this generate a water like surface? more than one surface ie:land and water?

with land, water, clouds is there enough shader code space for other stuff like, DE advances, bulbs, sponges, julias, etc.etc.etc. I am concerned that only Titan+ will be able to use this code :(


Title: Re: Advanced Terrain Generator
Post by: cKleinhuis on November 11, 2015, 08:34:55 PM
yay, terrain based on noise maps instead of heightmaps is the way to go, excellent!


Title: Re: Advanced Terrain Generator
Post by: Crist-JRoger on November 11, 2015, 08:50:01 PM
 :D i`m doing it too ) just today  ;D ;D ;D
And I have an idea to create landscape fractal with different noise functions for field, mountains and water. Now i tested cnoise and pnoise - simple sum for a variety of relief

(http://img15.deviantart.net/37c6/i/2015/315/f/7/frag_11_11_15_01_by_crist_jroger-d9gbtqa.jpg) (http://orig04.deviantart.net/481a/f/2015/315/f/3/frag_11_11_15_01_by_crist_jroger-d9gbtqa.jpg)

(http://orig11.deviantart.net/dd58/f/2015/315/5/0/frag_11_11_15_02_by_crist_jroger-d9gbu1b.jpg) (http://orig11.deviantart.net/dd58/f/2015/315/5/0/frag_11_11_15_02_by_crist_jroger-d9gbu1b.jpg)


Title: Re: Advanced Terrain Generator
Post by: mclarekin on November 11, 2015, 08:58:18 PM
very impressive  O0 O0 O0


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 11, 2015, 09:01:07 PM
On my end the terrain itself seems to be even faster than the cnoise based one. It's based on the IQ noise functions I shared earlier (pinned now). The other noise stuff based on sin function is also very fast. The power of this is a flexibility of mixing various components (not coding needed for users).

Yes, It can do water if you decrease amplitude.

I am working on that within a wider 'framework' for adding other fractals as hybrids and as soon as it is more polished I'll eagerly share it.
I can easily add a few fractals. Depending on the complexity of course it can finally kill any GPU, but I tend to explore the stuff from outside instead of doing deep zooms (which are not impressive on a GPU anyways).

True, the raytracer itself got very complex, but I've kept it modular, so you can strip it down and work only with needed components or the stuff you can afford.
the terrain generator though is not dependent on the tracer.

What would help to make it all way more elegant is the feature I suggested long ago - filtering out uniforms on compile that are sitting within #ifdef blocks.


Title: Re: Advanced Terrain Generator
Post by: Crist-JRoger on November 11, 2015, 09:02:25 PM
with land, water, clouds is there enough shader code space for other stuff like, DE advances, bulbs, sponges, julias, etc.etc.etc. I am concerned that only Titan+ will be able to use this code :(
CRY-engine for fractals  :hrmm:


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 11, 2015, 09:35:58 PM
the terrain generator though is not dependent on the tracer.
Do you have to render the "land,air,sea" separately and merge object images after? does the terrain interact with bulbs and other stuff?
What would help to make it all way more elegant is the feature I suggested long ago - filtering out uniforms on compile that are sitting within #ifdef blocks.
I've done some simple stuff just to remove the defines I've added,AUTO_FOCUS and DEPTH_TO_ALPHA ifdef,else,endif but handling multiple and/or nested conditions is going to be tricky and require some testing, if it can be kept to two conditions #ifdef and #ifndef with no #else or nesting then it would be very easy but that might be an unreasonable imposition on programmers.


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 11, 2015, 09:40:58 PM
simple #ifdef or #ifndef would be sufficient for this usage. The purpose at the moment is only hiding unused uniforms in the GUI.


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 14, 2015, 02:24:59 PM
It looks like I already have a nice set of layers for creating alien worlds!



Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 14, 2015, 07:28:30 PM
"hiding" uniforms in the GUI means stripping out code before it goes for processing by the current parser that generates the widgetry, easy todo for #define + #ifdef + #endif,  a bit harder to have the parser test all uniforms for usage and compliance.


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 14, 2015, 07:36:02 PM
No need for testing and compliance. Just strip down the code for uniforms the same way normal code is stripped when sits within #ifdefs


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 15, 2015, 03:19:33 AM
Almost there!


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 15, 2015, 03:43:05 AM
that is looking really nice ;)

No need for testing and compliance. Just strip down the code for uniforms the same way normal code is stripped when sits within #ifdefs

the GPU compiler does this, not done in Fragmentarium C++ code that's why some stuff don't get cleaned.


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 15, 2015, 04:30:58 PM
Dear friends,

Sharing the first public version of the terrain generator.
As usually with me, it's become pretty complex, so it may be an overkill for slower systems.

Usage notes
There are four tabs working together.

T-GIZ
This is a primary tab where you have 3 gizmos available.
Each of them produces some sort of mountain-like geometry and can be modified using various parameters.
Use the gizmos to create rough formations of the landscape.
To test gizmos, set the global rendering mode TerOutput to 1. It makes only gizmos to be output thus makes testing easier.
Mode changes various curves used for gizmo generations. In fast terrain mode it's disabled and limited to simple cubic function.
Size modifies global gizmo scale. XY are width and height, Z is normalized amplitude. To make taller formations use LevOut
Shape vec2s depend on the Modes at the top and usually modify details of the curves.
LevOut is an output amplitude of formations. Note that in general, the steeper formations you want to use, the lower Fudge factor must be set to avoid artifacts.

T-SIN
This is a sinusoidal noise generator module.
There's a primary 1-component XYZ sin noise, that I recommend using for another round of global formations creation.
There's a secondary 3-component noise that you can use to shape finer details of the landscape
The engine uses optimizations, so if Freqs or Amps are cranked down to 0.0 it'll skip processing these portions - may help to boost performance on slower machines.

T-TER
This is a tab with a modified and extended version of terrain presented earlier in Fragmentarium.
The old version was based on Ashima Noise and CNoise that was pretty slow. Currently there are 4 modes available:
- IQ2D that is a very fast alternative for previous terrain functions
- IQ3D is also fast, but can be used to create rougher terrain
- IQ2D/3D mode - this one is a mix of the two above, but with optimized processing, so if you plan on using both 2D and 3D noise, just crank them down and only use Mixed 2D/3D. TerMixer blends between 2D mode and 3D mode, both operating on the same set of parameters.
- Finally there's the old CNoise producing the terrain as originally.

Please note that controls for these noises are visible in the T-MIX tab.
Finally, I added also a pre-iteration (TerInitAmp, TerInitFreq) that can be used to generate a large-scale terrain formations independently of the further iterations.

T-MIX
This is the control center.
TerFastMode-  drops some features to gain on performance. Make sure to keep it locked.
TerOutput - switches between various modes of the module - you can output flat terrain, only gizmos, only noises or full composited terrain. Unlock it for designing your landscape, Lock it before rendering in 5 mode.
Global transformations PT Rotate/Translate/Scale are global transformations. I use it across all my modular blocks to enable better compositing with other stuff (fractals etc).
Output vec4 uniforms - this is your control center where you decide where each of noises go. 4 components of vec4 are as follows:
- X: global output - adds to heightmap regardless of other features, globally
- Y: mixes the noise into Gizmo 1
- Z: mixes the noise into Gizmo 2
- W: mixes the noise into Gizmo 3

Notes
- I know it's quite complex, so takes a while to learn, but I believe it's a powerful tool.
- I plan on optimizing it a bit more and adding a texture-based mode to enable large-scale terrain modeling based on predefined textures - this can improve performance significantly and adds more control.
- There are some rendering issues on steeper formations. It's generally related to the raytracer (present with most of tracers). The problem is that (without noises) these are well-defined structures with a nice and clean DE (sin, cos, parabolas etc...) - so in theory they should render well. But for some reason the tracer misses surface particularly in the areas where the surface is perpendicular to the ray. It requires to significantly decrease fudge factor to compensate for it. I am not sure how to address that - maybe some of the gurus here can help.

@3dickulus - please feel free to create PatrykKizny folder in examples and include the stuff.


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 15, 2015, 05:45:56 PM
If you wanted to give it a shot, here's a version with my beefy 1.3 raytracer in full glory.



Title: Re: Advanced Terrain Generator
Post by: M Benesi on November 15, 2015, 05:49:06 PM
Neat... if only I could run it on an 8 year old mobile GPU.... :p


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 15, 2015, 05:55:58 PM
Neat... if only I could run it on an 8 year old mobile GPU.... :p

The first version is way lighter. You can give it a try.


Title: Re: Advanced Terrain Generator
Post by: Crist-JRoger on November 15, 2015, 06:17:17 PM
Patryk Kizny, thanks for sharing.
Unfortunately both scripts don't run on my gtx660. Failed... 7 seconds of compiling  :'(
Strange, you said that this noise code by IQ is faster  :-\  Maybe some optimizations...  :snore:


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 15, 2015, 06:38:01 PM
Any errors or just crashed?
Here it compiles in 25s (in theory), in practice about 10.


Title: Re: Advanced Terrain Generator
Post by: Snicker02 on November 15, 2015, 08:12:02 PM
I am getting errors

error log
Parse: D:/documents/downloads/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Examples/_Terrain_v3.frag
Including file: D:/documents/downloads/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Examples/DE-Kn2.frag
Including file: D:/documents/downloads/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Examples/3DKn-1.0.1.frag
Including buffershader: D:/documents/downloads/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Examples/BufferShader-1.0.0.frag
Parse: D:/documents/downloads/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Examples/BufferShader-1.0.0.frag
BufferShader 1.0.0
Including file: D:/documents/downloads/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Examples/Math-Utils.frag
Including file: D:/documents/downloads/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Examples/Space-Transforms.frag
Including file: D:/documents/downloads/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Examples/Classic-Noise.frag
Including file: D:/documents/downloads/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Examples/DEF-Terrain_v3.frag
Including file: D:/documents/downloads/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Fragmentarium-1.0.13-Qt_5_3_2_MinGW/Examples/Presets-Camera-Z-Up.frag
Advanced Terrain Generator v3.0
Patryk Kizny / http://kizny.com
Based on moise functions by IQ
------
DE-RaytracerKn-0.9.12.frag: Original shader by syntopia. Modifications by knighty + Eiffie + ChrisJRodgers:
- Added multiple reflections
3DKn-0.9.12.frag: Original shader by syntopia. Modifications by knighty:
- Assumes square shaped pixel -> no over-blurring along x axis if the rendering window width is small wrt its height
- Added polygon shaped diaphragm.
- Added Control for the width of the in-focus region.
Simple 3D Setup
Terrain Generator by Patryk Kizny
http://kizny.com
Based on IQ noise functions
float terDE(vec3 p)
float gndGizmo(vec3 p)
make sure to include "Classic-Noise.frag"
Basic camera presets. Z is up
4 locked variables: TerFastMode,TerOutput,ApertureNbrSides,ApStarShaped
Camera: Use W/S to fly. 1/3 adjusts speed. Q/E rolls. Click on 3D window for key focus. See Help Menu for more.
Created front and back buffers as RGBA32F
Could not create fragment shader: Fragment shader failed to compile with the following errors:
ERROR: 1:2250: error(#202) No matching overloaded function found: opTRS
ERROR: error(#273) 1 compilation errors.  No code generated


Failed to compile script (156 ms).




here are the params

// Output generated from file: X:/Projects/20151106 - LetThereBeLight/TerrainGenerator_20151115.frag
// Created: Sun Nov 15 15:59:15 2015
#info Advanced Terrain Generator v3.0
#info Patryk Kizny / http://kizny.com
#info Based on moise functions by IQ
#info ------

// Flags
#define USE_EIFFIE_SHADOW
#define PERFECT_DE

// PK raytracer flags
//#define USE_PK_IBL
//#define USE_PK_BACKGROUND
//#define USE_PK_DIRT
//#define USE_HYBRID_1
//#define USE_OLD_GLOW
//#define USE_CLIPPING
//#define USE_KN2_SHADER
//#define USE_IQ_CLOUDS
//#define USE_IQ_CLOUDS_MULTILAYER // Multilayer Clouds by PK
//#define USE_KN_VOLUMETRIC
//#define TAG_LAYERS // Uncomment this to output alpha pass

// Raytracer
//#include "SunSky_PK_v1.frag"
//#include "DE-PK-1.3.frag"
#include "DE-Kn2.frag"

// Core
#include "Math-Utils.frag"
#include "Space-Transforms.frag"
#include "Classic-Noise.frag"

#include "DEF-Terrain_v3.frag"

// Presets
#include "Presets-Camera-Z-Up.frag"

float DE(vec3 z){
   
   float de;

   #ifdef DE_Terrain
   de = terDE(z);
   #else
   de = z.z;
   #endif
   
   return de;
}




#preset Default
FOV = 1.35
Eye = -9.097912,-8.895071,2.877836
Target = -6.880132,-6.401387,1.822852
FocalPlane = 1
Aperture = 0
InFocusAWidth = 0
ApertureNbrSides = 5
ApertureRot = 0
ApStarShaped = false
Gamma = 1
ToneMapping = 1
Exposure = 1
Brightness = 1
Contrast = 1
Saturation = 1
GaussianWeight = 1
AntiAliasScale = 2
Bloom = false
BloomIntensity = 0.25
BloomPow = 2
BloomTaps = 4
Detail = -2.5
DetailAO = -0.5
FudgeFactor = 0.5
MaxRaySteps = 256
MaxDistance = 35
MaxRayStepsDiv = 1.8
Dither = 0.5
NormalBackStep = 1
AO = 0,0,0,1
AoCorrect = 0
Specular = 0.4
SpecularExp = 16
SpecularMax = 10
CamLight = 1,1,1,0.35
CamLightMin = 0
Glow = 1,1,1,0
GlowMax = 20
Reflection = 1,1,1
ReflectionsNumber = 0
SpotGlow = true
SpotLight = 1,1,1,1
LightPos = 2.554744,6.253041,10
LightSize = 0.1
LightFallOff = 0
LightGlowRad = 0
LightGlowExp = 1
HardShadow = 1
ShadowSoft = 10
BaseColor = 0.4745098,0.4745098,0.4745098
OrbitStrength = 0
X = 0.5,0.6,0.6,0.7
Y = 1,0.6,0,0.4
Z = 0.8,0.78,1,0.5
R = 0.4,0.7,1,0.12
BackgroundColor = 0.6,0.6,0.6
GradientBackground = 0.3
CycleColors = false
Cycles = 1.1
EnableFloor = false
FloorNormal = 0,0,1
FloorHeight = 0
FloorColor = 1,1,1
HF_Fallof = 0.1
HF_Const = 0
HF_Intensity = 0
HF_Dir = 0,0,1
HF_Offset = 0
HF_Color = 1,1,1,1
sTranslate = 0,0,0
sRotate = 0,0,0
sScale = 1
sTwist = 0
clipToBox = false
showClippingBox = false
cBoxSize = 5,5,5
cBoxTranslate = 0,0,0
cBoxRotate = 0,0,0
TerPTRotate = 0,0,0
TerPTTranslate = 0,0,0
TerPTScale = 1
TerSin1Out = 0,0,0,0
TerSin2Out = 0,0.5,1,1
TerIQ2DOut = 0.15,1,1,0
TerIQ3DOut = 0,0,0,0
TerIQMixOut = 0,0,0,0
TerCNoiseOut = 0,0,0,0
TG1_Mode = 4
TG2_Mode = 0
TG3_Mode = 3
TG1_Center = -5,2
TG2_Center = 2,4
TG3_Center = 4,-5
TG1_Size = 2,2,1
TG2_Size = 5,5,1
TG3_Size = 25,5,1
TG1_Shape = 0.25,0.5
TG2_Shape = 0,0
TG3_Shape = 0.1,0.25
TGLevOut = 1.5,3,-2.55
TerSinNoisePrimFreq = 1.75,3.5,0.05
TerSinNoisePrimAmp = 0.5,0.5,0.5
TerSinNoisePrimOffset = 0,0,0
TerSinNoiseSecFreq = 2,3,1.5
TerSinNoiseSecAmp = 0.35,0.35,1
TerSinNoiseSecFM = 1.15,2.66,4.77
TerSinNoiseSecAM = 0.75,0.5,1
TerIterations = 5
TerSlope = 0.5
TerInitFreq = 1
TerInitAmp = 0.5
TerFreqMul = 5
TerAmpMul = 1
TerOffset = 0
TerMixer = 0.5
TerFastMode = false
TerOutput = 4
Up = 0.2009916,0.2238526,0.9516473

#endpreset



Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 15, 2015, 11:29:19 PM
Weird, cause it was exported while rendering so a working example.
What could have happened is frag included an older file version from some other path.
I'll check tomorrow and upload again.


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 16, 2015, 12:12:12 AM
GeForce GTX 760
CUDA Version: 7.5
nVidia Driver Version: 352.55

Could not link fragment shaders:
Fragment info
-------------
0(134) : warning C7011: implicit cast from "int" to "float"
0(1427) : warning C7011: implicit cast from "int" to "float"
0(1427) : warning C7011: implicit cast from "int" to "float"

Failed to compile script (32154 ms).

not too many instructions so must be something else,
when locating included files Fragmentarium begins in the folder containing the fragment and only if not found does it check other locations.

I also get...

Tried path: /home/uname/Fragmentarium/Examples/Include/Milkyway_BG.jpg
Could not resolve path for file: Milkyway_BG.jpg

in the preset we have...

BgMap = C:/DATA/Fragmentarium/1.0.10/Examples/Include/Milkyway_BG.jpg
EnvMapSpecular = C:/DATA/Fragmentarium/1.0.10/Examples/Include/Ditch-River_2k.hdr
EnvMapDiffuse = C:/DATA/Fragmentarium/1.0.10/Examples/Include/Ditch-River_Env.hdr

...even though the paths are wrong for Ditch-River files they do get found because they exist in Examples/Include/ folder as part of the distribution so every one should have these...

Added texture: EnvMapSpecular -> /home/uname/Fragmentarium/Examples/Include/Ditch-River_2k.hdr
Added texture: EnvMapDiffuse -> /home/uname/Fragmentarium/Examples/Include/Ditch-River_Env.hdr

...however, Milkyway_BG.jpg should be in the folder with the frag because it cannot be expected to exist on everyone else's systems.

when distributing a "fragment bundle" you need to include all support files, maybe I can write it into the bit that gathers the included files to also include textures, backgrounds, envmaps etc... maybe next patch release, just posted v1.0.15 source and exe ;)

@Snicker02: I note BufferShader 1.0.0 while PK's fragments, as bundled, uses BufferShader 1.0.2 this may be caused by you using #include "DE-Kn2.frag" which (just guessing) probably has no idea what PK's terrain generator is doing or how to handle it.

hope this helps :D


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 16, 2015, 03:46:30 AM
hmmm.....
Code:
QOpenGLShader::compile(Fragment):
  0(134) : warning C7011: implicit cast from "int" to "float"
0(1423) : warning C7011: implicit cast from "int" to "float"
0(1423) : warning C7011: implicit cast from "int" to "float"
0(1430) : warning C7011: implicit cast from "int" to "float"
0(1439) : warning C7011: implicit cast from "int" to "float"
0(1439) : warning C7011: implicit cast from "int" to "float"
0(1439) : warning C7011: implicit cast from "int" to "float"
0(1616) : warning C7011: implicit cast from "int" to "float"
0(1617) : warning C7011: implicit cast from "int" to "float"
0(1658) : warning C7011: implicit cast from "float" to "vec3"
0(1720) : warning C7623: implicit narrowing of type from "vec3" to "float"
0(2022) : warning C7011: implicit cast from "int" to "float"
0(2068) : warning C7011: implicit cast from "float" to "vec3"
0(2240) : warning C7505: OpenGL does not allow swizzles on scalar expressions
0(2240) : error C1031: swizzle mask element not present in operand "xyz"
0(2240) : warning C7505: OpenGL does not allow swizzles on scalar expressions
0(2240) : error C1031: swizzle mask element not present in operand "w"
0(2253) : warning C7011: implicit cast from "int" to "float"
0(2253) : warning C7011: implicit cast from "float" to "vec3"
0(2363) : warning C7011: implicit cast from "int" to "float"
0(2367) : warning C7011: implicit cast from "int" to "float"
0(2372) : warning C7011: implicit cast from "vec4" to "vec3"
0(2379) : warning C7011: implicit cast from "vec4" to "vec3"
0(2393) : warning C7011: implicit cast from "int" to "float"
0(2401) : warning C7011: implicit cast from "int" to "float"
0(3232) : warning C7011: implicit cast from "int" to "float"
0(3428) : warning C7623: implicit narrowing of type from "vec3" to "float"
0(3428) : warning C7011: implicit cast from "float" to "vec3"

I had a look at the code and I'm afraid I can't fix it but I can help a bit...
when the GPU compiler detects something wrong with the shader it dumps the errors + source to console out on stderr, this does not showup in the Fragmentarium logs because it is before linking (panic afaik),  you can capture it by running from a cmd prompt like...
Code:
./Fragmentarium-1.0.15 2> FDUMP.txt
the "2>" redirects stderr to a file
load FDUMP.txt into a text editor with line numbers
cut and paste everything upto an including the line "*** Problematic Fragment shader source code ***" into another (you need 2) text editor window
now you will be looking at a list of errors and line numbers in one window and the raw source in the other
from this you will be able to track down the line and the file it's in so you can fix it ;)


Title: Re: Advanced Terrain Generator
Post by: JohnVV on November 16, 2015, 08:53:06 AM
hi 3dickulus and Patryk Kizny

i too am getting the same C7011 error  on linux
 opensuse 13.2
 A older  nvidia gts 240
 The older cuda 6.5

Code:
Could not link shaders: Fragment info
-------------
0(1189) : warning C7011: implicit cast from "int" to "float"
0(1193) : warning C7011: implicit cast from "int" to "float"
0(1198) : warning C7011: implicit cast from "vec4" to "vec3"
0(1205) : warning C7011: implicit cast from "vec4" to "vec3"
0(1219) : warning C7011: implicit cast from "int" to "float"
0(1227) : warning C7011: implicit cast from "int" to "float"
0(2057) : warning C7011: implicit cast from "int" to "float"
0(2135) : warning C7011: implicit cast from "int" to "float"
0(2140) : warning C7011: implicit cast from "int" to "float"
0(2253) : warning C7011: implicit cast from "float" to "vec3"
0(2265) : warning C7011: implicit cast from "int" to "float"
Internal error: assembly compile error for fragment shader at offset 399814:
-- error message --
line 15861, column 1:  error: too many instructions
-- internal assembly text --
!!NVfp4.0
OPTION NV_parameter_buffer_object2;
# cgc version 3.4.0001, build date Aug 19 2015
# command line args:
#vendor NVIDIA Corporation
#version 3.4.0.1
#profile gp4fp
#program main
#semantic subframe
#semantic backbuffer
#semantic FocalPlane
#semantic Aperture
--- and so on for 17,000+ lines
]


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 16, 2015, 03:38:20 PM
Code:
-- error message --
line 15861, column 1:  error: too many instructions

let me guess, you fixed a few things, it got a past syntax err, but now the compiler says it's too big for your GTS 240


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 16, 2015, 06:45:50 PM
Hey,

I have opened and compiled both packages that I posted, also removing the extra include paths of my fragmentarium.
So I don't know what's wrong.

Regarding the maps - just forget about it - you can replace it with anything you like. These don't matter.
On a weaker GPUs it can probably exceed the amount of instructions it can handle.

Regarding the opTRS - that's weird with no matching overloaded function, but you can try working around it by commenting line 420 of DEF-Terrain

// Global transformations
// z = opTRS(z, TerPTTranslate, TerPTRotate, TerPTScale);


You can also replace the space-transform file with the one attached.



Title: Re: Advanced Terrain Generator
Post by: M Benesi on November 16, 2015, 07:18:36 PM
  I made a better version of my sphere to cube thing since the "convert3" version you put in your "space-transforms.frag".

  It eliminates the trig functions... so ya know.  It's in the HELP! Thread, and I think it's in the "circle math" thread as well.  No trig functions= faster... even on a super duper GPU.  ;)

Code:
	// cubic to spheric   (can be optimized... dot (z,z)...  mclarekin's idea ;)  )
rCyz = z.y*z.y / (z.z*z.z);
if (rCyz < 1.0) {rCyz = sqrt(rCyz + 1.0);} else {rCyz=sqrt( 1.0 / rCyz + 1.0);}

rCxyz = (z.y*z.y + z.z*z.z ) / (z.x*z.x) ;
if (rCxyz < 1.0) {rCxyz =s qrt(rCxyz + 1.0);} else {rCxyz=sqrt(1.0/ rCxyz + 1.0);}

z.yz *=rCyz;
z *= rCxyz;



Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 16, 2015, 08:30:35 PM
Hmmm, did you try it's really faster? Sqrt is a though one too.


Title: Re: Advanced Terrain Generator
Post by: Snicker02 on November 16, 2015, 10:51:44 PM

Regarding the opTRS - that's weird with no matching overloaded function, but you can try working around it by commenting line 420 of DEF-Terrain

// Global transformations
// z = opTRS(z, TerPTTranslate, TerPTRotate, TerPTScale);


You can also replace the space-transform file with the one attached.




That worked for me!!!  O0 :embarrass: :D


Title: Re: Advanced Terrain Generator
Post by: M Benesi on November 16, 2015, 10:59:22 PM
Hmmm, did you try it's really faster? Sqrt is a though one too.
 2 sqrt vs. 4 trig functions?  According to what I've read it's faster...  and.. well,  according to what I've experienced.

  Pine Tree with trig takes about 2 times as long as the algebraic pine tree.  sooo...


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 17, 2015, 05:34:17 AM
lol at myself ;) I didn't mean that last comment to sound so snarky, sorry 'bout that.
My hope is that it does work, the images you've presented look really good.

opTRS is declared in Space-Transforms.frag as vec3 opTRS(vec3 , float , vec3 , vec3 )
                          and called in DEF-Terrain_v3.frag as opTRS(vec3 , vec3 , vec3 , float )

it needs to be called the same way it is declared ;)

I am curious what this is used for?
Code:
// http://www.fractalforums.com/fragmentarium/fractal-plus-change-of-domain-mashup/
// performs the active c = T(s)
vec3 domainMap(vec3 c, vec3 invC){
float s = dot(c,c);
return c/s + invC;
}



Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 17, 2015, 11:24:50 AM
opTRS is declared in Space-Transforms.frag as vec3 opTRS(vec3 , float , vec3 , vec3 ) and called in DEF-Terrain_v3.frag as opTRS(vec3 , vec3 , vec3 , float )

→ I posted a corrected file version a few posts above.

Code:
// http://www.fractalforums.com/fragmentarium/fractal-plus-change-of-domain-mashup/
// performs the active c = T(s)
vec3 domainMap(vec3 c, vec3 invC){
float s = dot(c,c);
return c/s + invC;
}

It's used by some fractals and as I said the code I posted is a part of a larger thing I have, so not every bit is used.


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 18, 2015, 05:44:00 AM
Would this be a Roquen-bulb? http://www.fractalforums.com/fragmentarium/fractal-plus-change-of-domain-mashup/msg65040/#msg65040


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 18, 2015, 04:40:50 PM
Here's a quick RnD workinprogress test done with the generator.
How you guys like it?

https://vimeo.com/146126385


Title: Re: Advanced Terrain Generator
Post by: Han Maokun on November 19, 2015, 03:08:14 AM
Here's a quick RnD workinprogress test done with the generator.
...

Patryk Kizny, I am new to this, my questions is how can I try out this Adavanced Terrain Generator, is it a standalone software or plugin within another tool? I didn't find any downloadable link, Thanks!


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 19, 2015, 03:15:56 AM
looks really nice  :thumbsup1:

@Han Maokun you can download Fragmentarium here (http://www.digilanti.org/fragmentarium/) and the fragment scripts posted earlier in this thread.


Title: Re: Advanced Terrain Generator
Post by: M Benesi on November 19, 2015, 03:17:03 AM
I'd like it more if my computer could run your frags!  Awesome looking though.  ;)


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 19, 2015, 03:23:08 AM
@Patryk Kizny can it be refined down to just the terrain generator and default DE-Raytracer.frag  :hmh:


Title: Re: Advanced Terrain Generator
Post by: Crist-JRoger on November 20, 2015, 12:20:43 AM
water waves from combining cnoise and pnoise with different iterations.
based on the famous game FarCry  ;D

(http://pre07.deviantart.net/7d59/th/pre/f/2015/323/c/f/far_cry_memories_by_crist_jroger-d9h8s0l.jpg)


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 20, 2015, 02:59:05 AM
 :o


Title: Re: Advanced Terrain Generator
Post by: M Benesi on November 20, 2015, 03:14:48 AM
Beautiful Crist! 



Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 20, 2015, 12:23:00 PM
Crist, but that wasn't rendered with my script?


Title: Re: Advanced Terrain Generator
Post by: Crist-JRoger on November 20, 2015, 03:11:23 PM
Thank you guys!  :dink:

Crist, but that wasn't rendered with my script?
Because it's don't work  :'( I have the same problems as sayed before. Too long comlilation... and error. So today i opened your code and got only your noise functions (n1,n2,n3,n4) and i runed it in course. Full script too hard.
Maybe if you have some time, could you set default values in sliders to most-faster-mode  :hmh:  Thank you.


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 20, 2015, 03:44:52 PM
@Crist-JRoger I agree, most people will not be able to use the full script. :sad1:


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 20, 2015, 10:37:43 PM
I've split the complex stuff into multiple smaller generator components, so people with weaker GPUs could run it.
I'm on Synthclipse now, so I'd need to convert it back to frag standard.

Anyways, today I implemented also a terrain based on domain warp noise.
It's still pretty fast and results are decent.

(https://c1.staticflickr.com/1/771/23130747936_14a4970f0e_b.jpg)


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 20, 2015, 11:15:52 PM
Here's the stripped down and modular version of the TG 3.3.


Title: Re: Advanced Terrain Generator
Post by: Syntopia on November 21, 2015, 01:17:31 AM
Looks great! You latest frag (TG 3.3) works fine on my machine.


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 21, 2015, 04:10:26 AM
Awesome... will be testing TG 3.3 tonight ;) TY!


Title: Re: Advanced Terrain Generator
Post by: Crist-JRoger on November 21, 2015, 09:29:30 AM
Default set of frags works well! Thank you. But when i uncommented #include "DEF-Terrain_IQT.frag" and compiled script, i get the same error, again. My videocard is not weak, because for example i played in Witcher3 on high graphic options and it was very fast  :)

Log (not full)
Code:
Could not link shaders: Fragment info
-------------
0(712) : warning C7011: implicit cast from "int" to "float"
0(716) : warning C7011: implicit cast from "int" to "float"
0(742) : warning C7011: implicit cast from "int" to "float"
0(750) : warning C7011: implicit cast from "int" to "float"
0(986) : warning C7011: implicit cast from "int" to "float"
0(991) : warning C7011: implicit cast from "int" to "float"
0(1267) : warning C7011: implicit cast from "int" to "float"
Internal error: assembly compile error for fragment shader at offset 403840:
-- error message --
line 15745, column 1:  error: too many instructions
-- internal assembly text --
!!NVfp5.0
OPTION NV_gpu_program_fp64;
OPTION NV_bindless_texture;
# cgc version 3.4.0001, build date May 11 2015
# command line args:
#vendor NVIDIA Corporation
#version 3.4.0.1
#profile gp5fp
#program main
#semantic EquiRectangular
#semantic subframe
#semantic backbuffer
#semantic FocalPlane
#semantic Aperture
#semantic Gamma
#semantic ToneMapping
#semantic Exposure
#semantic Brightness
#semantic Contrast
#semantic Saturation
#semantic GaussianWeight
#semantic AntiAliasScale
#semantic FOV
#semantic Detail
#semantic DetailAO
#semantic FudgeFactor
#semantic MaxRaySteps
#semantic Dither
#semantic AO
#semantic Specular
#semantic SpecularExp
#semantic SpecularMax
#semantic SpotLight
#semantic SpotLightDir
#semantic CamLight
#semantic CamLightMin
#semantic Glow
#semantic GlowMax
#semantic Fog
#semantic HardShadow
#semantic ShadowSoft
#semantic QualityShadows
#semantic BaseColor
#semantic OrbitStrength
#semantic X
#semantic Y
#semantic Z
#semantic R
#semantic BackgroundColor
#semantic GradientBackground
#semantic CycleColors
#semantic Cycles
#semantic FloorNormal
#semantic FloorHeight
#semantic FloorColor
#semantic sTranslate
#semantic sRotate
#semantic sScale
#semantic sTwist
#semantic clipToBox
#semantic showClippingBox
#semantic cBoxSize
#semantic cBoxTranslate
#semantic cBoxRotate
#semantic TerSinNoisePrimFreq
#semantic TerSinNoisePrimAmp
#semantic TerSinNoisePrimOffset
#semantic TerSinNoiseSecFreq
#semantic TerSinNoiseSecAmp
#semantic TerSinNoiseSecFM
#semantic TerSinNoiseSecAM
#semantic TerSin1Out
#semantic TerSin2Out
#semantic TerIQIterations
#semantic TerIQInitFreq
#semantic TerIQInitAmp
#semantic TerIQFreqMul
#semantic TerIQAmpMul
#semantic TerIQOffset
#semantic TerIQAmpGain
#semantic TerIQ2DOut
#semantic TerIQ3DOut
#semantic TerIQMixBalance
#semantic TerIQMixOut
#semantic TerDWIterations
#semantic TerDWwf
#semantic TerDWwa
#semantic TerDWFreq0
#semantic TerDWAmp0
#semantic TerDWAmp
#semantic TerDWFreqMul
#semantic TerDWAmpMul
#semantic TerDWOffset
#semantic TerDWNoiseOut
#semantic TerFastMode
#semantic TerOutput
#semantic TerPTRotate
#semantic TerPTTranslate
#semantic TerPTScale
#semantic TG1_Mode
#semantic TG2_Mode
#semantic TG3_Mode
#semantic TG1_Center
#semantic TG2_Center
#semantic TG3_Center
#semantic TG1_Size
#semantic TG2_Size
#semantic TG3_Size
#semantic TG1_Shape
#semantic TG2_Shape
#semantic TG3_Shape
#semantic TGLevOut
#var float4 gl_FragColor : $vout.COLOR : COL0[0] : -1 : 1
#var float gl_FragDepth : $vout.DEPR : DEPR : -1 : 1
ENDIF;
ENDIF;
ENDIF;
IF    NE.x;
MOV.F R15.w, R16.x;
.................................
END
# 15645 instructions, 37 R-regs, 1 D-regs


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 21, 2015, 08:03:41 PM
-- error message --
line 15745, column 1:  error: too many instructions

too much code  :sad1:


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 22, 2015, 06:18:41 AM
TG 3.3 works well and, I think, warrants further development/refinement.

Currently Fragmentarium takes the user source with all included files and makes a single vertex/fragment shader for DE, maybe we are seeing the limits of a single shader doing all of the work, I think eventually it will have to be split into, object(xyz surface), lighting(ambient directional point source), shadow(hard soft), color? in such a way that each successive shader uses the data from the previous, like object(s) first for depth values, light second for surface normals, then shadows for intensity mask, and finally color derived from the data generated by these passes. Just need to break it down and figure out how to work it into a "shader swapping" version of Fragmentarium :D so added objects or effects are relatively contained to one vertex/fragment shader and executed as needed.

The BufferShader.frag is a good place to start as it already gets used like this, it's a separate shader that draws the rendered image on a quad with some "Post" image processing, I think maybe some stuff could be moved out of DE.frag and into BufferShader.frag.

For instance TG 3.3's DE shader is about 13767 lines of GPU assembler (on my GTX760) and TG 3.3's buffer shader is 208 lines, this tells me that there is a lot of room in buffershader.frag that could be used for calculating shadows from light and depth information generated in DE.frag, maybe some of the oh so elusive fragment shader wizards are looking at this already ;) with maybe a geometry shader or 2 thrown in just for fun...

hmm... this is my Christmas wish  :nerd: :/


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on November 22, 2015, 02:58:14 PM
3dickulus, that sounds good.
I understand capacity of single shader is becoming a bottleneck.
However, I am not sure how would it work performance-wise. Would it be beneficial to have tasks spread into various shaders?

I am a total newbie to understanding how GPUs and shaders work on a deeper level, however, I assume that given I have 3K shader cores, they just process 3K frags/pixels at once in parallel. Right? So if we had 2 shaders (running in parallel) they would be simply spread across avail cores, so not much gain on the performance?

I can see multi-shaders beneficial at various levels.

1 - Procedural textures for materials and displacement
2 - materials
3 - post effects

and probably even more.
Also, having a few outputs per shader (or on multiple shaders but within the same let's say 'project' would be beneficial as we could render various layers like depth,  alpha, AO, lights, shadows etc… and output to multiple files (or multi-layer EXR) for further compositing. That would be powerful not only for my personal use, but also for VFX industry.

I would suggest considering a node-based approach for handling projects and shaders relationships.


Title: Re: Advanced Terrain Generator
Post by: 3dickulus on November 22, 2015, 03:53:47 PM
on my GTX760 the hardware is setup like this...

( 6) Multiprocessors x (192) CUDA Cores/MP:          1152 CUDA Cores
Warp size:                                                                 32
Maximum number of threads per multiprocessor:  2048
Maximum number of threads per block:                 1024
Maximum sizes of each dimension of a block:        1024 x 1024 x 64
Concurrent copy and kernel execution:                   Yes

figuring out how many threads a shader might use is difficult because it is dependant on the data layout and instruction flow, GPUs are not good at boolean conditionals like if-then-else or iterative functions like mandelbrot, fortunately the voodoo is done for us and all we have to do is work within the constrains of the pipeline and GLSL.

if something can be moved from the DE shader into the buffer shader then the performance hit will be 0, it will execute at the same speed as it does now, maybe faster due to some room for the GPU compiler to optimize better, as long as you don't add anything, when the new space in DE is then filled by more features or processing there will be an obvious performance hit.


Title: Re: Advanced Terrain Generator
Post by: Crist-JRoger on December 14, 2015, 09:46:19 PM
Patryk`s iq-noise and heightmap from M Benesi

(http://orig03.deviantart.net/e64d/f/2015/348/0/c/tex_bump_test_by_crist_jroger-d9k4u2a.jpg) (http://orig03.deviantart.net/e64d/f/2015/348/0/c/tex_bump_test_by_crist_jroger-d9k4u2a.jpg)


Title: Re: Advanced Terrain Generator
Post by: cKleinhuis on December 15, 2015, 01:33:42 AM
these non-heightmaps but noise fields make excellent landscapes, has anybody information on algorithms that apply erosion effects on such a structure?


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on December 15, 2015, 07:41:20 PM
these non-heightmaps but noise fields make excellent landscapes, has anybody information on algorithms that apply erosion effects on such a structure?

Modelling these realistically could be difficult and IMHO not worth it.
BUT you can easily mimic that and I did it with higher frequency sin noise applied. See my previous imagery in this thread.


Title: Re: Advanced Terrain Generator
Post by: laser blaster on February 05, 2016, 09:45:36 PM
Wow, some cool-looking pics in this thread! Funny that I also happened to be working on terrain .frag when I found this. Although I'm using 2D instead of 3D noise. I find that 2D Perlin noise, when modulated and combined in a certain way, gives great-looking mountains (pictures attached). The black and white picture is colored using one of the second order partial derivatives of the landscape surface. It highlights the ridges nicely.

Modelling these realistically could be difficult and IMHO not worth it.
BUT you can easily mimic that and I did it with higher frequency sin noise applied. See my previous imagery in this thread.
I've been trying to tackle just that problem. Specifically fluvial erosion, like this: http://www.nordiclandscapes.com/Mountains-Volcanic-landscapes-II/slides/mountain-erosion-vegetation.jpg. Of course there are simulation-based methods that run on a grid, but those would be inconvenient to do in Fragmentarium, and they're not fractal-based.

I found a neat trick for creating pseudo-erosion effects here: http://www.decarpentier.nl/scape-procedural-extensions, which I implemented in my terrain. But I also have a new method in the works.

My terrain implementation is very similar to his "swiss turbulence" example, with a few differences. He used the classic "ridged multifractal" approach to terrain, where you take the absolute value of each octave of Perlin noise to create sharp ridges, and then modulate the strength of each octave based on the sum of the previous octaves, to basically flatten out low regions of the map creating flat basins between the ridges. What I do differently is, for each octave I generate two different Perlin noise samples, take their absolute values, then multiply them together. This emphasizes individual peaks more, whereas his method tends to create longer ridges. Also, instead of just taking the absolute value, I use a "smooth abs" function that doesn't create such a hard crease, and is parametrized by a smoothness value. The function looks similar to this: http://amath.colorado.edu/faculty/becker/Dual_vs_Primal_smoothing.png (http://amath.colorado.edu/faculty/becker/Dual_vs_Primal_smoothing.png)

Another difference is that I modulate the strength of the noise not only on the height, but also on the cumulative derivative of the previous octaves, which smooths out any non-sloped region, whether high or low.

Another difference is that he forgot to multiply the derivative of each octave by it's frequency (which may have been intentional). I added that in. However, doing so can make the terrain look too messy, so I scale down the contribution of higher octaves by their own multiplier.

The last difference is that I use an additional sample of very low-frequency noise to modulate the height of the mountains, this creates large flat regions in the terrain and makes it look more believable.


Title: Re: Advanced Terrain Generator
Post by: laser blaster on February 05, 2016, 09:47:43 PM
Here's the black-and-white one colored using the second derivatives that I mentioned before:
(http://s9.postimg.org/hdnncnc7z/Terrain_Ridges.png) (http://postimg.org/image/wmdkqf5wb/full/)
 (http://postimage.org/app.php)
Another pic:


Title: Re: Advanced Terrain Generator
Post by: Patryk Kizny on February 08, 2016, 01:53:37 PM
This is so cool! Can you share a frag with this method?


Title: Re: Advanced Terrain Generator
Post by: laser blaster on February 10, 2016, 12:37:51 AM
Sure I will, but first I need to polish it up a bit and make it more efficient.


Title: Re: Advanced Terrain Generator
Post by: onehand on October 06, 2017, 12:34:24 AM
Sure I will, but first I need to polish it up a bit and make it more efficient.

Sorry to bump an old thread, but these (http://www.fractalforums.com/index.php?topic=22667.msg90551#msg90551) are some of the best mountains I've found IMO.  Would you be able to share this method @laser blaster ?  It would be greatly appreciated!

After numerous attempts, I've been unable to successfully reproduce this kind of noise.