Welcome to Fractal Forums

Fractal Software => Programming => Topic started by: David Makin on March 08, 2010, 10:33:38 PM




Title: Linearising a normal distribution (for the math geeks)
Post by: David Makin on March 08, 2010, 10:33:38 PM
Hi all, this is somkething I've wanted to do for a while but can't get a solution myself.

Originally I did this:

Take 2 fBm generators giving values from -1 to +1 and sum the results using:

 a*fbm1+(1-a)*fbm2

for a=0 to a = 1.
Which works except that as the 2 fbms are summed the distribution changes in a non-linear fashion producing a smoother final result when a!=0 and a!=1 with the worst case when a==0.5

My question is how can I avoid this ?
Specifically how can I compensate for the creation of a normal distribution curve when summing two distributions:

Forget the fbm and take a linear distribution say from -1 to +1 and add another (also from -1 to +1) to it using a*v1+(1-a)*v2 which will produce non-linear distributions for a!=0 and a!=1 including perfect standard distribution when a==0.5 - the question is what transform do I have to apply to the resulting data for values of "a" such that the final range is still -1 to +1 but the distribution is linear ?


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: cKleinhuis on March 08, 2010, 10:57:22 PM
just had to look it after by myself ;)
fBm= fractal Brownian Motion
and good ol'
http://en.wikipedia.org/wiki/Fractional_Brownian_motion

do you have graphs, demonstrating the current outcome?

0 and 1 clearly means either fbm1 or fbm2 is taken ;)

more i can not say at the moment :D
 :hmh: :hmh: :hmh: :hmh:


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: David Makin on March 08, 2010, 11:09:57 PM
just had to look it after by myself ;)
fBm= fractal Brownian Motion
and good ol'
http://en.wikipedia.org/wiki/Fractional_Brownian_motion

do you have graphs, demonstrating the current outcome?

0 and 1 clearly means either fbm1 or fbm2 is taken ;)

more i can not say at the moment :D
 :hmh: :hmh: :hmh: :hmh:


Well the fBm part is not really that important, it could be any set of data - it's being able to transform a normal distribution (or a mixed linear/normal distribution) into a linear distribution over the same range that I'm after :)

I'll post a gif anim of what happens when you use a*fbm1+(1-a)*fbm2 and vary a from 0 to 1 shortly....


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: David Makin on March 08, 2010, 11:39:57 PM
fBm Morph:

(http://www.fractalforums.com/gallery/1/141_08_03_10_11_38_42.gif)

This example shows a morph from one fBm pattern to another using:

colour = a*fBm1 + (1-a)*fBm2

where a varies from 0 to 1.

It's difficult to tell at full speed but the problem is that when a!=0 and a!=1 then the mixed results are smoother than the original fBm versions, worst case when a==0.5
The gif is probably too quick to see the problem but if you examine frames 0 (fBm1), 25 (fBm2) and compare the smoothness of these with frames 12/13 and 37/38 you'll see that the mixed frames are generally smoother.


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: cKleinhuis on March 08, 2010, 11:54:59 PM
so, it is simply, arent you basically doing a low-pass filter on
each of them,

if a <0.5 then it means you take more of the turbulence from fbm2 than from fbm1, and you are removing higher values
if a =0.5  ( would like to see 0,0.25,0.5,0.75 and 1.0) there is practically the same low-pass filter for both of them, resulting in smoothest result !
if a>0.5 then it means you take more of the higher values, and influence them with filtered values from the other

just a guess to describe what is going on, some kind of filtering is applied, and this fills in some gaps in the other formula


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: hobold on March 09, 2010, 12:13:22 AM
The sum of two random variables generally has a different distribution than either of the random variables. So any type of additive blend is going to mess with the fbm.

What are you trying to accomplish? Do you absolutely need a specific blend between two specific images? If not, then consider using three dimensional fbm, and intersect this volume with a moving plane. The image on the plane should be two dimensional fbm, and vary continuously as you move the plane.


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: Nahee_Enterprises on March 09, 2010, 12:27:07 AM
....consider using three dimensional fbm, and intersect this volume with a moving plane.
The image on the plane should be two dimensional fbm, and vary continuously as you move the plane.

Now that sounds like a good approach!!!    :dink:
 


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: David Makin on March 09, 2010, 12:31:11 AM
The fBm was the original reason I wanted to do this - but that's actually from around 5 or 6 years ago.

What brought me back to it was wanting a good way of colouring a 3D fractal based on angles relating to an orbit trapped "z" value.
Here I want a colouring value based on all three angles, i.e. in UF then based on:

atan2(zri)
atan2(imag(zri)+flip(zj))
atan2(zj+flip(real(zri)))

The simple solution is to sum these and divide by 3 *but* that suffers the same issue as the fBm i.e. the result will be a twice-over addition normal distribution rather than a linear distribution.


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: cKleinhuis on March 09, 2010, 12:35:25 AM
Quote
....consider using three dimensional fbm, and intersect this volume with a moving plane.
The image on the plane should be two dimensional fbm, and vary continuously as you move the plane.

Now that sounds like a good approach!!!    :dink:


ROFL, yeah, applying a non linear transform to the quadratic slice  could also yield interesting results!


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: David Makin on March 09, 2010, 12:36:09 AM
The sum of two random variables generally has a different distribution than either of the random variables. So any type of additive blend is going to mess with the fbm.

What are you trying to accomplish? Do you absolutely need a specific blend between two specific images? If not, then consider using three dimensional fbm, and intersect this volume with a moving plane. The image on the plane should be two dimensional fbm, and vary continuously as you move the plane.

Generating 3D fBm is considerably more expensive than 2D ;)
As I said I was just using fBm as an example, I'm really looking for a general method to remove the tendency towards a normal distribution when mixing datasets using addition or proportional addition.


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: Tglad on March 09, 2010, 01:18:55 AM
2 linear distributions become a binomial/quadratic distribution when added.
My guess is that (if you have 2 linear distributions -1 to 1) then if you square the values first before adding them it might result in a linear distribution, but not sure.

May be simpler to blend the input parameters rather than the output colours. So instead of (fbm(angle1) + fbm(angle2) + fbm(angle3))/3 you do fbm((angle1 + angle2 + angle3)/3)


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: Timeroot on March 09, 2010, 04:00:29 AM
I'm supposed to be working on my science project right now, but...

When the two are added, you get a distribution curve like p(x)=6*x*(1-x). For just one linear distribution you get p(x)=1. You're looking for a function t(x) so that Integral(p(x)dx|0 to t(x))=Integral(1 dx). So Int(6x-6x^2)(t(x))-Int(6x-6x^2)(0)=x -> 6*t(x)-6*t(x)^2=x. Then you just need some solution for t(x)... that is, assuming I made sense...


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: kram1032 on March 09, 2010, 09:41:57 AM
Hmmm...
did you try to use different gradients?
Like for instance (x²+y²+z²)^(1/2)/(3)^(1/2) or (x*y*z)^(1/3) or 3/(1/x+1/y+1/z)

They're a bit biased and tend to either the higher or the lower values, making the result probably less smooth. (x+y+z)/3 tends exactly towards the middle...


Title: Re: Linearising a normal distribution (for the math geeks)
Post by: David Makin on March 09, 2010, 10:59:33 PM
Hi all,

I found a solution that doesn't involve calculus just a conditional change to the final value.

Here's the relevant UF code for testing summing x/y values in a square from -1 to +1 both ways, note that real(#pixel)%1 and imag(#pixel)%1 both return -1 to +1 around the origin:

Code:
    float c = @scale*(real(#pixel)%1) + (1.0-@scale)*(imag(#pixel)%1)
    if @scale<=0.5
      if c<-1.0 + 2.0*@scale
        c = c + 2.0 - 2.0*@scale
      endif
      c = (c - @scale)/(1.0 - @scale)
    else
      if c<-1.0 + 2.0*(1.0-@scale)
        c = c + 2.0 - 2.0*(1.0-@scale)
      endif
      c = (c - (1.0-@scale))/(1.0 - (1.0-@scale))
    endif

This produces a completely linearly distributed result from -1 to 1 for all values of @scale from 0 to 1.

The method would make an interesting transform to use in Mandelbox-style fractals ;)

Edit: I just realised it's basically just a more sophisticated version of "folding" :)




Title: Re: Linearising a normal distribution (for the math geeks)
Post by: David Makin on March 10, 2010, 02:27:14 AM
In case anyone wants to play with the method here's a UF parameter file with test colouring (just copy and paste the whole lot into an open UF fractal window - needs UF5):

Code:
LinearDistribution {
::kDpHyjn2FiZSvNOOQU47BI/HE89Wh8pV3D4hpRwca65UfPgRi2mIaDSyZZ+1PUSO2cpSP5kV
  xivi6RWhfQHGlVzymvf/dRRz65GlY3fr7UyxH1Tzj6nPPr772F9mue+kIPlFdSpPeaWkWyia
  kfoGnE8l5WNqq1zTidPKfVXH9T5L6u/I5BO7Bw4sd3f3ayrlpSOsIqY3PkVvccs/cX9uo+BZ
  le+DBnxiaVzn6rFtnbm1Dypp7vrVOMo7OuNdV3saUwewko8Y3S5P0PaSWuOcr8d9y4LCNoGr
  OpqeR0f4Q0BdjqT2aeBbnjPfodXkRoxPWe8bD63VN7Wm+wT6uJdtS01f/db/czbGldTHM621
  3pu/u+zzhD1sacRjqBlc2koVJnVTmiW1cto1227Whv8af4HtmHmmVDCWMjx4mnkPL4Ps8jql
  YLDXJNbVs4ks9syit5bmI3820eI+czzf/n/8ve6PbmNB3dZs4Xf6oqTNqrWMnbzxE3MJpJ3n
  bF8V5XD32bshdPqfNinfVjJlqWwRSaWe5eLNWOH85vPYOTd7pWp24BD3CIfVNKPqE/jZXT2Y
  phedjd743nxq1VqbTt+cb7HCrV44wp1dqraoe3qSHlttylNiPzXXb8ssPfsp/NTyxZWT/k5w
  9ivnf9virZreV1MJKv6En6fzcucyt+HakHZiPUT2mBzoYBrgfzsYW2R7AbrH6zAdGFY2Cy9F
  kHII3Vi2BuVF6MKAc7FfRT4rJuopzmKcWnwRTzT8Y4Iayio2uhR3kg1aijoJOimISSiLy8c0
  U/FbagopOim67omA5xlFOr2MfVzCUNz3WzcWtZik9xJuqmT4B5BCn7sczdUNXgyYO8NhC/lb
  RgqFOqW4bCFis4c3dsSfRLDEtUAXPo0uJyIbpgvPO3zc3TYD7D0evAWCv3T49ik84s8AngH0
  hxDbxMh8W3rJZrvJQZcq3+HnTsy5EdccRqt4LZ4sSXKgpvLLOh7658gePOZznJqTFwaFsl30
  CmGzIMokgKE2GaCV6ZQLJxdLhphkFv33jSp8owmS+SbndBWSxb9uUkUBiL9dpg2TeGpLZaCt
  PEtmFLx71ITkiYe5n/nl539uq8Xjyhfd5i8dbJQdf5W4r34vCZcJoRtlBGIGc4JDAyka2c+7
  mMDP9vr3ivdVgdacxtJaQbqUD93ulZL87HsvpcLmWOP0/mJOuGa75U7ZuVCI+2Wrg9KhLSiT
  8Sc5ER6tL6xvBwAfBgBoBMAFgBuCYYN9bsFwhtA+sF4rYLABbBcYLgLbBCZLgNbB8YLgLbBC
  ZLwXzWAP2CQyWAf2CQwWAb2C4zWAH2C4zWAC2C4xWAH2C4zWAf2CQzWAb2C4wWgA2CQwWAC2
  CYzWAH2CQxWAf2CQwWAb2C4zWgQ2C4zWAC2C4xWAH2CEyWAC2CQwWAb2C4wWAS2C4zWAC2CY
  zWAf2CEwWAf2CQwWAP2C4xWAC2CQwWAC2CYzWAP2CQzWgA2CQxWAf2C4zWAC2CQxWAK2C4wW
  AK2CQxWgA2C8FsFwhtABsF4LYLQAbBoYLgPbBCZLAFbBoYLAFbBcZLANbBIYLQAbB+C2C4yW
  AK2CQzWg/P2CQzWAK2C87YLAFbBCYLQAbBoZLABbBCZLQIbBoYLABbBCYLOOKr1qu51vmyUb
  f/8p1tHdXt6dRGLqqvp3U6EeJA7SY+14ceSSRBKuMAyWNitxyyT3vvAXGK56cKTTTzX2sv8d
  nsLt5O9t0ZX/qUIzs3+fFtqu1C==
}

test.ucl:distribution {
::7zFuIjn29e1SPtNQQ47Vq/HWloK5H4gtbpHKYEtHKqH4WvRRVOb2QWJ/q2bAn/9d2He31bcg
  QLtRCi9Mfz38wzMez9F1LzL+0bfDCRLbqbZoZllrXstY5MbZ46yy6Kt4bu5r/8zFs1fpEB/l
  gyQVkHRXxv2L2nj47t5VdrJtIWvWNcpS7Y7TtsPdf7TN2PoluGdVJhtpeVWWKXASEHLuWkNf
  riyUAFyT3XOpaFdtgpKG6hbn/IdFbTYydcRrLqzZodgXjSWEbk0vnEGIBu9cJNdEyKQQS67/
  wZfcgbqb4KMvl0QyZyA0mWjmzRiPgl9XErvFJcZU3vaZeRxLOLo3XrjU0RcAKw5CTm5G+3tH
  /MUEy4idPpLYoQjb2dI3IKUAA2pCspqnNSFX1hzLIB9ATeQdISevfwuBQ8ynQ2FZAJDSFyZX
  wLdglpLiDkgM6NRIoFyppwobEMG4xtIxxC/T9cJRnEyyyhiK7c6pjtpR+8RotdjizRK0RrNf
  UgNoXuu1jFIHC8H9QLfZnnHvxODmHXVXKuGcyc5tw/vn4HeUoM9cafybJkeNgpjvHuleHgQ8
  FklSxbrY0CkXfWf4VdMSj/lqJGlidZ7Gp4cpbkXNamqpFGL9onK9LvD/kZoZnwdnMEVMSzoh
  J+XmJxZtzYNtaYhp7YNfgvRndtk8Cv5d4WCpqh2TK8H2dNfDhe/GWEtM/+xIu4hbbuLQpPIJ
  OeRMfkxLJM9UZ+pKTz7qLo81Ns2tkRb04dj2RWiKxnTrWR6FDipnF4pCPpbfXiPf6TGP2igu
  oh9oiNeYz86UEYP+6ymkmpnlBp4naSm7W8TMHPebDHtH+FOErMKCphKteEM7Jd3Ye6p39j8p
  w9sx/LbEXnLOmJ5bCTs2JY1mAdJ4wkhBXZ+Keod+4cbYiT37hvMbfcR24syXtD5Up6JM1Z4M
  DLG9+c4QAjFsXjpwEGpNnR8knKgwPpxQroP6H2rh5fc6cFuwQQqLBqAX5Y+xbwSG5Bh4hCmX
  nGwNk+mSioUGPq+oenO23qSOCgQvRtVBUrN1PA//YKTFVOdKJhH9Mn5dBaOedmXjmAEeysV0
  lqP1l9jpUjQrxi/1T+HZN942D8813Xt9JRH0Asu64uF9Po6fo9TAHhY7T0visOfbBT8+5m82
  8Sk89gD8stEMd21t5Nbmx/muC+CGunJBoMnfMeFpCSMsxfHsLU4l0xxqDq8Atjusgw3GN6oB
  7zU+ShFGa8SOZ4tkHPL4JCmXKHWDG28cmLarf1m0Q+21D6sUXz1/kOpxwaaXfydAgqZA5xz9
  UhW6fRol+aFa/GgjZCsJ
}

For the fractal as it appears try "explore"ing the "scale" parameter from 0 to 1 and you'll see that although the fractal morphs nicely it does not change in relative smoothness.
Note that there is a slight extra "fix" to the calculations which basically applies an offset to the value based on the scale to avoid the colours/heights sliding around the gradient as the scale is changed.