Welcome to Fractal Forums

Fractal Software => Tutorials => Topic started by: visual.bermarte on April 02, 2015, 02:53:25 PM




Title: Blobby (How to write simple m3f)
Post by: visual.bermarte on April 02, 2015, 02:53:25 PM
(http://orig07.deviantart.net/99d7/f/2015/092/2/8/mandelbulb3d_tryout_by_bermarte-d8o35or.jpg)
This is my test using mandelbulb3d, I was curious about the method used to create formulas  in assembly ;D
What was interesting to me was the method and not the formula itself (it's rather meaningless).
I took a formula and I have used http://www2.onlinedisassembler.com/odaweb/ to disassemble the hex code.
Then I have modified the most basic _FlipXY.m3f formula like that (it's just a test, just trying out things):

push ebp
mov ebp,esp
fld QWORD PTR [ecx]
fld QWORD PTR [eax]
fsin
fcos
fstp QWORD PTR [ecx]
fstp QWORD PTR [eax]
pop ebp
ret 0x8

Latest step was using https://defuse.ca/online-x86-assembler.htm#disassembly to assemble back in Hex code (for x86)  the assembly code, to obtain
5589E5DD01DD00D9FED9FFDD19DD185DC20800

For my image first I have used the modifier I did and then I have added a formula called 'Integer Power' (here is 4).


Title: Re: Blobby
Post by: visual.bermarte on April 02, 2015, 05:48:16 PM
I have to write again that just throwing two functions one after the other (sinus and cosinus)  like I did in the example, does not make sense at all.  :tease:


Title: Re: Blobby
Post by: knighty on April 02, 2015, 06:59:10 PM
 :thumbsup1:
Also, thank you for the links.
They use GCC for the assembler. Maybe it is possible to automate the process. C/C++ to assembly code to MB3D formula? Anyone?


Title: Re: Blobby
Post by: DarkBeam on April 02, 2015, 07:04:04 PM
Never worked here! ;) ;)


Title: Re: Blobby
Post by: 3dickulus on April 04, 2015, 10:31:12 PM
https://gcc.gnu.org/onlinedocs/
gcc option -mfpmath=sse
Quote
For the i386 compiler, you need to use -march=cpu-type, -msse or -msse2 switches to enable SSE extensions and make this option effective.


Title: Re: Blobby
Post by: DarkBeam on April 04, 2015, 10:38:10 PM
Knighty do you know an instant method to get just the mb3d code, without spaces, without copypaste it everytime? :)


Title: Re: Blobby
Post by: 3dickulus on April 04, 2015, 11:00:47 PM
Code:
> /usr/bin/hexdump obj.o | sed -ne '/^0000...\ /s/^0000...\ //g;p' | sed -ne '/\ /s/\ //g;p' 
will output only hex values. (tail end of a random file) looks like...
Code:
414178452070203d0a31614773756973
6e6141413d20742075720a656523646e
727073657465000a

edit:can add to a make file/script like...
Code:
...make cmds...
cat obj.o | hexdump  | sed -ne '/^0000...\ /s/^0000...\ //g;p' | sed -ne '/\ /s/\ //g;p' > hexfile.txt 
...creates file named hexfile.txt


Title: Re: Blobby
Post by: marius on April 05, 2015, 01:36:47 AM
It wokrs here!  :horsie:
I used this example posted here a looong time ago by jesse:
Code:
__attribute__((packed)) struct TIteration3Dext {
double Cw,Rold,RStopD,x,y,z,w; // use with neg indizes before C1. w is also used for 3d ifs and abox analytic DE
double Px, Py, Pz;             //      actual position, never change these!  Can be used as input.
double Cx, Cy, Cz;             //+24   these are the constants for adding. Pxyz or the julia seed.  Cw is @-56 (begin of struct)
void*  PVar;                   //+48   the actual formulas adress of constants and vars, constants at offset 0 increasing, user vars below -8
float  SmoothItD;              //+52
double Rout;                   //+56   the square of the vector length, can be used as input
int    ItResultI;              //+64
int    maxIt;                  //+68
float  RStop;                  //+72
int    nHybrid[6];             //+76   all formulas iteration counts or single weights in interpol-hybrid
void*  fHPVar[6];              //+100  all formulas pointer to constants+vars, PVars-8=0.5, use PVar for the actual formula
void*  fHybrid[6];             //+124  the formulas adresses
int    CalcSIT;                //+148
int    DoJulia;                //+152
float  LNRStop;                //+156
int    DEoption;               //+160
float  fHln[6];                //+164  for SmoothIts
int    iRepeatFrom;            //+188
double OTrap;                  //+192
double VaryScale;              //+200  to use in vary by its
int    bFirstIt;               //+208  used also as iteration count, is set to 0 on it-start
int    bTmp;                   //+212  tmpBuf, free of use.
double Dfree1;                 //+216
double Dfree2;                 //+224
double Deriv1;                 //+232  for 4D first deriv or as full derivs
double Deriv2;                 //+240
double Deriv3;                 //+248
float  SMatrix4[4][4];         //+256  for 4d rotation, used like most other values only by the programs iteration loop procedure
};

// fastcall is not quite delphi fastcall.
// first two args are ok, third is in ecx in delphi, on stack here.
void __attribute__((fastcall)) formula(
             double* x,      // [eax]
             double* y,      // [edx]
             double* arg,    // [ebp+8], points to TIteration3Dext.C1
             void* dummy     // so we end w/ ret 8 as delphi expects
             ) {
  // Compute ptr to proper start of TIteration3Dext struct.
  struct TIteration3Dext* cfg = (struct TIteration3Dext*)(arg-7);

  // Read / write some fields as demonstration.. not a real formula ;-)
  double r = cfg->x + cfg->y - cfg->z * cfg->w;
  cfg->Deriv1 = r;
}

compile with 'gcc -c -Os -m32'.

Objdump -D can be used to show the generated code.

Heh, that code looks familiar ;D Wow, been over 3.5 years already.. I hope you're not the first to actually try it  :hmh:
Too bad m3d is not 64 bit, eh?


Title: Re: Blobby
Post by: knighty on April 05, 2015, 03:58:32 PM
@Marius: It was this post (http://www.fractalforums.com/mandelbulb-3d/asking-permission/msg27618/#msg27618). It must be your code. ;D

@3Dickulus: Thank you. I'll try them ASAP.

@Darkbeam: I think 3Dickulus partially answers your question. The idea is to write a script (sh, bat or so) that given the source will:
- call the compiler to generate the object file.
- extract the machine code from the object file. Also verify that no external function is used. objdump have a lot of options to explore.
- construct the m3f file using the extracted machine code and some informations from the source (for example: user defined variables and constants).
This requires using existing (if any, otherwise write them) programs to parse the source and object files and extract the informations.
The best IMHO is to have a system where the formula is written this way:
Code:
[OPTIONS]
.Version = 2
.DEoption = -1
[constants and variables]
[SOURCE]
  cfg->x=fabs(cfg->x);
  cfg->y=fabs(cfg->y);
  cfg->z=fabs(cfg->z);
[END]

Description:

Test of compiling a formula with GCC.
It simply set (x,y,z)=abs(x,y,z)
and then generates the c file automagicaly.  ::)
That needs some work but it is not impossible.


Title: Re: Blobby
Post by: 3dickulus on April 05, 2015, 06:39:23 PM
compile with GCC debug options turned on so objdump can insert the C source as comments in the assembly language output :)
you may find that the GCC optimizations are hard to improve ;)


Title: Re: Blobby
Post by: knighty on April 05, 2015, 09:35:29 PM
Yes! it is very good at optimizing.
I've tried to use sse but for fabs, it needs a constant (0x7fffffffffffffff) that is stored in .rdata section :hurt:. Hopefully it didn't freez the computer. So it is better not to use sse (or perhaps as intrinsic/inlined asm?)

Knighty do you know an instant method to get just the mb3d code, without spaces, without copypaste it everytime? :)
using:
Code:
objcopy -Obinary -j .text MB3D-formula-ex.o out.bin
extracts the binary machine code then:
Code:
bin2hex out.bin out.txt
converts it to hexadecimal. bin2hex can be found here (http://sourceforge.net/projects/bin2hex/files/v.%200.1b/).

I forgot to say that I use mingw and msys.


Title: Re: Blobby
Post by: DarkBeam on April 05, 2015, 10:00:48 PM
Ouch, that was the reason why I avoided the c compiler! Those issues are irritating to me. There are even worse ones sometimes


Title: Re: Blobby
Post by: knighty on April 05, 2015, 10:09:33 PM
Maybe MB3D-v2 will feature a formula compiler.  :evil1:


Title: Re: Blobby
Post by: DarkBeam on April 05, 2015, 10:45:14 PM
Yes! Andreas will try to do that :) so that pain will end? :D
Some of my formulas need serious work.
Also I would like to see a serious ifs general work like the one you did, together with many otger things I am unable to do.
I still wonder how a data section can be replicated in a plain assembly code...


Title: Re: Blobby (How to write simple m3f)
Post by: visual.bermarte on April 07, 2015, 10:33:33 AM
 :) How can one write a trigonometric function using this c example? for example sin(x).


Title: Re: Blobby (How to write simple m3f)
Post by: DarkBeam on April 07, 2015, 12:53:27 PM
I bet there's why the doom begins :(

https://msdn.microsoft.com/en-us/library/26td21ds(VS.80).aspx

I had a "tinymath.h" library, not sure of the name, that translated directly into assembly macros.
EDIT! Found: http://www.cubic.org/docs/download/tnymath2.h
https://github.com/HotKeyIt/ahkdll/blob/master/source/qmath.h

Still, there's the problem you can't deal with constants and user input...


Title: Re: Blobby (How to write simple m3f)
Post by: knighty on April 07, 2015, 09:22:40 PM
Ah yes haven't tested that. sorry!
One needs to set -ffast-math in order to have gcc compile to native cpu floating point operations. Here are my experiments:
The source. It is just like visual's OP but used x and y instead:
Code:
#include <math.h>
__attribute__((packed)) struct TIteration3Dext {
double Cw,Rold,RStopD,x,y,z,w; // use with neg indizes before C1. w is also used for 3d ifs and abox analytic DE
double Px, Py, Pz;             //      actual position, never change these!  Can be used as input.
double Cx, Cy, Cz;             //+24   these are the constants for adding. Pxyz or the julia seed.  Cw is @-56 (begin of struct)
void*  PVar;                   //+48   the actual formulas adress of constants and vars, constants at offset 0 increasing, user vars below -8
float  SmoothItD;              //+52
double Rout;                   //+56   the square of the vector length, can be used as input
int    ItResultI;              //+64
int    maxIt;                  //+68
float  RStop;                  //+72
int    nHybrid[6];             //+76   all formulas iteration counts or single weights in interpol-hybrid
void*  fHPVar[6];              //+100  all formulas pointer to constants+vars, PVars-8=0.5, use PVar for the actual formula
void*  fHybrid[6];             //+124  the formulas adresses
int    CalcSIT;                //+148
int    DoJulia;                //+152
float  LNRStop;                //+156
int    DEoption;               //+160
float  fHln[6];                //+164  for SmoothIts
int    iRepeatFrom;            //+188
double OTrap;                  //+192
double VaryScale;              //+200  to use in vary by its
int    bFirstIt;               //+208  used also as iteration count, is set to 0 on it-start
int    bTmp;                   //+212  tmpBuf, free of use.
double Dfree1;                 //+216
double Dfree2;                 //+224
double Deriv1;                 //+232  for 4D first deriv or as full derivs
double Deriv2;                 //+240
double Deriv3;                 //+248
float  SMatrix4[4][4];         //+256  for 4d rotation, used like most other values only by the programs iteration loop procedure
};

// fastcall is not quite delphi fastcall.
// first two args are ok, third is in ecx in delphi, on stack here.
void __attribute__((fastcall)) formula(
             double* x,      // [eax]
             double* y,      // [edx]
             double* arg,    // [ebp+8], points to TIteration3Dext.C1
             void* dummy     // so we end w/ ret 8 as delphi expects
             ) {
  // Compute ptr to proper start of TIteration3Dext struct.
  struct TIteration3Dext* cfg = (struct TIteration3Dext*)(arg-7);

  // Read / write some fields as demonstration.. not a real formula ;-)
  double tmp=cfg->x;
  cfg->x=cos(sin(cfg->y));
  cfg->y=tmp;
}
Compile with
Code:
gcc -c -m32 -O3 -mfpmath=387 -ffast-math sineswap.c
Otput assembly juste to see if there are no library calls and no data sections:
Code:
objdump -D sineswap.o
Code:
sineswap.o:     file format pe-i386


Disassembly of section .text:

00000000 <@formula@16>:
   0:   55                      push   %ebp
   1:   89 e5                   mov    %esp,%ebp
   3:   8b 45 08                mov    0x8(%ebp),%eax
   6:   83 e8 38                sub    $0x38,%eax
   9:   8b 50 18                mov    0x18(%eax),%edx
   c:   8b 48 1c                mov    0x1c(%eax),%ecx
   f:   dd 40 20                fldl   0x20(%eax)
  12:   d9 fe                   fsin
  14:   d9 ff                   fcos
  16:   dd 58 18                fstpl  0x18(%eax)
  19:   89 50 20                mov    %edx,0x20(%eax)
  1c:   89 48 24                mov    %ecx,0x24(%eax)
  1f:   c9                      leave
  20:   c2 08 00                ret    $0x8
  23:   90                      nop
Then take only the machine code:
Code:
objcopy -Obinary -j .text sineswap.o sineswap.bin
Then convert the binary file to hexadecimal values in text file:
Code:
bin2hex sineswap.bin sineswap.txt
then add MB3D stuff and save as sineswap.m3f :
Code:
[OPTIONS]
.Version = 2
.DEoption = -1
[CODE]
5589E58B450883E8388B50188B481CDD4020D9FED9FFDD5818895020894824C9C2080090
[END]

Description:

Test of compiling a formula with GCC.

Now the parameters:
Code:
Mandelbulb3Dv18{
b.....Y/...g2...w....26...U2xSE.FEG1.TLyGzcFHn2Ezkskg1RPYyvTB4OmJcqszipiyP.9Q4zj
................................3CcamACAtz1.......EO./..................y.2...wD
...Uz.....UR..../wU0/......q/...W/....E3.....2LNaiUt1joD/w.........m/dkpXm1.....
z.UaNadD12..0..........wz..................................u1....y1...sD...../..
.z1...kDmxchI3E6bxnNtGvRevNwy6EccKb8ZDlDyOAa4uMzGwvZyxtOCCPKzUDDlGyIvAqD7f84AINX
xvfDDzwW9qnMz8kgNCnrmapD......Ia2...g7........sD.6....sD..E.....................
.............oAnAt1...sD....z.Lv15.2iDQ.kuyk/.pv15.wjDQ.E0zk/.Hw15......Q....k1.
..................kz.wzzz1EM.c6.P....61...EK....Q/...c3...UD....5/...I1.....SF52
...U.ydelyjeYFnzTeOgzf8No.6.2c..zzzz............4............0........k.8.kXWF1.
.sM93P58iz9.MmnWK2zwz0........../EU0.wzzz1...........s/...................E.2c..
zzzz.............0...................2./8.kzzzD............8....................
/EU0.wzzz1....................................uBZ.U7EgU7tj/eVv1.oQm2oElCnueM.YXB
bYX7Ot6Ul/.8Gsm0Drb3jC4.a.l0aUWTUSH7.M029MmWy/uBZ.U7EgU7hv5crI0.a.l0aIoTUSH7.M02
9Mmfz/uBZ.U7EgU7...crIGJzzFoTuIdyzngi8qdxzZX.4rU................................
E....6....E.....I.........kQdtKNnRLMk/..........................................
...................6./..........................................................
................................................................................
.....................2..........0....YYPoJqNZ756ExqRZ75.........................
8............................./E........kz9.....................................
................................................................................
................................}
The picture :)[/code]


Title: Re: Blobby (How to write simple m3f)
Post by: knighty on April 07, 2015, 09:29:08 PM
Still, there's the problem you can't deal with constants and user input...
In the TIteration3Dext structure there is:
void*  PVar;                   //+48   the actual formulas adress of constants and vars, constants at offset 0 increasing, user vars below -8

That's something I'll try to see how to use it. Marius, any tip?


Title: Re: Blobby (How to write simple m3f)
Post by: DarkBeam on April 07, 2015, 09:54:19 PM
Uhm. As for the constants they sit in EDI, EDI+8h, EDI+(8h*N) and so on if they are doubles!
User data.
In case of doubles first one is EDI-10h, -18h, -20h etc, integers have a size of 4... so correct addresses accordingly
;)
angles are stored in matrix form, I always paste the whole Jesse code everytime but manually correcting offsets.
Be careful with nonstandard data like boxscale, reciprocal...
ESI contains Cx Cy and Cz, don't remember offsets but see any escapetime formula.
Pay attention, some ABox formulas swap ESI / EDI...
We are doing a mosaic of sorts!


Title: Re: Blobby (How to write simple m3f)
Post by: knighty on April 07, 2015, 10:46:50 PM
There are a lot of things that I don't understand.  :embarrass:
What are boxscale and reciprocal?
What do .DEoption mean?
How does MB3D know that a formula is a dIFS a 3D or 4D ...?

It should be possible to make structs for constants and variables. Something like:
Code:
struct constants{
    double const1;
    double const2;
    ...etc.
};
struct variables{
   ...etc. //they should be inserted in reverse order?
   float var4;//[edi-20h]
   longint var3;//[edi-1ch]
   double var2;//[edi-18h]
   double var1;//[edi-10h]
   double dummy; //corrsponding to [edi-08h] because the "1st" accessible variable is at -0x10
}
...
//In formula
   struct constants *pconst=(struct constants *) cfg->pVar;
   struct variables *pvar=(struct variables *)(((char *)pconst)-sizeof(struct variables));//not sure about the syntax or even if this is correct.
...
:help:


Title: Re: Blobby (How to write simple m3f)
Post by: DarkBeam on April 08, 2015, 12:07:45 AM
Dear friend I am in bad phisical conditions and I cannot be of big help just now. :(
Boxscale is used in Amazing box in conjunction with minr, but don't remember well how. However there should be two doubles for a boxscale.
Reciprocal is a type that loads in memory 1./the user input, there are also others like .singleangle and the aforementioned .3singleangle ones, also stuff for 4d but my memory doesn't help me now. Those types are less frequent but you may want to use them.


Title: Re: Blobby (How to write simple m3f)
Post by: DarkBeam on April 08, 2015, 12:10:24 AM
The deoption tells mb3d if a formula is ads, escapetime, ifs etc
menger3 has a different option than ruckerbulb :)


Title: Re: Blobby
Post by: thargor6 on April 08, 2015, 12:13:52 AM
int    nHybrid[6];             //+76   all formulas iteration counts or single weights in interpol-hybrid
void*  fHPVar[6];              //+100  all formulas pointer to constants+vars, PVars-8=0.5, use PVar for the actual formula
void*  fHybrid[6];             //+124  the formulas adresses
Especially this has to change when we extend the number of possible formulas. Extending those arrays did not work for me, as it causes a lot of changes in ASM-code, some really tricky, because all offsets after this arrays will shift.
My current idea is to create new offsets for those arrays, rather than extending them, so all offsets after those structures remain the same, but the three arrays will shift.
Hopefully they are not used from within external formulas, but this currently is just a guess as I did not test a lot yet.


Title: Re: Blobby
Post by: thargor6 on April 08, 2015, 12:27:46 AM
Maybe MB3D-v2 will feature a formula compiler.  :evil1:
I also thought about something like your described, i. e. using some external compiler for this (in the first way).
It is rather cheap, but of great use, I think. Even if it is not optimized for death, it could be really great fun to be able to just enter some C-style code (or whatever compiler to use).




Title: Re: Blobby (How to write simple m3f)
Post by: knighty on April 08, 2015, 10:03:36 PM
Thank you very much DarkBeam for the infos. I remember you already gave me the full explanation about the variables types. Thank you again and get well soon.  :)

@thargor6:
I don't think those tables are used by any formula but the only way to be sure it to verify all of them. This could be done automatically I gess: disassemble them then look which parameters are used/modified.  :D
I also thought about something like your described, i. e. using some external compiler for this (in the first way).
It is rather cheap, but of great use, I think. Even if it is not optimized for death, it could be really great fun to be able to just enter some C-style code (or whatever compiler to use).
Maybe it would be better to use dlls loaded at runtime (like gnofract4d) in order to avoid all the complications of integrating the machine code from the object file. For even better execution speed one can generate the source for the whole parameter (including all the used formulas) -> no overhead due to calls and more opportunities for the compiler to optimize.
But the best of all (and the most difficult to do) is a specialized compiler with jit using LLVM. but LLVM is a monster.  ;D
Both approaches would need to rewrite all the existing formulas for backward compatibility. :hurt:

Good news! the code I posted for accessing the constants and variables is correct!
see attachement.


Title: Re: Blobby (How to write simple m3f)
Post by: thargor6 on April 08, 2015, 10:32:15 PM
This could be done automatically I gess: disassemble them then look which parameters are used/modified.  :D
Right, good idea, I will have to do that!

Maybe it would be better to use dlls loaded at runtime (like gnofract4d) in order to avoid all the complications of integrating the machine code from the object file.
Reminds me a little bit on the dll-hell from Apo :-), but is also a good idea, compared to the current approach. I think, we should just try out some things to actually decide, rewriting all the formulas is a nightmare ;-)

Good news! the code I posted for accessing the constants and variables is correct!
Nice!

Best regards,
Andreas


Title: Re: Blobby (How to write simple m3f)
Post by: DarkBeam on April 08, 2015, 11:34:43 PM
Rewriting all the formulas?! Andreas you are trying to kill me by heart attack :dontcare:


Title: Re: Blobby (How to write simple m3f)
Post by: thargor6 on April 09, 2015, 12:38:32 AM
Rewriting all the formulas?! Andreas you are trying to kill me by heart attack :dontcare:
At least you are awake now ;-)  :beer:


Title: Re: Blobby (How to write simple m3f)
Post by: DarkBeam on April 10, 2015, 07:10:41 PM
 :dink:
Published a resource about how to program DIFS shapes, including some examples. I hope everyone finds it interesting.  :beer:

http://fav.me/d8p65ye


Title: Re: Blobby (How to write simple m3f)
Post by: knighty on April 10, 2015, 08:32:16 PM
Thank you very much DarkBeam.
Reminds me a little bit on the dll-hell from Apo :-)
Nooo! :D
It's not a good idea to have a dll for each formula because most formulas are very small in size. What I was meaning is to take the whole fractal "parameter" that is the formulas sequence, generate the source from it then a dll by invoking the compiler/tools and finally loading and executing it. Maybe this would not be a good approache for small "parameters".


Title: Re: Blobby (How to write simple m3f)
Post by: thargor6 on April 10, 2015, 10:06:49 PM
generate the source from it then a dll by invoking the compiler/tools and finally loading and executing it.
Hmmm :-), but this would be a rather large change. And would it slow down to change parameters (e.g. to try out new combinations of formulas) "on-the-fly".
I like the idea of an integrated compiler where the user does not need much external tools and it is clear what the formula does, maybe even in the Delphi language like this: http://www.paxcompiler.com/


Title: Re: Blobby (How to write simple m3f)
Post by: DarkBeam on April 10, 2015, 10:36:33 PM
But afterall Andreas a fractint-like environment is largely enough... just provide x y z w to users then they do all they need :D just my 5 cents


Title: Re: Blobby (How to write simple m3f)
Post by: knighty on April 12, 2015, 11:39:39 PM
Thank you targor for the link. Very interesting.
DarkBeam is right. Those embedded compilers are too heavy fully featured for "simple" fractal formulas. There is also eval compiler (http://advsys.net/ken/download.htm) which is small but still too big IMHO and is not mantained anymore. I did once a simple math formula compiler (in turbo pascal). It compiles to a bytecode not native code generation... too complicated. :embarrass:

Here is my attempt to dIFS. This simple example draws filleted boxes. The generated code is ugly but it works.  :horsie:


Title: Re: Blobby (How to write simple m3f)
Post by: DarkBeam on April 13, 2015, 01:32:24 PM
How in the world you did this :o you never fail to amaze me :beer: :beer: :beer:
The generated code is indeed ultra-long but fantastic solution!


Title: Re: Blobby (How to write simple m3f)
Post by: knighty on April 16, 2015, 03:46:27 PM
Just noticed that there is a problem with the dIFS example: The following formulas are not called at all. Maybe because some registers are overwritten? I was assuming that the formula is called from delphi with delphi calling convention. It doesn't seem to be the case. I'll have to investigate more. :-/
The generated code becomes nicer when using gcc switch: -march=pentium4. It uses conditionnal fpu moves which saves many jumps.
Regarding using sse instructions, the main problem is with absolute value and negating. Both operations are translated to a logical sse operation with a constant that is stored in .rdata section. In theory, the obvious solution would be to make it use constants that are stored in the m3f file (in [CONSTANTS] section) Just like in boxIFS.m3f. In practice this is far from being obvious apart from manually editing the assembly code. Pity! the generated sse code is so elegant.


Title: Re: Blobby (How to write simple m3f)
Post by: DarkBeam on April 16, 2015, 05:05:40 PM
Ok I didn't want to put you down and I hoped it weren't a problem but I somewhat expected some kind of trouble.
Jesse told me DIFS were "simpler" than normal escapetime 'cuz edi and esi are preset with the correct values by mb itself, so the only thing needed is taking care of stuff, preloaded into ESP after the applyscale intro code... (as explained in my commented assembly :) )
So I guess we need to replicate this with a function with no ESI EDI modification at all, and the predefined intro/ out code :)
Can we do this mixing c and assembly? I hope so :)
The same goes for the rotation matrix :)


Title: Re: Blobby (How to write simple m3f)
Post by: knighty on April 17, 2015, 12:03:54 AM
Here is another ugly solution: The registers are saved before calling the actual formula then restored. I don't know which registers must be restored. The compiler tries to not modify ebx, esi and edi. beside that I'm lost.
Now it works fine.
Code:
/*
Mandelbulb 3d Formula example.
Original code by marius and jesse.
Modified by knighty. Apr 2015.
Work in progress.
No warranty.
Compilation using mingw and msys:
-Compile with (replace "theFormulaFilename" by actual c file name):
gcc -c -m32 -O3 -mfpmath=387 -ffast-math -march=pentium4 theFormulaFilename.c
  For now sse is not used because Gcc is too
 good at optimizing :)) for example abs() translates to an sse and operation against 0x7fff...
 but unfortunately that constant is put in .rdata section.
 Trying to emulate it in c gives ugly code with a lot of registers/memory moves.
 Maybe using intrinsics? Any idea?
-Verify asm code:
objdumb -D theFormulaFilename.o
  This is necessary to check that there are no library functions call and that there is
 only .text "segment" no .rdata or something like that or a call to external function.
-Extract machine code:
objcopy -Obinary -j .text theFormulaFilename.o theFormulaFilename.bin
-Convert machine code from binary to hexadecimal:
bin2hex theFormulaFilename.bin theFormulaFilename.m3f
(bin2hex is not part of msys or mingw. any other binary file editor would do the job)
-edit theFormulaFilename.m3f in a text editor to add MB3D stuff.
*/
/*Includes*************************************************/
#include <math.h>
/*MB3D structures definitions*************************************************/
//This structure is specific to dIFS. The use of most of it is unknown and should not be modified. 
__attribute__((packed)) struct TIteration3Dext {
double something; // -0x88 ; used in sphereheightmap.m3f
double dum0; // -0x80 ; unknown
double x; // -0x78
double y; // -0x70
double z; // -0x68
double dum1[8]; //unknown
double DE2T; // -0x20 ; Output: distance estimate to current object
double dum2[17]; //unknown
double accumulatedScale; // +0x70
double dum3; // +0x78; unknown
double OTforCol; // +0x80
double dum4[16]; //unknown
//void * sphericalMap; // +0x108; pointer to function
};
__attribute__((packed)) struct Sconsts{
double Pie;//just to use a constant
long long int abscst;
};
__attribute__((packed)) struct Svars{//in the reverse order wrt m3f file
double Zadd;
double Yadd;
double Xadd;
double Scale;
double Bevel;
double HalfLengthZ;
double HalfLengthY;
double HalfLengthX;
double Dummy;//not used. What is it good for?
};
/*Macros**************************************************/
#define MAX(x,y) ((x)>(y) ? (x) : (y))
/*Local functions declaration*************************************************/
//Routines must be declared here and implemented after formula().
//This is in order to set the beginning of the machine code at the beginning of formula.
//Notice that if optimizations are disabled, the compiler may output routines code before formula's code.
inline void TheFormula(char* siarg, char* diarg);
/*Formula implementation*************************************************/
// Not a standard calling convention. this seems to be called from an asm code in MB3D.
// So we need to save almost all registers. the compiler generates the code to save/restore ebp
// esi and edi are not modified (or restored?)!???
//
// the arguments of this function are in edi and esi registers.
// esi points to the context structure
// edi points to constants (negative displacements for variables)
//

void formula(void) {//Big overhead. The only solution I could find. Any Idea?
asm("push %eax\n\tpush %ebx\n\tpush %ecx\n\tpush %edx\n\t");//save registers :/
asm("push %edi\n\tpush %esi\n\t");//arguments to TheFormula
asm("call _TheFormula");
asm("pop %edx\n\tpop %edx\n\t");//clean stack because the call convention is C's. It coud be: addl $0x08,esp instead.
asm("pop %edx\n\tpop %ecx\n\tpop %ebx\n\tpop %eax\n\t");//restore registers :/
}

/*Local functions implementation*************************************************/
inline void TheFormula(char* siarg, char* diarg) {
//DO NOT MODIFY BEGIN
// Compute ptr to proper start of TIteration3Dext struct.
  struct TIteration3Dext* pctx = (struct TIteration3Dext*)(siarg-0x88);
  // get pointer to constants.
  struct Sconsts* pconst=(struct Sconsts*) (diarg);
  // get pointer to variables.
  struct Svars*   pvar=(struct Svars*) (((char*)pconst)-sizeof(struct Svars));
//DO NOT MODIFY END
  // Draw a box
  double x=pctx->x, y=pctx->y, z=pctx->z;
  x=fabs(x) - (pvar->HalfLengthX - pvar->Bevel);
  y=fabs(y) - (pvar->HalfLengthY - pvar->Bevel);
  z=fabs(z) - (pvar->HalfLengthZ - pvar->Bevel);
  double DE = MAX(x,MAX(y,z));
  if (DE>0.){
x = MAX(0.,x);
y = MAX(0.,y);
z = MAX(0.,z);
DE = MAX(DE, sqrt(x*x+y*y+z*z));
  }
  pctx->DE2T = DE - pvar->Bevel;
  // Now do translation and scaling
  pctx->x = pctx->x * pvar->Scale + pvar->Xadd;
  pctx->y = pctx->y * pvar->Scale + pvar->Yadd;
  pctx->z = pctx->z * pvar->Scale + pvar->Zadd;
  pctx->accumulatedScale = pctx->accumulatedScale * pvar->Scale;
}
The MB3D formula:
Code:
[OPTIONS]
.Version = 6
.DEoption = 20
.Double X halfwidth = 1
.Double Y halfwidth = 1
.Double Z halfwidth = 1
.Double Fillet = 0
.Double Scale = 1
.Double X add = 0
.Double Y add = 0
.Double Z add = 0
[CONSTANTS]
Double = 0
INT64 = $7FFFFFFFFFFFFFFF
[CODE]
5589E5505351525756E8080000005A5A5A595B585DC35589E583EC088B4D0C8B45082D880000
008D51B8DD4010DD4018DD4020DD55F8DD4220D9C3D9E1DC6238D8C1D9C3D9E1DC6230D8C2D9
CBD9E1DC6228D8C2D9CBDBF1D9C1DBC1DBF4DAC4D9EED9C9DBF1762ED9CBDBF1DAC1D8C8D9CA
DBF1DAC1D8C8D9CDDBF1DAC1DDD9D8C8D9CCDEC1DEC3D9CAD9FAD9CADBF2DAC2DDDAEB08DDDD
DDD8DDD8DDD8DEE9DD5868DD4218DCCAD9CADC4210DD5810D8C9DC4208DD5818DD45F8D8C9DC
41B8DD5820DC88F8000000DD98F8000000C9C3909090
[END]

Box shape dIFS with rounded edge. no OTrap coloring for now.
[/code]


Title: Re: Blobby (How to write simple m3f)
Post by: DarkBeam on April 17, 2015, 12:14:25 AM
Hummm... should be ok as edx and ebx are usable while you keep intact other ones.
But I must test it extensively  O0 lol
And why you "call" an inline formula :D funny but if it works I am fine ;)


Title: Re: Blobby (How to write simple m3f)
Post by: knighty on April 17, 2015, 08:07:24 PM
Well, I had to save all those registers in order to make it work correctly.
I'm studying MB3D source code to understand what is going on. Very complex and informative. Most interesting things -for formulas- happen in formulas.pas beside TypeDefinitions.pas. Some formula types are called from asm functions (see do*Hybrid*() procedures and functions) most of them seem to be related to formulas with sse code.
The inline "instructions" just tells the compiler that the functionction should be inlined. Inlining is not automatic. Here the function is called from assembly so the compiler won't be able to inline it. There must be a better solution... This didn't work (https://gcc.gnu.org/onlinedocs/gcc/Local-Reg-Vars.html#Local-Reg-Vars). :help: again!  :)


Title: Re: Blobby (How to write simple m3f)
Post by: knighty on April 20, 2015, 09:18:23 PM
Some progress:
Finally only ecx have to be saved in dIFS formulas.
Using sse2 is possible but with some constraints for absolute value and negation operation. functions using intrinsics and macros are provided in the source code below. Still not perfect but one can still edit the assembly code after all. Anyway fpu is fast enought for me.
Code:
/*
Mandelbulb 3d Formula example.
Original code by marius and jesse.
Modified by knighty. Apr 2015.
Work in progress.
No warranty. This is made by a noob ;)
Compilation using mingw and msys (stricly speaking msys is not required... in principle):
-Compile with (replace "theFormulaFilename" by actual c file name):
gcc -c -m32 -O3 -mfpmath=387 -ffast-math -march=pentium4 theFormulaFilename.c
  SSE2 can be used by setting: -mfpmath=both and adding the switch: -msse2
  If you want to use sse2, better use assembler. c's sse generated code is just annoying! and not necessarily faster. :(
  Anyway, it is still possible to edit the assembler code generated by the compiler...
  If you use sse2, use the macros defined below. Otherwise the compiler will use variables stored in .rdata whith will make the formula unusable.
-Verify asm code:
objdump -D theFormulaFilename.o
  This is necessary to check that there are no library functions call and that there is
 only .text "segment" no .rdata or something like that or a call to external function.
-Extract machine code:
objcopy -Obinary -j .text theFormulaFilename.o theFormulaFilename.bin
-Convert machine code from binary to hexadecimal:
bin2hex theFormulaFilename.bin theFormulaFilename.m3f
(bin2hex is not part of msys or mingw. any other binary file editor would do the job)
-edit theFormulaFilename.m3f in a text editor to add MB3D stuff.
*/
/*Defines**************************************************/
#define USE_SSE2
/*Includes*************************************************/
#include <emmintrin.h>
#include <math.h>
/*MB3D structures definitions*************************************************/
//This structure is specific to dIFS. The use of most of it is unknown and should not be modified.
//Edit: it seems that it is almost the same as for non dIFS. 
__attribute__((packed)) struct TIteration3Dext {
double something; // -0x88 ; used in sphereheightmap.m3f
double dum0; // -0x80 ; unknown
double x; // -0x78
double y; // -0x70
double z; // -0x68
double dum1[8]; //unknown
double DE2T; // -0x20 ; Output: distance estimate to current object
double dum2[17]; //unknown
double accumulatedScale; // +0x70
double dum3; // +0x78; unknown
double OTforCol; // +0x80
double dum4[16]; //unknown
//void * sphericalMap; // +0x108; pointer to function
};
__attribute__((packed)) struct Sconsts{//in the same order as m3f file's [constant] section
double Pie;//just to use a constant
long long int abscst;//for ABS()
long long int negcst;//for NEG()
};
__attribute__((packed)) struct Svars{//in the reverse order wrt m3f file
double Zadd;
double Yadd;
double Xadd;
double Scale;
double Bevel;
double HalfLengthZ;
double HalfLengthY;
double HalfLengthX;
double Dummy;//not used. What is it good for?
};
/*Macros**************************************************/
//this macro changes the type of a variable without conversion.
#define REINTERPRET(x,type) (*((type *) &(x)))
//
#define MAX(x,y) ((x)>(y) ? (x) : (y))
#define MIN(x,y) ((x)<(y) ? (x) : (y))
//with sse2, fabs() and negating generates a logical instruction with a constant stored in .rdata
//use these instead of writing: fabs(bar); or foo=-bar; when using sse2.
#ifdef USE_SSE2
#undef fabs
#define ABS(x) (Abs(x, pconst))
#define NEG(x) (Neg(x, pconst))
#else
#define ABS(x) (fabs((x)))
#define NEG(x) (-(x))
#endif
/*Local functions declaration and/or implementation*************************************************/
//if the function is declared static it will not be exported and it won't be in the obj file if inlined
inline void __attribute__((fastcall)) TheFormula(char* siarg, char* diarg);//implemented after formula because formula code must be placed before this function's code.
#ifdef USE_SSE2
static inline double Abs(double x, struct Sconsts* pconst){
__m128d v=_mm_set1_pd (x);//not perfect
__m128d w=_mm_set1_pd (REINTERPRET(pconst->abscst,double));//not perfect either
__m128d r=_mm_and_pd (v,w);
return _mm_cvtsd_f64 (r);
}
static inline double Neg(double x, struct Sconsts* pconst){
__m128d v=_mm_set1_pd (x);
__m128d w=_mm_set1_pd (REINTERPRET(pconst->negcst,double));
__m128d r=_mm_xor_pd (v,w);
return _mm_cvtsd_f64 (r);
}
#endif
/*Formula implementation*************************************************/
// Not a standard calling convention. this is called from an asm code in MB3D which assumes that %ecx is not modified.
// We need to save %ecx register.
// esi and edi (and ebx?) are not modified (or restored?)!???
//
// the arguments of this function are in edi and esi registers.
// esi points to the context structure
// edi points to constants (negative displacements for variables)
void formula(void) {//Big overhead. The only solution I could find. Any Idea?
asm("push %ecx\n\t");//save ecx
asm("mov %esi,%ecx\n\t");
asm("mov %edi,%edx\n\t");//arguments to TheFormula
asm("call @TheFormula@8\n\t");//this is the compiler generated name for TheFormula. If cdecl the generated name is "_TheFormula"
//is it possible to get TheFormula inlined? The following doesn't work.
//void (*foo)() = (void (*)())&TheFormula;
//foo();
asm("pop %ecx");//restore ecx
}

/*The actual formula*************************************************/
inline void __attribute__((fastcall)) TheFormula(char* siarg /*ecx*/, char* diarg/*edx*/) {
//DO NOT MODIFY BEGIN
// Compute ptr to proper start of TIteration3Dext struct.
  struct TIteration3Dext* pctx = (struct TIteration3Dext*)(siarg-0x88);
  // get pointer to constants.
  struct Sconsts* pconst=(struct Sconsts*) (diarg);
  // get pointer to variables.
  struct Svars*   pvar=(struct Svars*) (((char*)pconst)-sizeof(struct Svars));
//DO NOT MODIFY END
  // Draw a box
  double x=pctx->x, y=pctx->y, z=pctx->z;
  x=ABS(x) - (pvar->HalfLengthX - pvar->Bevel);
  y=ABS(y) - (pvar->HalfLengthY - pvar->Bevel);
  z=ABS(z) - (pvar->HalfLengthZ - pvar->Bevel);
  double DE = MAX(x,MAX(y,z));
  if (DE>0.){
x = MAX(0.,x);
y = MAX(0.,y);
z = MAX(0.,z);
DE = MAX(DE, sqrt(x*x+y*y+z*z));
  }
  pctx->DE2T = DE - pvar->Bevel;
  // Now do translation and scaling
  pctx->x = pctx->x * pvar->Scale + pvar->Xadd;
  pctx->y = pctx->y * pvar->Scale + pvar->Yadd;
  pctx->z = pctx->z * pvar->Scale + pvar->Zadd;
  pctx->accumulatedScale = pctx->accumulatedScale * pvar->Scale;
}
Code:
[OPTIONS]
.Version = 6
.DEoption = 20
.SSE2
.Double X halfwidth = 1
.Double Y halfwidth = 1
.Double Z halfwidth = 1
.Double Fillet = 0
.Double Scale = 1
.Double X add = 0
.Double Y add = 0
.Double Z add = 0
[CONSTANTS]
Double = 0
INT64 = $7FFFFFFFFFFFFFFF
INT64 = $8000000000000000
[CODE]
5589E5505351525756E8080000005A5A5A595B585DC35589E583EC088B4D0C8B45082D880000
008D51B8DD4010DD4018DD4020DD55F8DD4220D9C3D9E1DC6238D8C1D9C3D9E1DC6230D8C2D9
CBD9E1DC6228D8C2D9CBDBF1D9C1DBC1DBF4DAC4D9EED9C9DBF1762ED9CBDBF1DAC1D8C8D9CA
DBF1DAC1D8C8D9CDDBF1DAC1DDD9D8C8D9CCDEC1DEC3D9CAD9FAD9CADBF2DAC2DDDAEB08DDDD
DDD8DDD8DDD8DEE9DD5868DD4218DCCAD9CADC4210DD5810D8C9DC4208DD5818DD45F8D8C9DC
41B8DD5820DC88F8000000DD98F8000000C9C3909090
[END]

Box shape dIFS with rounded edge. no OTrap coloring for now.
[/code]