Prexonite Script

Prexonite Macro Support

Over a year ago I considered the implementation of meta programming facilities in Prexonite Script. In that post, I concluded that advanced meta-programming features like  quotation and splicing were not realistic for a compiler architecture as convoluted as Prexonite.

I, however, think that the implementation of macro functions is feasible and useful. Macros won’t revolutionise how the language is used, but will complement the existing dynamic features to enable concise solutions, should the programmer decide to invest enough time into the macro system.

And that’s just what I did. Prexonite now supports a new keyword `macro` that can be used to define macro functions. Macro functions are invoked at compile time (more precisely just before code generation) with the AST nodes of their arguments passed as parameters. They’re expected to return a single AST node that is then inserted into the final AST in place of the macro function call.

The mechanism is already in use: The implementation of the `struct` mechanism has been ported to the macro system.

Detour: struct

Prexonite Script does not provide any mechanism for users to define their own data types (classes, structures, records). Instead there is a special type of object that you can extend with new fields and methods: The `Structure`. However, using this object to create custom structures/objects is quite verbose and cumbersome. Fortunately there is a PSR file `psr\struct.pxs` that contains the handy function of the same name. `struct` creates an empty structure and adds all nested functions of the calling function to it. This comes quite close to JavaScript-prototype-definition level of verbosity.

build does require(@"psrstruct.pxs");
function create_foo(x){
  function report() {
    println(x);
  }
  function increment(this) {
    x++;
    this.report();
    return x;
  }
  return struct;
}

function main(){
  var foo1 = new foo(5); 
  //fancy syntax for `create_foo(5)`
  var foo2 = create_foo(foo1.increment()); 
  //prints 6
  foo2.report; 
  //prints 6 too
}

The function `create_foo` (you can call it a constructor if you like) creates a new `foo` object every time it is invoked. The resulting object has two methods `report` and `increment`. The variable `x` is shared by all methods via the nested function/closure mechanism. It is not formally part of the foo object¹.

Macro functions

The implementation of the struct function is actually quite simple. It makes heavy use of helper functions defined in `psr\macro.pxs`. All of the `ast<something>` functions plus `tempalloc` and `tempfree` are defined in that file.

//part of `macro struct()` 
//  in `psr\struct.pxs`
//creates a new block expression node and allocates a local
//  variable for the structure object.
var block = ast("BlockExpression");

var structV = tempalloc;

//create the structure object and assign it to
//  the local variable
var assignStructure = astlvar(SI.set, structV);
assignStructure.Arguments.Add(ast\new("Structure"));
block.Add(assignStructure);
//don't forget to add the statement to the block
//assign the "ctorId" (name of the constructor function) 
//  to the structure
var assignCtorId = astmember(ast\lvar(SI.get, structV), 
                               SI.get,@"");
assignCtorId.Arguments.Add(ast\const(CTORID));
assignCtorId.Arguments.Add(ast\const(parentId));
block.Add(assignCtorId);
//Add all of the methods to the struct
foreach(var method in methods){
    var addMethod = ast\member(ast\lvar(SI.get, structV), 
                                 SI.get, "\\");
    addMethod.Arguments.Add(ast\const(method.Key));
    addMethod.Arguments.Add(ast\lvar(SI.get, method.Value));
    block.Add(addMethod);
}
//return the constructed structure 
objectblock.Expression = astlvar(SI.get, structV);
//Don't forget to free our temporary variable and
//  actually return the code 
blocktempfree(structV);
return block;

The macro code is not that difficult to understand. So writing macros in Prexonite Script is easy right? Not exactly, no. You’re directly operating within the compilers AST, an API that was never designed to be consumed by user code. That `ast(”BlockExpression”)` creates an object of type `Prexonite.Compiler.Ast.AstBlockExpression` and the `Arguments` member of the `assignCtorId` node is the same member that the compiler uses. Why is that a bad thing? Well for one it creates a very strong dependency on an implementation detail of the Prexonite Script compiler, and it relies heavily on good IDEs, that means the API is ugly (often many parameters) and irregular (I myself have to constantly lookup the ever changing parameter ordering and exact member names). In other words: You won’t get far without a copy of the compiler source code next to your Prexonite Script code editor.

Ok, so its a bit tricky to use, anything else I should know?

Yes.

• macro functions can only be used after they have been defined. It is not possible to forward-declare a macro function.

• macro functions cannot be nested. They have to appear on the global level.

• macros used as part of macro functions behave like in any other function. If you want to call another macro (for code reuse), you’ll have to use the `call\macro` function (defined in `psr\macro.pxs`). It behaves just like the `call` command.

• A macro function has access to a number of implicitly defined variables:

  • loader, a reference to the Prexonite.Compiler.Loader instance
  • callType, the call type of this invocation (get vs. set), see Prexonite.Types.PCall
  • justEffect, a boolean indicating whether the caller just wants the side effects. (return values can safely be dropped or not computed at all)
  • locals, access to the collection of local variable definitions (not PVariable objects! We’re at compile-time)
  • macroInvocation, a reference to the AST node for this macro invocation
  • target, a reference to the current compiler target, Prexonite.Compiler.CompilerTarget; provides access to the calling function, amongst other things

• macros are applied from the outside to the inside, that is, the outermost macro is applied first. This means that if your macro appears as an argument to another macro, it might actually disappear from the AST and never be applied.

• macro functions are not automatically stripped from the application after compilation (simply because Prexonite has no concept of "after" compilation). You can use the function `unload_compiler` defined in `psr\ast.pxs` to remove all macros and functions/variables marked with the `compiler`tag (including any nested functions)

• Global variables are only initialized at runtime, not at compile-time, and they don’t exist until they’re defined (and not just declared)

• You don’t have access to local symbols (declarations) in macro functions.

  • The symbol table provided by `target.Symbols` reflects the state at the end of the end of the calling function. Don’t ask.

  • Any symbols you add to the symbol table won’t be available to the calling function. But if you add them to `loader.Symbols` they will be available to functions defined after the calling function. This might be interesting for initialization code.