Metaprogramming in Julia

Anand Bisen bio photo By Anand Bisen Comment

Recently I came across a clever application of Metaprogramming in Julia by John Myles White and Randy Zwitch. I am not much of a coder and have only dabbled a bit in Python, it’s possible that this is a standard approach. Nevertheless I found it inspiring so writing about it :)

Wouldn’t it be convenient to just create a specification list that provides the name, type and default value for the fields and call some function that would automatically create a corresponding composite type and it’s constructor method(s). And also handle default values and more housekeeping tasks automagically.

Sidebar: If you have not tried yet, I would recommend you to check out two cool visuzlization libraries ECharts.jl and Vega.jl.

The procedure explained below would allow one to create specification (spec) in form of a list of tuple and call a function makespec to parse. This approach comes in handy especially when writing code that requires maintaining many different “composite types”. Also it makes the code clean and easy to extend (check out ECharts.jl).

spec = [
        (:name,   AbstractString, nothing),
        (:title,  AbstractString, nothing),
        (:height, Number,         400),
        (:width,  Number,         600),
        (:x,      AbstractArray,  nothing),
        (:y,      AbstractArray,  nothing),

function makespec(:Scatter, spec)

And generate/execute the following code (or some version of it depending upon your needs).

type Scatter
  name::Union{AbstractString, Void}
  title::Union{AbstractString, Void}
  x::Union{AbstractArray, Void}
  y::Union{AbstractArray, Void}

function Scatter()
  Scatter(nothing, nothing, 400, 600, nothing, nothing)

This is possible by exploting powerful Metaprogrammig capabilities in Julia. The code block below is responsible for dynamically creating the type block and function above.

# Create the composite type
function maketype(_name::Symbol, spec)
  n = length(spec)
  lines = Array{Expr}(n)
  for idx in 1:n
    entry = spec[idx]
    # Create Union{} of type specified in spec file and Void
    # to be able to handle missing values
    lines[idx] = Expr(:(::), entry[1], Union{entry[2], Void})

  return Expr(:type,
              true, _name,
              Expr(:block, lines...)

# Create the constructor for the type
function makefunc(_name::Symbol, spec)
  return Expr(:function,
              Expr(:call, _name),
                   Expr(:call, _name, map(entry -> entry[3], spec)...))

# Wrapper function for calling the two functions above
function makespec(_name::Symbol, spec)
  eval(maketype(_name, spec))
  eval(makefunc(_name, spec))

Function maketype( ) creates the composite type taking spec as the input and makefunc( ) creates the constructor using the defaults. I hope the readability of the code is easy to understand what each piece is doing. But writing complex code using metaprogramming is still not my cup of tea due to the prefix notation tha metaprogramming expects.

I will describe the approach I took to generate the code for One simple modification that I wanted perform to the code block above (makefunc). The objective was to add the ability to process optional keyword arguments such that:

  • Scatter(): Would use the default values from the specification
  • Scatter(;width=100): Would use the specified value of width and all other values would default to spec

The challenge here is that writing code in prefix notation is error prone and quite confusing for me. I used a workaround to get to the desired result where I wrote the end result and worked my way backwards. Below is the function that I would like the new makefunc to generate.

function Scatter(;args...)
  obj=Scatter(nothing, nothing, 400, 600, nothing, nothing)
  for entry in args
    if isdefined(obj, entry[1])           # If the argument is defined 
      setfield!(obj, entry[1], entry[2])  # in composite type (spec) 
    end                                   # Use the values provided as the arguments
  return obj

Then in Julia console I fed the code through parse() followed by Meta.show_sexpr().

julia> instr = """function Scatter(;args...)
  obj=Layout(nothing, nothing, 400, nothing)
  for entry in args
    if isdefined(obj, entry[1])
      setfield!(obj, entry[1], entry[2])
  return obj
julia> parsed = parse(instr)
julia> sexpr = Meta.show_sexpr(parsed)

Function Meta.show_sexpr(parsed) would produce the following code in S-expression form. Converting that to Expr() form is as simple as converting each ( ) to Expr( ).

(:function, (:call, :Scatter, (:parameters, (:..., :args))), (:block,
    :( # none, line 2:),
    (:(=), :obj, (:call, :Layout, :nothing, :nothing, 400, :nothing)),
    :( # none, line 3:),
    (:for, (:(=), :entry, :args), (:block,
        :( # none, line 4:),
        (:if, (:call, :isdefined, :obj, (:ref, :entry, 1)), (:block,
            :( # none, line 5:),
            (:call, :setfield!, :obj, (:ref, :entry, 1), (:ref, :entry, 2))
    :( # none, line 8:),
    (:return, :obj)

This gave me some code to work with where I could make simple modifications to get the function to do what we desired.

function makefunc(_name::Symbol, spec)
  return Expr(:function, 
    Expr(:call, _name, 
        Expr(:..., :args))), 
      Expr(:(=), :obj, 
        Expr(:call, _name, map(entry -> entry[3], spec)...)),
        Expr(:(=), :entry, :args), 
            Expr(:call, :isdefined, :obj, 
              Expr(:ref, :entry, 1)), 
              Expr(:call, :setfield!, :obj, 
                Expr(:ref, :entry, 1), 
                Expr(:ref, :entry, 2)) 
    Expr(:return, :obj)

Executing makefunc(:Layout, spec) would produce the following code that can be used to validate what code would be executed when the function is called using eval(makefunc(:Layout, spec))

julia> makefunc(:Layout, spec)
:(function Layout(; args...)
        obj = Layout(nothing,nothing,400,600,nothing,nothing)
        for entry = args
            if isdefined(obj,entry[1])
        return obj

I hope this post was useful to somebody :) Do drop me a line if the approach I took could be improved upon.

comments powered by Disqus