Is there a `with` operator or usage pattern?

New to Julia
21

I think your problem comes from the fact this little claim “Julia already knows what fields are in my tuple” is not well defined. Inside a macro, Julia, in fact, does not know what fields are in your tuple. The macro are executed before the function is compiled, and it has no information about the values (and therefore the types) of the symbols/expressions passed to them, they only know about the symbols themselves.

This is, a macro takes as arguments the text/“source code” you are passing to it, without any values inside the bindings or anything. It has no way to know the binding var you passed has a value of type T that has fields a, b, and c which it could expand to a, b, c = var.a, var.b, var.c. This is the reason @unpack works the way it works, it needs the user to pass to it the names of the variables, so it knows what to extract from the symbol var that may have any value (or even not be defined) in runtime.

A macro can expand to a call of fieldnames (which return a list of the struct fields) to construct a Dict (mapping the name of a field to the field in the correct struct) in runtime and the macro can also expand the reference to bindings in your expression to queries in such Dict (which would also happen in runtime). The result would be a code many times slower than the one done by hand.

4 Likes
22

First, I fully agree with everyone here that this kind of thing tends to be a footgun and using UnPack leads to far more readable code.

That said, worth pointing out that while you can’t do this with macros since as said above they only operate on expessions, you can do it quite cleanly with generated functions, which also have access to the type of your object, as well as lowered code to get the scoping right. Here’s a MWE:

using MacroTools: postwalk
@generated function with(func, obj::T) where {T}
    method = Base._methods_by_ftype(Tuple{func}, -1, typemax(UInt64))[1][3]
    code = Base.uncompressed_ast(method).code
    ir_to_expr(i) = postwalk(x -> x isa Core.SSAValue ? ir_to_expr(x.id) : x, code[i])
    postwalk(ir_to_expr(length(code))) do x
        x isa GlobalRef && hasfield(T,x.name) ? :(obj.$(x.name)) : x
    end
end

subtotal = (price=12.99, shipping=3.99)
handling = 0.99

with(subtotal) do
    price + shipping + handling
end # 17.97

And if you check out the generated code, its optimal / inferred:

julia> @code_warntype with(subtotal) do
           price + shipping
       end

Body::Float64
1 ─ %1 = Base.getproperty(obj, :price)::Float64
│   %2 = Base.getproperty(obj, :shipping)::Float64
│   %3 = (%1 + %2)::Float64
└──      return %3

This works in a local scope too as long as you don’t use other local variables, although thats just a limitation of the way I wrote it, you could fix that by doing a better job of modifying the IR code than the simple thing I did above. Someone with more expertise can probably do something even cleaner, maybe with Cassette or IRTools or something, but it was fun to work out this proof of concept.

8 Likes
23

@MA_Laforge If you update your example to be a complete MWE that defines the structure and uses let, I’d be delighted to switch the “solution” for new Julia developers who are looking for the simple, canonical replacement pattern.

@marius311 – that’s amazing. I’d love a performant implementation that works at multiple levels where prefixes aren’t needed. Given how many attempts at @with there are, it’d be great if one were type stable.

1 Like
24

One easier possibility to implement efficiently would be to write @with macro where you mark the expressions that are fields, e.g.:

@with subtotal begin
    _.price + _.shipping + _.handling
end
3 Likes
25

@stevengj So, I’m trying to separate two concerns: (a) where the variables are coming from, with (b) how they are used. Sometimes this is helpful (and sometimes it’s not). In @marius311’s example, the handling variable comes from the global context and is not treated any differently as price and shipping.

subtotal = (price=12.99, shipping=3.99)
handling = 0.99

with(subtotal) do
    price + shipping + handling
end # 17.97
26

@stevengj that’s exactly what I did above: Is there a `with` operator or usage pattern? - #13 by rdeits

2 Likes
27

Not certain I fully understand what you are looking for in your book, esp. given the complexity of your example here:

A simple solution

but I’ll try my best to provide an example of what I think you are seeking in a “simple, canonical replacement pattern”:

mutable struct Customer
	name::String
	city::String
	country::String
	additionalInfo::Array{String}
end
Customer() = Customer("", "", "", [])

function generateDummyCustomer()
	newCustomer = Customer()
	let c = newCustomer
		c.name = "Alfred Customer"
		c.city = "Boston"
		c.country = "United States"
	end
	let i = newCustomer.additionalInfo
		push!(i, "Good customer")
		push!(i, "Provides useful feedback")
	end
	return newCustomer
end

Other suggestions for temporary variable

Since I don’t often like using “i” for anything but iterators (or Complex(0,1)), other alternatives are:

  • let info = newCustomer.additionalInfo (Use a succinct/clear alias)
  • let x = newCustomer.additionalInfo (x is usually a good “go-to”)
  • let o = newCustomer.additionalInfo (o is often an acceptable variable for “objects”)

In theory, I also like @stevengj’s idea of using “_” - but I found Julia complains when an “all-underscore identifier” is used on the RHS:

julia> _ = 3; a=4
4

julia> b = _ + a
ERROR: syntax: all-underscore identifier used as rvalue
Stacktrace:
 [1] top-level scope at REPL[11]:1
2 Likes
28

Note that I changed the example a bit because I’m guessing you don’t want to put in a copy Microsoft code in your book.

I also don’t like that (I can only guess) the example they used probably needs to be manipulating global variables somehow.

  • In VB Private Sub AddCustomer() can’t return a value (last I checked).
  • That means the Customer() constructor would have to store itself somewhere accessible through a form of global variable to be useful.

Then again: I don’t think I’ve programmed in VB in over 10 years (Mostly Julia’s fault).

29

I can fix this by throwing in a Base.invokelatest invocation into the macro.

julia> function genfunc(nt, body)
           local props = propertynames(nt)
           f = :( (; $(props...) )-> $body )
           g = Meta.eval(f)
       end
genfunc (generic function with 1 method)

julia> macro with(nt,body)
           local b = QuoteNode(body)
           quote
               let f = genfunc($(esc(nt)), $b)
                   Base.invokelatest(f; $(esc(nt))...)
               end
           end
       end
@with (macro with 1 method)

julia> function f(subtotal)
           @with subtotal begin
                price + shipping
           end
       end
f (generic function with 1 method)

julia> subtotal = (price=12.99, shipping=3.99)
(price = 12.99, shipping = 3.99)

julia> f(subtotal)
16.98
30

One last thought:

julia> function with(f; kwargs...)
           d = Dict(kwargs...)
           f(d)
       end
with (generic function with 1 method)

julia> with(;subtotal...) do d
           d[:price] + d[:shipping]
       end
16.98
31

The following is a restatement of with from Visual Basic, and an comparable pattern in Julia that doesn’t rely upon macros.

Thank you @MA_Laforge

2 Likes
32

Quick update! I’ve changed PropertyUtils.@with to no longer use eval.

Voila, no more Any with your example:

julia> @code_warntype f((a = 1, b = 2))
...
5 ┄ %13 = @_5::Int64
│   %14 = (%7 + %13)::Int64
└──       return %14