Problem with where clause in macro?

I dislike the somevariable = [somevariable; somevalue] concatenation syntax, due to the repetition of somevariable. So I tried to write an append macro, but I’m having difficulty.

Initially, I tried writing an infix macro called <<+, but the REPL complained that < was not valid in that location. I assume it’s taking the three symbols separately. Can we not combine symbols into macro or function names? I was able to make a macro called ++, so presumably some combinations are valid…

So I changed the name, but the macro still won’t compile, and I don’t understand why:

macro appendto(a::Vector{T}, b::T) where T
  quote
    $(esc(a)) = vcat($(esc(a)), $(esc(b)))
  end
end

It works if I put where T inside the parens, but of course that would not restrict the instances of T to the same type.

(As a side note, I tried to write a macro called ++=, but that would not compile either. Is it not possible to use = in a macro name?)

A macro only sees syntax - it’s an AST transform that happens before types exist in the pipeline. How would you expect it to be able to match types of variables, when it runs before they exist?

That’s probably due to it not being a valid identifier.

A macro only sees syntax

Then I don’t see the purpose of being able to specify the types of macro arguments at all. Or do you mean it allows such things as “restrict to lvalue” or “restrict to literal number”?

That’s probably due to it not being a valid identifier.

So…I can make a ++ operator, but I can’t make a ++= operator? I have to instead write thing = thing ++ otherthing? Seems an odd limitation.

Also, why isn’t <<+ a valid name for a macro? They are all valid identifiers.

I think the function you are looking for already exist:

x = [1,2,3]
push!(x, 4)
append!(x, [5,6])

Both of these functions add the data to the existing Vector.


These functions are (probably) also faster than using vcat since they do not create new Vectors.

2 Likes

Precisely - you can restrict the macro to accept only a subset of things the parser spits out directly. OTOH that’s literal strings, literal numbers, Symbols and Expr for more complex expressions, like calls or array literals (which get parsed as Expr with :vect as their head).

No, that’s not what I’m saying. There’s a certain list of infix operators, such as ++. These can be used like a = a ++ c or a ++= c. The important thing is that the latter is equivalent to the former (and in fact, a transform of the latter to the former happens during lowering), and both are exactly equal to a = ++(a, c). ++= on its own is not a thing, as such it can’t be used as the name of a macro (there is a workaround in the form of var"++=", which I’m not 100% sure on works, but would have has to be invoked like @var"++=" either way).

What are you referring to with “they all are valid identifiers”? Being a valid identifier is not necessarily a property that persists through concatenation.

julia> Base.isidentifier("<<+")
false

Not this example. The latter isn’t valid syntax at all:

julia> :(a ++= 1)
ERROR: syntax: unexpected "="

Right, I forgot that those are special cased again… They are defined here:

or here for the upcoming rewrite of the parser in julia:

Julia has some restrictions on it’s identifiers you might find surprising coming from Lisp. Basically, we follow a strict rule that our code should be parseable without a running julia program present. That means that we don’t have reader macros, and you can’t customize on the fly how an identifier is interpreted by the parser. Since we chose infix syntax rather than S-expressions, this means that a lot of identifiers are special cased, and combinations of such identifiers would need to also be special cased in order to work.

If you really wanted though, you could use the var string macro to accomplish this.

julia> macro var"++="(a, b)
           esc(:($a = vcat($a, $b)))
       end;

julia> let a = [1, 2, 3, 4]
           @var"++=" a [5, 6]
       end
6-element Vector{Int64}:
 1
 2
 3
 4
 5
 6

Though this is obviously not as elegant as you had wanted.

1 Like

I think the function you are looking for already exist

Oh, interesting! I didn’t know about those. Maybe I was looking in the wrong place in the documentation. Thank you.

(I do find the Base and Standard Library quite difficult to browse; owing to the fact that it’s all in a big list. Would be easier with a more terse, grouped layout; with the option of viewing more detailed information if desired.)

I forgot that those are special cased again

Wouldn’t it make more sense for a op= b to always equate to a = a op b?

Well…strikes me you can’t put two operators together lexically under any circumstances (except for . broadcast). For example +* would never be a valid usage of addition and multiplication. So I don’t really understand the difficulty with allowing any arbitrary sequence of “operator characters” to be used in the definition of a new operator.

For example, when I tried to define an infix <<+ function, I don’t see how that could be ambiguous. Even in the presence of < and + infix operators, <<+ together without intervening spaces couldn’t possibly indicate an intention to combine existing operators.

In short, I fail to see the problem.

Yes. As I said “combinations of such identifiers would need to also be special cased in order to work”. We could add something in the parser so that +*+-+/^ is a valid operator, but it hasn’t been done. Probably because it also needs a precedence rule. I.e. should +* bind as tightly as + or as tightly as *? Or higher than both? Lower?

One thing we do allow is arbitrary unicode subscripts or superscripts following operators:

julia> +ₐ(x, y) = x*2 + y
+ₐ (generic function with 1 method)

julia> 1 +ₐ 2
4

and these take the precedence of the parent operator. But since Unicode subscripts and superscripts are woefully incomplete (ref Unicode subscripts and superscripts - Wikipedia and proposed fix GitHub - stevengj/subsuper-proposal: Draft proposal for additional sub/superscript characters in Unicode) you can’t have whatever characters you like as subscripts or superscripts.

I think another reason we don’t treat +* and co as valid operators is that it’s propably much more likely that those are the result of a typo rather than someone wanting a very whacky operator.

1 Like

Well, + is also a unary operator.

julia> 2<<+3
16
1 Like

Yeah, but personally I’d be okay with “you need a space between << and + if you’re gonna use the completely superfluous unary +:wink:

Hell, even 2<<-3 should have a space. (And of course would be a confusing way to use shift operators.)