# How is a non-standard literal different from calling its corresponding macro?

**URL:** <https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349>\
**Category:** Internals & Design\
**Tags:** macros, strings\
**Created:** [August 30, 2023, 1:23am UTC](https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349 "2023-08-30T01:23:51Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [August 30, 2023, 1:23am UTC](https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349/1 "2023-08-30T01:23:51Z")

</div>

```julia
julia> x = "whoops"
"whoops"

julia> @raw_str("$x") # huh...
ERROR: UndefVarError: x not defined
Stacktrace:
 [1] top-level scope
   @ REPL[48]:1

julia> "$x", raw"$x" # expected
("whoops", "\$x")

julia> @macroexpand @raw_str("$x") # interpolates from definition scope, hence error
:("$(Base.x)")

julia> @macroexpand "$x" # interpolates from call scope
:("$(x)")

julia> @macroexpand raw"$x" # literal appears to evaluate first
"\$x"

```

The example shows the discrepancy. The non-standard literal expression even evaluates despite `@macroexpand` usually preventing that. I want to know what’s going on because this seems important for custom parsing and perhaps interpolation.

It doesn’t seem like `@raw_str("$x")` is simply doing `"$x"` first then applying `raw_str`. If that were the case, I would expect `@raw_str("$x")` to do the same thing as `"$x"` because `macro raw_str(s); s; end` actually does (closest to) nothing to the text.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [August 30, 2023, 2:58am UTC](https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349/2 "2023-08-30T02:58:03Z")

</div>

I think you’re just confused about how “$x” is displayed / represented, raw\_str macro can’t be simpler:

```julia
julia> macro my_str(x)
           x
       end
@my_str (macro with 1 method)

julia> my"$x"
"\$x"

```

“$x” with interpolation is similar to “a\nb”, this is just how you write things. If you need a raw “$” in your string, you need to either write “\$x”, or, use `raw_str `. Similar to if you need a literal backslash in your string, you need to write:

```julia
julia> print("\\")
\

```

* * *

in other words, the point of `@raw_str` is just so you don’t have to type `\` all the time.

```julia
julia> raw"a\b"
"a\\b"

```

---

<div class="post-metadata">

**Author:** ![savq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/savq/32/22063_2.png) [@savq](https://discourse.julialang.org/u/savq)\
**Post date:** [August 30, 2023, 3:30am UTC](https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349/3 "2023-08-30T03:30:07Z")

</div>

I’ve also stumbled upon this problem. Indeed, non-standard string literals parse differently. You can verify this by quoting the ns-string and the macro call:

```julia-repl
julia> :(raw"good ol' string") == :(@raw_str "good ol' string")
true

julia> :(raw"$x") == :(@raw_str "$x")
false

```

EDIT:

> The non-standard literal expression even evaluates despite `@macroexpand` usually preventing that.

It’s more complicated than that. In Julia, literals cannot be quoted. This means that you can, for example, write macros that only take numeric literals:

```julia
julia> macro is_num_literal(n::Number) true end;

julia> macro is_num_literal(n) false end;

julia> @is_num_literal 1
true

julia> x = 1;

julia> @is_num_literal x
false

```

But, strings are weird. Strings with interpolations _can_ be quoted, and the result is an `Expr` with a `string` head!

```julia
julia> :"good ol' string" |> dump
String "good ol' string"

julia> :"$x" |> dump
Expr
  head: Symbol string
  args: Array{Any}((1,))
    1: Symbol x

```

All this to say that `@macroexpand` really didn’t evaluate anything, but since ns-strings cancel interpolations, then the expansion returns a literal string (not a string Expr).

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [August 30, 2023, 5:47am UTC](https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349/4 "2023-08-30T05:47:32Z")

</div>

> **Trying \`dump\` like you did,**
>
> ```julia
> julia> dump(:( raw"$x" ))
> Expr
> head: Symbol macrocall
> args: Array{Any}((3,))
> 1: Symbol @raw_str
> 2: LineNumberNode
> line: Int64 1
> file: Symbol REPL[4]
> 3: String "\$x"
> 
> julia> dump(:( @raw_str "$x" ))
> Expr
> head: Symbol macrocall
> args: Array{Any}((3,))
> 1: Symbol @raw_str
> 2: LineNumberNode
> line: Int64 1
> file: Symbol REPL[5]
> 3: Expr
> head: Symbol string
> args: Array{Any}((1,))
> 1: Symbol x
> 
> ```

it looks like `:(@raw_str "$x")` contains `:"$x"` except macro hygiene rules look for the `x` in the definition scope instead of the call scope. On the other hand, `:( raw"$x" )` is equivalent to `:(@raw_str "\$x")`. You can interpolate the raw string without manual escapes `:(@raw_str $(raw"$x"))`. You get an equivalent result without interpolation `:(@raw_str raw"$x")` because the parser acts on literals before macros expand, but there will be two `@raw_str` macro calls. Before macros expand, the parser makes `$` do interpolation for standard string literals (there’s no `$` character remaining in the expression) and replaces `'$'` with the equivalent `'\$'` for non-standard string literals. Given a name like `blah"$x"`, it’ll make up a corresponding `@blah_str` symbol, even if the macro doesn’t exist.

I can’t control parsing at all, but at least I’ll know to expect that `$` does not do interpolation in non-standard string literals, leaving whatever it does up to the macro. I’ll leave this open because I don’t know what other decisions that the parser makes before macro expansion, depending on whether a string literal is standard or non-standard.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [August 30, 2023, 7:24am UTC](https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349/5 "2023-08-30T07:24:52Z")

</div>

I think the rules are relatively straightforward:

`"something"` is a string literal

`"something_with_$interpolation"` is syntax sugar for `string("something_with_", interpolation)`

`@some_macro "something_with_$interpolation"` is syntax sugar for `@some_macro string("something_with_", interpolation)`

`prefix"something_with_$interpolation"` is syntax sugar for `@prefix_str "something_with_\$interpolation"`, so all `$`s are escaped

`@prefix_str "something_with_$interpolation"` does not get special parser treatment so like example 3 it is syntax sugar for `@prefix_str string("something_with_", interpolation)`. One can argue whether the parser should give `@prefix_str` macros in their macro form special treatment as well, but it doesn’t. I guess that makes sense as you can reassign names, so you could call the same macro differently depending on whether you give it a name with `_str` at the end or not.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [August 30, 2023, 11:27am UTC](https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349/6 "2023-08-30T11:27:43Z")

</div>

> [@jules](#):
>
> `@some_macro "something_with_$interpolation"` is syntax sugar for `@some_macro string("something_with_", interpolation)`

Not true, the `string` call enters the macro as an expression, it’s not yet evaluated to make 1 input string. ~~I don’t know if the parser does a `string` call or something equivalent to it.~~

```julia
julia> macro headof(ex) Expr(:quote, ex.head) end
@headof (macro with 1 method)

julia> @headof "something_with_$interpolation"
:string

julia> @headof string("something_with_", interpolation)
:call

```

The parser does not do a `string` call for `$`-interpolated standard literals, I type pirated the primary callee method `@eval Base function print_to_string(xs...)` to insert 1 space after every argument. While interpolated standard literals do get those extra spaces (and messes up REPL text display), the spaces are not in the expression, so it seems to happen later in `eval`.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [August 30, 2023, 12:17pm UTC](https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349/7 "2023-08-30T12:17:31Z")

</div>

Eh yeah sorry I was thinking about the `Expr(:string` not the `string` function call. But they seem to be equivalent. `Expr(:string` probably just exists so that macros know that the user has written the interpolation syntax.

```julia
julia> @generated function test(v::Val)
           if v === Val{true}
               quote
                   x = 1
                   $(Expr(:string, "a string and a ", :x))
               end
           elseif v === Val{false}
               quote
                   x = 1
                   "a string and a $x"
               end
           end
       end
test (generic function with 1 method)

julia> @code_warntype test(Val(true))
MethodInstance for test(::Val{true})
  from test(v::Val) @ Main REPL[17]:1
Arguments
  #self#::Core.Const(test)
  v::Core.Const(Val{true}())
Locals
  x::Int64
Body::String
1 ─ (x = 1)
│ %2 = Base.string("a string and a ", x::Core.Const(1))::String
└── return %2

julia> @code_warntype test(Val(false))
MethodInstance for test(::Val{false})
  from test(v::Val) @ Main REPL[17]:1
Arguments
  #self#::Core.Const(test)
  v::Core.Const(Val{false}())
Locals
  x::Int64
Body::String
1 ─ (x = 1)
│ %2 = Base.string("a string and a ", x::Core.Const(1))::String
└── return %2

```

---

<div class="post-metadata">

**Author:** ![savq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/savq/32/22063_2.png) [@savq](https://discourse.julialang.org/u/savq)\
**Post date:** [August 31, 2023, 5:18pm UTC](https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349/8 "2023-08-31T17:18:27Z")

</div>

> [@Benny](#):
>
> I don’t know what other decisions that the parser makes before macro expansion

From my understanding, which is mostly based on [this JuliaSyntax file](https://github.com/JuliaLang/JuliaSyntax.jl/blob/main/src/literal_parsing.jl), the only things that can be escaped in non-standard strings are the string delimiters themselves: `\"` (if string), `\` ` (if cmd). Everything else is treated as a regular character.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [August 31, 2023, 11:35pm UTC](https://discourse.julialang.org/t/how-is-a-non-standard-literal-different-from-calling-its-corresponding-macro/103349/9 "2023-08-31T23:35:30Z")

</div>

I’m still not fully understanding how the raw string flags are determined in the new parser, but it’s a lot more readable, showing the divide between standard string and raw string (really everything else) and how parsing works there. Also found this old stackoverflow post “[How does a non-standard string literal avoid a syntax error generated by a standard string literal?](https://stackoverflow.com/questions/58107022/how-does-a-non-standard-string-literal-avoid-a-syntax-error-generated-by-a-stand)” about the current/old parser showing the same divide. I’ll close this thread, thanks for chipping in, everybody.