# How Would You Compare Metaprogramming in Julia and Tcl?

**URL:** <https://discourse.julialang.org/t/how-would-you-compare-metaprogramming-in-julia-and-tcl/103342>\
**Category:** General Usage\
**Tags:** metaprogramming, tcl\
**Created:** [August 29, 2023, 6:53pm UTC](https://discourse.julialang.org/t/how-would-you-compare-metaprogramming-in-julia-and-tcl/103342 "2023-08-29T18:53:38Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![TheCedarPrince](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thecedarprince/32/17323_2.png) [@TheCedarPrince](https://discourse.julialang.org/u/TheCedarPrince)\
**Post date:** [August 29, 2023, 6:53pm UTC](https://discourse.julialang.org/t/how-would-you-compare-metaprogramming-in-julia-and-tcl/103342/1 "2023-08-29T18:53:38Z")

</div>

Hi folks! 👋

I had a very interesting discussion today with an actual Tcl programmer (never met one before!). He and I were discussing a domain specific language (DSL) he had created in Tcl that had syntax that looked something like this:

```plaintext
I wish this was $blue

```

Which would produce a little png that prints the exact same Tcl code and changes the color of the text to blue. He had found that writing DSLs in Tcl was very easy given that everything was treated as a string. Additionally, with his DSL, he was trying to keep the semantics and syntax very close to the English language which he found Tcl well-suited to do.

When he found out that I was a Julia programmer, he was very curious – he said that if Julia was better at creating DSLs he’d probably give Julia a serious look. In my mind, I see replicating this sort of syntax as somewhat a challenge but could be done with Julia’s metaprogramming support and tools within the ecosystem. Now, as I am not a Tcl programmer, I was wondering if folks could help me with thinking through the following questions I had:

1. How is metaprogramming within Julia and Tcl similar?
2. How is metaprogramming within Julia and Tcl different?
3. Would you say that metaprogramming within Julia is easier than Tcl?
4. How does metaprogramming within Julia compare to Tcl?

I tried googling around to answer these questions but the intersection of Tcl programmers and Julians is very small. I found a few perspectives like this: [Julia Programming Language](https://wiki.tcl-lang.org/page/Julia+Programming+Language) where Tcl-ers seemed positive on Julia but other than that, it’s very sparse.

Could anyone help me with understanding this space better?

Thanks,

~ tcp 🌳

P.S. I found some Tcl-ers in the Julia Discourse so I am CC’ing them in case they want to comment (sorry if this is rude to ping you!): @lmiq @jessymilare @goncharovdk @lkadgrjh

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [August 29, 2023, 8:20pm UTC](https://discourse.julialang.org/t/how-would-you-compare-metaprogramming-in-julia-and-tcl/103342/2 "2023-08-29T20:20:43Z")

</div>

I know nothing of TLC and I am not even a programmer, but if it is true that in TLC “everything was treated as a string”, then it is a very different approach to Julia metaprogramming where “everything is an expression”…

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [August 29, 2023, 8:33pm UTC](https://discourse.julialang.org/t/how-would-you-compare-metaprogramming-in-julia-and-tcl/103342/3 "2023-08-29T20:33:12Z")

</div>

> [@TheCedarPrince](#):
>
> everything was treated as a string.

This reminds me of non-standard string literals, which is really just an abbreviated macro taking in a standard string literal. Metaprogramming usually goes from Julia expression to expression, which is flexible but still constrained by the AST; for example, the expression `:(1 + * 1)` does not have a valid AST structure, so an attempt throws an error. You can get around that if you start with strings and take the responsibility of parsing and evaluation. This is standard practice for embedding other languages’ source code in Julia files, though in the case of interpreted languages like Python and R, not much happens to the string, it’s just evaluated in a running interpreter.

Point is, it can get as flexible as you like, but whether it is _as easy as TCL_ I have no idea, I tried looking up some demonstrations of DSLs in TCL but I couldn’t find a straight answer on how flexible the metaprogramming is or whether it’s more like text-substitution or syntactic macros (roughly, working with strings vs structured expressions).

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [August 29, 2023, 8:38pm UTC](https://discourse.julialang.org/t/how-would-you-compare-metaprogramming-in-julia-and-tcl/103342/4 "2023-08-29T20:38:53Z")

</div>

As of 1.9 Julia’s non-standard string literals aren’t as flexible as they could be, because of their current behavior around nested quoting and escaping. That is, `"` is the same character for both opening and closing a string, so nesting is awkward. The solution proposed in this issue is a paired delimiter syntax like

```julia
htl⟪
  <table><caption><h3>Selected Books</h3></caption>
  <thead><tr><th>Book<th>Authors<tbody>$(htl⟪
    <tr><td>$(book.name)<td>$(join(book.authors, " & "))
    ⟫ for b in books)</tbody></table>⟫

```

> <https://github.com/JuliaLang/julia/issues/38948>
>
> \## Executive Summary
> 
> This is a request to add a string literal syntax using p…aired Unicode delimiters, perhaps ⟪ and ⟫, for use in non-standard string literal macros. This is proposed as an alternative \*complementary\* to, but not as a replacement for single or triple double-quoted raw strings.
> 
> \## Description of Requested Syntax
>  
> \- Paired delimiters \`'⟪': U+27EA (Ps: Punctuation, open)\` and \`'⟫' (U+27EB, Pe: Punctuation, close)\` are employed.
> \- Following a string macro name, such as \`htl\` for \`@htl\_str\`, the open delimiter, \`⟪\`, begins a string using this syntax.
> \- The parser knows the extent of the string when the \*corresponding\* closing delimiter, \`⟫\`, is encountered.
> \- Nested pairs of these delimiters are seen as content. This could be done by tracking depth, the open delimiter increases the depth, while the close delimiter decreases the depth -- when the depth reaches zero, scanning is done.
> \- The entire extent of the scanned buffer, less the very first opening and the very last closing delimiters become the string value that is passed along to the string macro.
> \- There is no further complications with regard to scanning or processing of the string done by Julia. In particular, from Julia's perspective, there is no mechanism to escape content, interpolate content, or enter arbitrary Unicode code points.
> \- The interactive Julia environment could add \`\\\>\>\` and \`\\\>\>\` as a way to enter these paired delimiters.
> 
> Critically, this non-standard string literal syntax provides no mechanism to escape either of the delimiters, excepting that nested pairings are permitted within content. In particular, unbalanced use of the given delimiters are simply not valid syntax. Julia provides no mechanism to enter unbalanced delimiters within this syntax.
> 
> \## Motivation
> 
> Let's define the term \_notation\_ to mean what is currently in the documentation as "non-standard string literal". The word notation is used by SGML and other standards for this concept.
> 
> For those doing data munging to interoperate with other systems, there is an opportunity for the Julia language to better utilize notations, enhancing developer experience and improving code readability. While developing HypertextLiteral (providing Julia-style string interpolation to HTML construction), I ran into 3 challenges with existing string "non-standard string literals" (notations).
> 
> 1) They are not succinct. Since a great many subordinate syntaxes include the double quote character, use of the triple double-quoted form is the norm. The double quote character is already loud, tripling it on both ends... becomes a distraction. Note that this deficiency applies also to the use of \`@macros()\`.
> 
> 2) They can be surprising. For cases where someone tries to use the single double-quoted form, novice users can be caught off guard with the raw\_str escaping semantics and how it interacts with the backslash. As noted on the discourse forums, this escaping mechanism is not a "homomorphism over string concatenation", e.g. raw(a) \* raw(b) != raw(a\*b).
> 
> 3) They can't be used recursively. If one would like to embed one notation inside another, a round of character escaping is required. This is unlike, for example, \`@macros()\` which nest perfectly well.
> 
> A promising option emerged on in the \[discussion forums\](https://discourse.julialang.org/t/addressing-raw-string-syntax-and-semantics-for-julia-2-0/51343): the use of paired Unicode delimiters together with a matching parsing algorithm in place of traditional character escaping. You could think of this approach as bringing to string construction what we already know about function calling and data structures -- that they are seldom flat structures.
> 
> Specifically, we could employ \`'⟪': U+27EA (Ps: Punctuation, open)\` and \`'⟫' (U+27EB, Pe: Punctuation, close)\` as paired delimiters. This particular glyph combines a doubling (reminiscent of double quotes) with that of parenthesis (implying nestability). It's not perfect, but it is visually distinct in most fonts and in mono-space fonts appears to take the space of one regular character.
> 
> When Julia encounters a name token, say \`htl\`, followed by \`⟪\`, it would enter "notation" parsing state. Here it would keep track of the nesting depth, increasing depth when additional \`⟪\` are encountered, and decreasing depth when \`⟫\` is encountered. When the depth reaches zero, the entire span (less outer most tokens) of the string is sent unprocessed to \`@htl\_str\`, and Julia parsing resumes. The REPL could add \`\\\<\<\` and \`\\\>\>\` shorthand to permit these two characters to be easily entered.
> 
> This addresses the three deficiencies noted above. This paired delimiter is much more succinct and visually attractive as compared to tripled double-quotes. The rule is unsurprising since there is no escaping, only the counting of depth, as one would find with parenthesized expressions. The rule naturally supports nesting, any construction using this method could be directly embedded as a subordinate notation. Moreover, if Unicode is used, these delimiters are unlikely to collide with those used in traditional systems, and if they do, so long as those systems use only paired form, there is no difficulty.
> 
> \## What about content having a non-paired delimiter?
> 
> This is a two part answer. Primarily, how to avoid the chosen delimiter pair becomes the notation's concern, not Julia's. For example, HTML has ampersand escaping, so the opening delimiter could be written as \`&#10218;\`. URLs use percent-encoding. Traditional double-quoted syntax (e.g. \`"\\u27EA"\` for the opening delimiter) could be used by a Python notation. For example, to encode a non-paired opening delimiter, a use of this feature might look like...
> 
> \`htm⟪\<html\>\<body\>We start these string literals with \<code\>&#10218;\</code\>\</body\>\</html\>⟫\`
> 
> Asking a notation to provide its own delimiter escaping is not without precedent. In web pages, embedded Javascript begins with \`\<script\>\` and ends when the HTML parser encounters \`\</script\>\` -- with no escape mechanism. Javascript developers who need to represent this sequence within their logic use regular double quoted strings, with the delimiter encoded as as \`"\<\\/script\>"\`.
> 
> As a fallback, for notations such as \`@raw\_str\` which lack such features, if the user must include a non-paired delimiter, they could use the existing raw string syntax which would not go away. Alternatively, they could be creative and build their string in chunks, using this syntax for most of the content and concatenating with regular double quoted strings for the non-paired delimiter. This proposed syntax aims to be \*complementary\* to existing approaches and represents different set of sensibilities. 
> 
> \## Increased Usability 
> 
> With this feature, a regular expression to detect quoted strings might be written as \`r⟪(\["'\])(?:\\\\?+.)\*?\\1⟫\` with no need to triple double-quote or worry about slashes. Moreover, other notations could embed regular expression notation without having to worry about a round of additional escaping. 
> 
> I believe these rules would permit developers integrating with foreign data producers and consumers to create their own succinct, unsurprising and nested function-like transformations that mix native languages within a Julian data processing context. Here is an example.
> 
> \`\`\`
> render(books) = htl⟪
> \<table\>\<caption\>\<h3\>Selected Books\</h3\>\</caption\>
> \<thead\>\<tr\>\<th\>Book\<th\>Authors\<tbody\>$(htl⟪
> \<tr\>\<td\>$(book.name)\<td\>$(join(book.authors, " & "))
> ⟫ for b in books)\</tbody\>\</table\>⟫
> \`\`\`
> 
> In HypertextLiteral, the functionality above is currently written as...
> 
> \`\`\`
> render(books) = @htl("""
> \<table\>\<caption\>\<h3\>Selected Books\</h3\>\</caption\>
> \<thead\>\<tr\>\<th\>Book\<th\>Authors\<tbody\>$(@htl("""
> \<tr\>\<td\>$(book.name)\<td\>$(join(book.authors, " & "))
> """)) for b in books)\</tbody\>\</table\>""")
> \`\`\`
> 
> While one might argue that the latter form is particularly fine, this example works because HTL uses Julia's syntax and excellent parser. Notations defined outside of Julia's ecosystem won't have this luxury.
> 
> In conclusion, a succinct, unsurprising, and nestable way to incorporate foreign notations as Julia expressions will open up opportunities for innovative uses of Julia's excellent macro system and dynamic programming environment. What are the costs? A relatively simple parser rule and integration with existing string macros and... the assignment of a Unicode pair.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [August 29, 2023, 10:13pm UTC](https://discourse.julialang.org/t/how-would-you-compare-metaprogramming-in-julia-and-tcl/103342/5 "2023-08-29T22:13:01Z")

</div>

Personally not sold on that proposal because

1. another delimiter would have its own character nesting and escaping problems.
2. it doesn’t seem accurate to lump all non-standard string literals together because the macros themselves do parse the input strings differently. `raw_str` actually does nothing to an input string, so the standard string does other things on top of that, like `$`-interpolation.
3. preserving the nested text of non-standard strings seems inconsistent with how I expect macros to work. It can be useful, especially for other language’s nonstandard strings, but that could be implemented as part of the particular macro’s parsing (maybe it makes `$` or some other special characters exclude text from special parsing) and isn’t always what I’d want. If I do `remove_all_b""" printbln(b"hellob, bworld") """`, I expect all the b’s to be removed `""" println("hello, world") """`, no matter what delimiters replace the quotes. This isn’t particular to non-standard strings, standard strings do things to nested text:

```julia
julia> x = "whoops"
"whoops"

julia> "$x", raw""" "$x" """ # nested standardness not preserved
("whoops", " \"\$x\" ")

julia> raw"$x", """ raw"$x" """ # nested rawness not preserved
("\$x", " raw\"whoops\" ")

julia> y = raw"$x"; "$x $y" # but standard literals can interpolate
"whoops \$x"

```

But this is quite a tangent from the Tcl thread, should be its own thread or conversation.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [August 29, 2023, 10:25pm UTC](https://discourse.julialang.org/t/how-would-you-compare-metaprogramming-in-julia-and-tcl/103342/6 "2023-08-29T22:25:46Z")

</div>

As for me, I never used meta programming neither in TCL or Julia, so I cannot contribute.

TCL for what I used it, is a scripting language which integrates well with external programs (awk, sed, etc), but I never saw it as a programming language for more than that.

---

<div class="post-metadata">

**Author:** ![George9000](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/george9000/32/23619_2.png) [@George9000](https://discourse.julialang.org/u/George9000)\
**Post date:** [August 29, 2023, 10:46pm UTC](https://discourse.julialang.org/t/how-would-you-compare-metaprogramming-in-julia-and-tcl/103342/7 "2023-08-29T22:46:53Z")

</div>

This is old, but a good quick [overview](http://antirez.com/articoli/tclmisunderstood.html). Also [this](https://philip.greenspun.com/tcl/) which has [roots](https://philip.greenspun.com/panda/) in MIT, before the _Software Engineering for Internet Applications_ book and course. First came into contact with Tcl while learning [OpenACS](https://openacs.org/), the Millennium Falcon of web frameworks.

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [August 30, 2023, 12:29am UTC](https://discourse.julialang.org/t/how-would-you-compare-metaprogramming-in-julia-and-tcl/103342/8 "2023-08-30T00:29:11Z")

</div>

> [@TheCedarPrince](#):
>
> He and I were discussing a domain specific language (DSL) he had created in Tcl that had syntax that looked something like this:
> 
> ```julia
> I wish this was $blue
> 
> ```

You can do something similar in Julia:

```julia
julia> macro _(args...)
           (args..., )
       end
@_ (macro with 1 method)

julia> @_ I wish this was $blue
(:I, :wish, :this, :was, :($(Expr(:$, :blue))))

```