When I read or hear about domain-specific languages in Julia I tend to cringe a little. In most cases it is not about a language, but rather an API. The difference? A DSL uses a syntax appropriate for its domain. An API is restricted by the syntax rules of the hosting language, in this case Julia. For example, DSL SELECT * FROM table1 WHERE age>50 API # QueryVerse @from i in table1 @where i.age>50 @select i # JuliaDB filter( p->p.age > 50 , table1) I don’t mean to pick on QueryVerse or JuliaDB specifically. There are many other examples in other domains. What they all have in common is that they are restricted to either Julia macros and/or Julia functions, both of which must adhere to Julia syntax rules. In my opinion each Domain-Specific Language needs its own parser. The only example I know of in Julia are regular expressions. Those are parsed by a regular expression parser. Why not do something similar with other DSL. SQL""" SELECT * FROM table1 WHERE age > 50 """ GAMS""" Parameter c(i,j) transport cost in thousands of dollars per case ; c(i,j) = f * d(i,j) / 1000 ; Variables x(i,j) shipment quantities in cases z total transportation costs in thousands of dollars ; Positive Variable x ; Equations cost define objective function supply(i) observe supply limit at plant i demand(j) satisfy demand at market j ; cost .. z =e= sum((i,j), c(i,j)*x(i,j)) ; supply(i) .. sum(j, x(i,j)) =l= a(i) ; demand(j) .. sum(i, x(i,j)) =g= b(j) ; Model transport /all/ ; Solve transport using lp minimizing z ; Display x.l, x.m ; """ XPath""" // All elements where the first child element has the value Bob. author[last-name [position()=1]= "Bob"] """ A general Julia parser-generator that can convert a true DSL to corresponding Julia expressions might be helpful here (For inspiration see ANTLR4) Let the discussion begin :slight_smile:

Parser generators do exist in the package community, See packages like [image] GitHub - chakravala/Reduce.jl: Symbolic parser for Julia language term... Symbolic parser for Julia language term rewriting using REDUCE algebra - GitHub - chakravala/Reduce.jl: Symbolic parser for Julia language term rewriting using REDUCE algebra That’s an example of a Julia parser generator that translates to and from Julia expressions from another lang

Following the theme of your DB examples, this Julia package seems to get very close to the syntax you want: https://github.com/wookay/Octo.jl

Do you mean we should embrace the two language issue instead of solving it…

The two language issue refers to having to use two different generic programming languages to properly balance ease of use and performance. In this case we’re talking about adding Domain Specific Languages to a generic programming language. Regular expressions are good example of a DSL embedded in a generic language such as Julia. In my opinion packages like Octo.jl and Query.jl don’t go far enough. They both borrow from SQL, but can’t use the exact SQL syntax. As a result we have to learn multiple language dialects and remember how they differ from the SQL language standard.

[image] jandehaan: They both borrow from SQL, but can’t use the exact SQL syntax. As a result we have to learn multiple language dialects and remember how they differ from the SQL language standard. The idea of creating a language-specific API with the intention of imitating some imagined “master dialect” of SQL makes me feel kind of dirty. I don’t think the fact that SQL is a horrorshow as something that can or should be solved by any language-specific API, it would make much more sense to me to take a language-agnostic translation driver that transcompiles SQL code like what is done with some flavors of JavaScript. While this could be implemented in Julia, it seems somehow separate from a regular Julia SQL API. For the latter, I’d much rather some sort of lazy table abstraction like a Spark dataframe, something like that seems more worthy of people’s efforts than a SQL-lookalike, considering that the programming paradigms within SQL are, well, seemingly a bizarre relic of a time when COBOL was all the rage. I of course realize that many will disagree with this stance, and those people are certainly doing nice work, Octo is pretty cool, and it’s wild to see how far the Julia syntax can be pushed. I certainly agree with you that the proliferation of many distinct SQL dialects is a problem in need of solving.

Hello, From linked discussion by waldyrious it seems that the way to go on making own DSL is to create own REPL. On the other hand I read in https://github.com/JuliaLang/julia/pull/24945#issuecomment-349739529 that “REPL code will be considered non-public and unstable in 1.0”. Is there any canonical document how to deal with making own DSL considering the above? any examples are welcome.

I have a package ReplMaker.jl which exposes a simpler API for building repl modes. It’s currently very basic and limited but for now, it’s still a nice option if one doesn’t want to deal with the difficult and confusing process of interfacing with the REPL standard library directly. If anyone out there wants to collaborate on making ReplMaker.jl a bit more robust and versatile, I’d be very interested! Note that one does not necessarily need a separate repl mode for a DSL. It’s just a nice thing to have if you want a clear separation between interacting with your DSL and standard Julia or if you want to use semantics which are incompatible with Julia’s parser.

Thanks Mason for your input and ReplMaker.jl. I’ve spotted your package from linked discussion. It is really interesting. It is so small that it is actually more robust (in terms of reducing deps) to include same logic in a package that would like to use ReplMaker. REPL standard library seems to be going non-public, thus your package gets some extra rationale by handling interfacing to non-public API which is likely to change over time. On the other hand, still it is so small. My proposal would be to submit your ReplMaker as PR to REPL std lib, so it can be used as public API for interfacing with REPL. Isn’t that the ideal solution for everyone? Being able to build true DSL is important feature of Julia, but making REPL std lib a non-public is an obstacle for doing that. As for doing DSL without separate REPL, AFAIU (I actually understand very few as I am newcomer to Julia) you have to use macros, which are constrained by Julia macro definition, is that correct?

There are non standard string literals, which are not constrained to use julia syntax.

"Domain-Specific Languages" in Julia

Internals & Design

waldyrious August 10, 2018, 5:22pm 3

For reference, there’s been some relevant discussion in a previous thread: Seamlessly extending the Julia syntax.

Topic		Replies	Views
Love I Julia : the need for a direct object notation Internals & Design proposal	46	6338	January 20, 2017
How Would You Compare Metaprogramming in Julia and Tcl? General Usage metaprogramming , tcl	7	638	August 30, 2023
Julia included in Oreilly's "Emerging Programming Languages" report (June 2019) Community	52	3784	June 26, 2019
A Julia DSL for language models Machine Learning ai	14	2222	February 28, 2024
Multi-line macros for domain specific language General Usage macros , metaprogramming	5	1385	August 29, 2017

"Domain-Specific Languages" in Julia

Related topics