# Extract variables from strings

**URL:** <https://discourse.julialang.org/t/extract-variables-from-strings/72881>\
**Category:** General Usage\
**Tags:** macros, strings\
**Created:** [December 10, 2021, 8:02am UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881 "2021-12-10T08:02:45Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [December 10, 2021, 8:02am UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/1 "2021-12-10T08:02:45Z")

</div>

I’d like write a macro to extract variables from strings. So writing

```julia
@string_extract "test var_1 and var_2" "test $x and $y"

```

would set

```julia
  x = "var_1"  
  y = "var_2"

```

So, is there a function to convert `"test $x and $y"` into a list or dictionary like:

```julia
[
  "test " : string,
  x : variable,
  " and " : sting,
  y : variable 
]

```

---

<div class="post-metadata">

**Author:** ![Bardo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bardo/32/21601_2.png) [@Bardo](https://discourse.julialang.org/u/Bardo)\
**Post date:** [December 10, 2021, 9:37am UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/2 "2021-12-10T09:37:44Z")

</div>

Possibly related: [Parse structured string to dictionary - General Usage - JuliaLang](https://discourse.julialang.org/t/parse-structured-string-to-dictionary/8126)

A simple

```julia
_, x, _, y = split("test var_1 and var_2")

```

sets x and y as desired.  
It was not clear from your description if variable names x and y are hard-wired or to be parsed first.  
But using `$x` in a string does interpolation and requires x to be known.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [December 10, 2021, 9:54am UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/3 "2021-12-10T09:54:49Z")

</div>

```julia
julia> Meta.@dump "test $x and $y"
Expr
  head: Symbol string
  args: Array{Any}((4,))
    1: String "test "
    2: Symbol x
    3: String " and "
    4: Symbol y

```

The expression itself is already the solution that you want.

This could be one implementation, but it will only work if there are no two interpolations side by side:

```julia
macro string_extract(string::String, strexp::Expr)
    range = 0:0
    args = strexp.args
    strindices = map(args) do arg
        !(arg isa String) && return missing
        range = findnext(arg, string, range[end]+1)
    end

    ranges = map(1:length(strindices)) do i
        if ismissing(strindices[i])
            start = i == 1 ? 1 : strindices[i-1][end]+1
            stop = i == length(strindices) ? length(string) : strindices[i+1][1]-1
            start:stop
        else
            strindices[i]
        end
    end

    args .=> getindex.(string, ranges)
end

julia> @string_extract "test var_1 and var_2" "test $x and $y"
4-element Vector{Pair{A, String} where A}:
 "test " => "test "
      :x => "var_1"
 " and " => " and "
      :y => "var_2"

```

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [December 10, 2021, 10:18am UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/4 "2021-12-10T10:18:45Z")

</div>

Fantastic 🙂  
but how do you go from this to having variables x and y with values “var\_1” and “var\_2” ?

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [December 10, 2021, 12:46pm UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/5 "2021-12-10T12:46:43Z")

</div>

```julia
macro string_extract(string::String, strexp::Expr)
    range = 0:0
    args = strexp.args
    strindices = map(args) do arg
        !(arg isa String) && return missing
        range = findnext(arg, string, range[end]+1)
    end

    ranges = map(1:length(strindices)) do i
        if ismissing(strindices[i])
            start = i == 1 ? 1 : strindices[i-1][end]+1
            stop = i == length(strindices) ? length(string) : strindices[i+1][1]-1
            start:stop
        else
            strindices[i]
        end
    end

    pairs = args .=> getindex.(string, ranges)
    Expr(:block, [Expr(:(=), esc(sym), string)
        for (sym, string) in pairs if sym isa Symbol]...)
end

```

```julia
julia> @macroexpand @string_extract "test var_1 and var_2" "test $x and $y"
quote
    x = "var_1"
    y = "var_2"
end

```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [December 10, 2021, 12:52pm UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/6 "2021-12-10T12:52:05Z")

</div>

> [@Lincoln\_Hannah](#):
>
> I’d like write a macro to extract variables from strings. So writing

The input to a macro is already a parsed expression — the point of using macros is generally to employ Julia-parsable syntax that is rewritten in some custom way.

It’s impossible to say for sure without knowing what you are ultimately trying to do, but as a general rule I would tend to re-think using string-based symbolic expressions.

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [December 10, 2021, 10:05pm UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/7 "2021-12-10T22:05:47Z")

</div>

Sorry Jules.  
This is amazing but after running the macro I’d like to write `"extracted variables are: $x and $y"`  
and get `"extracted variables are: var_1 and var_2" ` but I get `UndefVarError: x not defined`

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [December 10, 2021, 10:24pm UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/8 "2021-12-10T22:24:58Z")

</div>

Final goal. Say I need to transform

```julia
Input = """"
Curve.My_Curve_1,Points=[(1,10),(2,11),(3,12)]
Curve.My_Curve_2,Points=[(5,8),(6,13),(7,15)]
Curve.My_Curve_3,Points=[(11,8),(12,13),(13,15)]
"""

```

to

```julia
Output = """
Curve My_Curve_1 x_values=[1,2,3] y_values=[10,11,12]
Curve My_Curve_2 x_values=[5,7,7] y_values=[8,13,15]
Curve My_Curve_3 x_values=[11,12,13] y_values=[8,13,15]
"""

```

Say we can use special characters `≪ ≫` to represent a newline separated list and `⪻ ⪼` a comma separated list. I’d like to write something like

```julia
@extract Input "≪Curve.$curve_name,Points=[⪻($x,$y)⪼]≫"

@insert "≪Curve $curve_name x_values=[⪻ $x ⪼] y_values=[⪻ $y ⪼]≫"

```

Maybe a function is better suited than macros.  
If it is possible in some form though, something like this, I think, is more readable than multiple split and join statements, or regex.

It would be easier if the required inputs / outputs were in a readable format like JSON or XML but this isn’t always the case.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 10, 2021, 11:29pm UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/9 "2021-12-10T23:29:54Z")

</div>

Definitely use a function for this. No need for macros here.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [December 11, 2021, 8:51am UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/10 "2021-12-11T08:51:12Z")

</div>

You could also use `ReadableRegex.jl` although your structure needs more like a parser program…

```julia
using ReadableRegex

input = "Curve.My_Curve_1,Points=[(1,10),(2,11),(3,12)]"

r = "Curve." *
    capture(one_or_more(ANY), as = "curve") *
    ",Points=[" *
    capture(one_or_more(ANY), as = "points") *
    "]"

m = match(r, input)
c = m["curve"]
p = m["points"]

```

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [December 11, 2021, 9:54am UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/11 "2021-12-11T09:54:22Z")

</div>

That’s really cool. Had no idea that existed

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [December 11, 2021, 10:07am UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/12 "2021-12-11T10:07:42Z")

</div>

Yeah I wrote that for long complicated regexes that are impossible to read otherwise.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [December 11, 2021, 2:04pm UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/13 "2021-12-11T14:04:07Z")

</div>

> [@Lincoln\_Hannah](#):
>
> Final goal. Say I need to transform
> 
> ```julia
> Input = """"
> Curve.My_Curve_1,Points=[(1,10),(2,11),(3,12)]
> Curve.My_Curve_2,Points=[(5,8),(6,13),(7,15)]
> Curve.My_Curve_3,Points=[(11,8),(12,13),(13,15)]
> """
> 
> ```

This is valid Julia syntax, so you can let Julia parse it. For example:

```julia
@mymacro begin
    Curve.My_Curve_1,Points=[(1,10),(2,11),(3,12)]
    Curve.My_Curve_2,Points=[(5,8),(6,13),(7,15)]
    Curve.My_Curve_3,Points=[(11,8),(12,13),(13,15)]
end

```

will pass the parsed expression (abstract syntax tree) your macro, which can then transform it in some way before it is executed.

What is your _actual_ final goal? It seems like you are trying to define some kind of domain-specific language? For what purpose?

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [December 12, 2021, 1:59am UTC](https://discourse.julialang.org/t/extract-variables-from-strings/72881/14 "2021-12-12T01:59:49Z")

</div>

Moving data between systems that have different proprietary text formats. In my example, the input could be the output from System 1 and the output could be the required input for System 2.

Rather than write lots of split statements I wanted to declarativly express the the format of each system.