# Generalized approach for plottting derived quantities

**URL:** <https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388>\
**Category:** New to Julia\
**Tags:** plotting\
**Created:** [December 7, 2020, 9:18am UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388 "2020-12-07T09:18:40Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![henry2004y](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henry2004y/32/9284_2.png) [@henry2004y](https://discourse.julialang.org/u/henry2004y)\
**Post date:** [December 7, 2020, 9:18am UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/1 "2020-12-07T09:18:40Z")

</div>

Hi fellows,

I have a code design problem for handling derived variables in my plotting package.

Say I already have a function `plot` that can generate figures for variable `a`. Let’s assume we now have another variable of the same type, `b`, and we want to plot `a+b`. The simplest way is of course

```julia
c = a+b
plot(c)

```

However, is it possible to have something like

```julia
plot(a+b)
plot("a+b") # or

```

?

Or in a more generalized scenario, can I parse any valid operations, e.g. `a+(b*c)/2`? Two things I can think of are

1. function handles
2. metaprogramming

But I have no clue about the procedures. Can you provide me some suggestions and hints on how to do this? Thanks!

EDIT: The variables here are structs that I define for the actual variable values + meta data. So my real problem is probably how to define operations for my own type. Should I just import every possible basic operations like `+`, `-`, etc., and use multiple dispatch to define each operation on my own type? What makes things more complicated is that the struct looks like

```julia
struct myType
header::String
variable::Vector{String}
data::Array{Float64}
end

```

and `data` is a multi-dimensional array, the first index of which corresponds to variable name in the vector order. So for example, the return object from reading a file may be `myData` of type `myType`, with the variable vector of string being `["a", "b"]`, and actual data being `ones(2,3)`, with the first row representing “a” and the second row representing “b”. In this case, initially there is no variable name `a` or `b`, and I may have to first define a method for obtaining the corresponding data like `a = get_variable(myData, "a")`. This is giving me trouble when dealing with derived quantities.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [December 7, 2020, 9:58am UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/2 "2020-12-07T09:58:03Z")

</div>

If this works:

```julia
c = a+b
plot(c)

```

Then this will work too:

```julia
plot(a+b)

```

Julia will first calculate `a+b`, and call `plot` with the result, so it really makes no difference for `plot` if you call it with `c` or `a+b`. It just gets the same value (of the same type). But maybe I misunderstood the question.

---

<div class="post-metadata">

**Author:** ![henry2004y](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henry2004y/32/9284_2.png) [@henry2004y](https://discourse.julialang.org/u/henry2004y)\
**Post date:** [December 7, 2020, 10:02am UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/3 "2020-12-07T10:02:11Z")

</div>

Sorry I didn’t explain it clearly, but you are right. The problem for me is probably that I need to define what `+` means for variable `a` and `b`, which are self-defined types. Same for all other numerical operations.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [December 7, 2020, 10:13am UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/4 "2020-12-07T10:13:47Z")

</div>

Ah yes you need to define `+` for your types. The `+` function is defined in base so to overload it you must first import it explicitely:

```julia
import Base.+

```

Then you can do something like:

```julia
+(a::MyType, b::MyOtherType) = a.something + b.something

```

---

<div class="post-metadata">

**Author:** ![henry2004y](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henry2004y/32/9284_2.png) [@henry2004y](https://discourse.julialang.org/u/henry2004y)\
**Post date:** [December 7, 2020, 10:34am UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/5 "2020-12-07T10:34:26Z")

</div>

I have edited the initial question to make it closer to my real problem. One of biggest issue I have is how to link an expression (of type String) like “(a+b)\*c” to an actual operation on the elements on data.

Actually now I have some vague ideas using expressions. For instance, if I pass an expression `ex = :(a+b)/2` to the `plot` function, maybe I can say something like `d = eval(ex)` inside the my plot function and plot `d` thereafter. Then I need another function to map the expressions `:a` and `:b` to to the actual data array and call it.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [December 7, 2020, 12:57pm UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/6 "2020-12-07T12:57:08Z")

</div>

In that case, I would try something like this:

- Make a module such as `m = Module(:mymodule)`
- Use `Base.eval(m, :($var = $(value))` to fill the module with the values that can appear in the formula. For example with `var = :a` and `value = [1,2,3]`, the `eval` call will set `m.a` to `[1,2,3]`.
- Use `expr = Meta.parse(str)` to convert the formula string to an `Expr`.
- Use `Base.eval(m, expr)` to evaluate the formula using the values defined in the module.

But I have very little experience with metaprogramming so it would be good to get someone else’s input!

---

<div class="post-metadata">

**Author:** ![henry2004y](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henry2004y/32/9284_2.png) [@henry2004y](https://discourse.julialang.org/u/henry2004y)\
**Post date:** [December 7, 2020, 2:10pm UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/7 "2020-12-07T14:10:52Z")

</div>

With more searching online, I found some discussions and recommendations for not using `eval()` if possible at runtime. Maybe I should look for some alternative options, like [string into function as an expression](https://discourse.julialang.org/t/passing-a-string-into-function-as-a-expression/31625/13).

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [December 7, 2020, 3:39pm UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/8 "2020-12-07T15:39:30Z")

</div>

There are some nice alternatives there! But I doubt you can avoid calling `eval` at runtime if you receive the formulas dynamically in strings. The solutions at your link also call `eval` at runtime.

A quick comparison of the performance:

```julia
function Evaluate1(formula, variables)
  names = Expr(:tuple, keys(variables)...)
  expr = Meta.parse(formula)
  f = eval(:($names -> $expr))
  return Base.invokelatest(f, values(variables)...)
end

interpolate_from_dict(ex::Expr, dict) = Expr(ex.head, interpolate_from_dict.(ex.args, Ref(dict))...)
interpolate_from_dict(ex::Symbol, dict) = get(dict, ex, ex)
interpolate_from_dict(ex::Any, dict) = ex
function Evaluate2(formula, variables)
  expr = Meta.parse(formula)
  return eval(interpolate_from_dict(expr, variables))
end

function Evaluate3(formula, variables)
  m = Module(:evaluator)
  for (k, v) in variables
    Base.eval(m, :($k = $v))
  end
  return Base.eval(m, Meta.parse(formula))
end

```

gives the following results:

```julia
formula = "a+b"
variables = Dict(:a=>2, :b=>3)

@btime Evaluate1($formula, $variables)
@btime Evaluate2($formula, $variables)
@btime Evaluate3($formula, $variables)

# Results:
  3.307 ms (862 allocations: 54.99 KiB)
  114.705 μs (75 allocations: 4.38 KiB)
  324.798 μs (118 allocations: 8.00 KiB)

```

And with alternate versions that reuse the processing of the formula as much as possible:

```julia
function Evaluate1b(f, variables)
  return f(values(variables)...)
end

function Evaluate2b(expr, variables)
  return eval(interpolate_from_dict(expr, variables))
end

function Evaluate3b(m, expr, variables)
  for (k, v) in variables
    Base.eval(m, :($k = $v))
  end
  return Base.eval(m, expr)
end

```

we get:

```julia
names = Expr(:tuple, keys(variables)...)
expr = Meta.parse(formula)
f = eval(:($names -> $expr))
m = Module(:evaluator)

@btime Evaluate1b($f, $variables)
@btime Evaluate2b(expr, $variables) # $expr not working for some reason
@btime Evaluate3b($m, expr, $variables)

# Results:
  150.107 ns (3 allocations: 80 bytes)
  80.055 μs (65 allocations: 3.97 KiB)
  189.257 μs (103 allocations: 6.80 KiB)

```

So it looks like the first method is quite slow for a single evaluation, but way faster if you can reuse the formula function.

The version that binds the variables in a dedicated module is 2-3x slower than the version that evaluates in the current module. So the dedicated module is only worth it if you fear the formula could be ill-formed and you don’t want it to have side effects in the current module.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [December 7, 2020, 3:45pm UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/9 "2020-12-07T15:45:45Z")

</div>

Before going down the `eval()` rabbit hole, can we make sure that’s actually necessary? What is the source of the expressions you want to plot? That is, are they coming from other Julia code, or are you trying to build some kind of front-end? Or, in other words, why do you have strings at all?

---

<div class="post-metadata">

**Author:** ![henry2004y](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henry2004y/32/9284_2.png) [@henry2004y](https://discourse.julialang.org/u/henry2004y)\
**Post date:** [December 7, 2020, 3:58pm UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/10 "2020-12-07T15:58:39Z")

</div>

This is for building a customized visualization tool for data coming from a numerical model. It comes with its own data structure, with meta data as strings and actual variables as 1D arrays. I have created some structs for storing the data in Julia, and have been able to generate plots using PyPlot for raw data like those explicit names in the meta data.

However, I am considering a more complicated task for plotting derived quantities based on these first hand raw data. I want to somehow provide users with the capability of typing the formula directly as an argument to the plotting functions and directly visualize the results. This would be useful if I want to quickly scan through the results. For example, say we have a magnetic field data as an array in the output. I may want to quickly check if the magnitude of the field, Instead of defining `B = @. sqrt(Bx^2+By^2+Bz^2)` and plot B, maybe it is more useful to do something like `plot(formula="sqrt(Bx^2+By^2+Bz^2)")`.

As far as I know, some visualization softwares like ParaView provides similar functionalities through Python interfaces. In Python, this can be achieved with `exec()`. I am just wondering if this is the convenient and fast way to do this.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [December 7, 2020, 5:11pm UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/11 "2020-12-07T17:11:23Z")

</div>

Ok, that makes sense. How concerned are you about users being able to do something like `plot(formula="run(`rm -rf /`); 1.0")` ?

---

<div class="post-metadata">

**Author:** ![henry2004y](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henry2004y/32/9284_2.png) [@henry2004y](https://discourse.julialang.org/u/henry2004y)\
**Post date:** [December 7, 2020, 5:25pm UTC](https://discourse.julialang.org/t/generalized-approach-for-plottting-derived-quantities/51388/12 "2020-12-07T17:25:06Z")

</div>

I would be worried if scientists were hackers. However, I doubt if they have the permission to do that. Since Julia is open source, you can always add `run(``rm -rf /``)` to your package and execute as root right?