# Reading matrices from an external file inside a function

**URL:** <https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558>\
**Category:** New to Julia\
**Created:** [August 14, 2019, 8:43pm UTC](https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558 "2019-08-14T20:43:14Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![qwerytyui1234](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/qwerytyui1234/32/31952_2.png) [@qwerytyui1234](https://discourse.julialang.org/u/qwerytyui1234)\
**Post date:** [August 14, 2019, 8:43pm UTC](https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558/1 "2019-08-14T20:43:15Z")

</div>

Hey guys,

I’m wondering if anyone can help with this issue.

So the minimal example is as follows.

Suppose I have a “A.jl” file in the present working directory and it contains the following.

```
f = [x+y; x+2]; 

```

And I want to read this file into another jl file called “B.jl”.

In the “B.jl” file, I would have the following

```
x = 2
y = 2
include("A.jl")

```

Then the output is f = [4;6].

And this works as intended, but what if I want to use include inside a function? I know that include brings variables only work in global scope, so there is a problem. Basically something like this in the “B.jl” file.

```
function f()
    x = 2 
    y = 2
    include("A.jl")
end

```

This would throw an error. I searched on Julia’s discourse and I found this [link](https://discourse.julialang.org/t/is-include-safe-to-use-inside-a-function/10867/11).

So this link says that I could come up with a macro to allow for some variables to be read in locally as well, but using macro is impossible in my case since the variables inside “A.jl” is around 70x80 matrices and probably more than 100 variables (which I created using SymEngine in by running another function). So creating macro one by one seems infeasible.

So I guess I’m wondering if you guys had any idea on how to read such a file into a local scope? This workflow works in Matlab, but this doesn’t work in Julia.

Thank you in advance guys.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [August 14, 2019, 9:03pm UTC](https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558/2 "2019-08-14T21:03:18Z")

</div>

> [@qwerytyui1234](#):
>
> read such a file into a local scope

If “such a file” is a normal julia source code and reading into local scope meaning create local variables depending on the variables defined in the file then no it’s impossible and won’t be possible.

> [@qwerytyui1234](#):
>
> So this link says that I could come up with a macro to allow for some variables to be read in locally as well, but using macro is impossible in my case since the variables inside “A.jl” is around 70x80 matrices and probably more than 100 variables (which I created using SymEngine in by running another function). So creating macro one by one seems infeasible.

No. That’s not what that link says. That link says that you could create a macro so that it’s as if you just included the content of `A.jl` in at compile time, i.e. when you define the function `f`. You will not read in a different value when you run `f` and can’t pass in the file name at runtime (as a parameter to the function).

Now you should treat your input file like any other data file and what you need is just a way to read in a file containing data. If you do this then yes there are many different ways to do what you want. I highly doubt your requirement contains being able to support full julia syntax so you likely don’t need to use a full julia script to store your data. You shouuld identify your actual requirements to help yourself and others to decide which format is the best fit for you.

It **is** also possible if you need to support full julia syntax. However, that means you are throwing anything remotely related to sercurity out of the door and is getting rid of any gard from any user mistake (let along users with ill-intent).

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [August 14, 2019, 9:28pm UTC](https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558/3 "2019-08-14T21:28:06Z")

</div>

Evaluating an expression for different values of variables is a job for a function.  
So rewrite your file `matrix.jl` to contain a function definition:

```julia
f(x, y) = [x + y, x + 2]

```

Now do `include("matrix.jl")` at the top level and then

```julia
x = y = 2
M = f(x, y)

```

---

<div class="post-metadata">

**Author:** ![qwerytyui1234](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/qwerytyui1234/32/31952_2.png) [@qwerytyui1234](https://discourse.julialang.org/u/qwerytyui1234)\
**Post date:** [August 14, 2019, 9:28pm UTC](https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558/4 "2019-08-14T21:28:25Z")

</div>

Ah, okay thanks for your help. I must’ve misunderstood the link.

Yeah so the file that I would be reading is something I would create using SymEngine. However, SymEngine creates Arrays of Basics. So if I create a function

```julia
function A()
f = [x, y] 
return f
end

```

where x,y are symbols, then f that returns it is an array of symbols.

I realize that I could then sub this f in using usual substitution, but it does take about couple of seconds with my application and when I end up calling this function and subbing stuff in iteratively (for example if I am estimating it), it does take longer than I thought. Using include(“.jl”) or calling from a Julia script(?) seemed to be faster because then this basically turns an Array of symbols to an Array of floats much faster.

> [@yuyichao](#):
>
> However, that means you are throwing anything remotely related to sercurity out of the door and is getting rid of any gard from any user mistake (let along users with ill-intent).

May I ask what this means? I’m just running functions locally on my computer and I am creating the script that I would call so I know what is on here (as in I know the exact contents on “A.jl” file). I guess in other applications, this is probably not safe?

I have also tried using JLD, but for some reason, I get some kind of memory error as well… But I realize this approach is probably not generally not recommended or possible, so thank you! I appreciate your help.

Sorry for perhaps a stupid question…

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [August 14, 2019, 9:30pm UTC](https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558/5 "2019-08-14T21:30:40Z")

</div>

It’s not a stupid question at all, but it is difficult to answer without full information. Please provide a MWE (minimal working example). If you use SymEngine then you should use that in a simple way in your MWE.

---

<div class="post-metadata">

**Author:** ![qwerytyui1234](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/qwerytyui1234/32/31952_2.png) [@qwerytyui1234](https://discourse.julialang.org/u/qwerytyui1234)\
**Post date:** [August 14, 2019, 9:31pm UTC](https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558/6 "2019-08-14T21:31:37Z")

</div>

I thought about doing this, but I was hoping to avoid it because there are just way too many variables (about 100 of them) that I would have to define.

However, if this is the only way, I guess I could do something like this, it will just be fairly tedious :(.

Anyway, thank you!

---

<div class="post-metadata">

**Author:** ![qwerytyui1234](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/qwerytyui1234/32/31952_2.png) [@qwerytyui1234](https://discourse.julialang.org/u/qwerytyui1234)\
**Post date:** [August 14, 2019, 9:40pm UTC](https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558/7 "2019-08-14T21:40:26Z")

</div>

Cool. Okay here’s a MWE that I could think of.

```julia
using SymEngine 
@vars x y

function model()
    return f = x+y
end
f = model() #give f as an symbol
f = subs(f,(x,1),(y,1)) #outputs f = 2 as a number

```

However, I could also do this!

```julia
using SymEngine 
@vars x y

function model()
    return f = x+y
end
f = model() #give f as an symbol
io = open("A.jl","w");
write(io,replace("nf = $f"),"Basic"=>"")
close(io)

```

Then open up another “B.jl” file then write

```julia
x = 1
y = 1
include("A.jl")

```

Then nf = 2. However, since I don’t have to use subs function, this turned out to be faster in larger examples. But if I ever want to use

```julia
function BB(x,y)
include("A.jl")
return nf
end

```

Then this wouldn’t work because include does not work inside a function (in this sense).

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [August 14, 2019, 9:53pm UTC](https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558/8 "2019-08-14T21:53:16Z")

</div>

Sorry I thought `around 70x80 matrices and probably more than 100 variables` are static data (also since that’s what the linked post is talking about).

In any case, most of what I said still apply. It’ll not be possible to do the evaluation by reading the file and you definately need to change your format.

> [@qwerytyui1234](#):
>
> May I ask what this means? I’m just running functions locally on my computer and I am creating the script that I would call so I know what is on here (as in I know the exact contents on “A.jl” file). I guess in other applications, this is probably not safe?

That’s why I also mentions errors. In general, whether your current setup is security sensitive or not, you should never treat data as code without strict checking.

The question now is, what’s the full problem you are trying to solve, why do you need `A.jl`.

Is `A.jl` your input? (From what you said that doesn’t seem to be the case. `A.jl` seems to be something you generate)

Is `A.jl` just a way for you to get the `[x + y, x + 2]` as an expression? In that case going through a file is definately the wrong way to do it. In this case, you are just looking for a way to go from `f` to the function of `f(x, y) = [..., ...]` and you should not have to go though a file to do that. I don’t know what tool you have from `SymEngine` so I can’t help you with that. In the worse case though, doing string interpolation would be better at least, without going through a file.

Is `A.jl` for caching intermediate result? AFAICT, the particular stage doesn’t seem like a good place to save the data. There’s probably a better way to store the `SymEngine` objects directly, or you should probably save the “compiled” version `f(x, y) = [..., ...]` mentioned in the previous case. In that case, you are basically doing source code generation.

---

<div class="post-metadata">

**Author:** ![qwerytyui1234](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/qwerytyui1234/32/31952_2.png) [@qwerytyui1234](https://discourse.julialang.org/u/qwerytyui1234)\
**Post date:** [August 14, 2019, 10:11pm UTC](https://discourse.julialang.org/t/reading-matrices-from-an-external-file-inside-a-function/27558/9 "2019-08-14T22:11:07Z")

</div>

> [@dpsanders](#):
>
> Evaluating an expression for different values of variables is a job for a function.  
> So rewrite your file `matrix.jl` to contain a function definition:
> 
> ```julia
> f(x, y) = [x + y, x + 2]
> 
> ```
> 
> Now do `include("matrix.jl")` at the top level and then
> 
> ```julia
> x = y = 2
> M = f(x, y)
> 
> ```

I think this is the best way to do this that I can think of at the moment. Thank you sincerely to both of you for your help and info!