"using" as a function?

General Usage
1

I am messing around with Pluto and decided I would like a shorter way of importing libraries. This question is sort of two fold. How can I dynamically import Modules using a macro or something else? And why is using a keyword a not a function? I suppose this could go into the Julia internals subject, but this just struck me as strange.

What I would like to do.

using Pkg
pkgs = ["HTTP","DataFrames","CSV"]
Pkg.add(pkgs)
using.(pkgs)

I understand this is probably something that I should just overlook, but I found it a good chance to try and learn macros, but I only got close to what i wanted.

 function toLine(arr)
	out = ""
	for i in 1:1:length(arr)
		if i == length(arr)
			out *= arr[i]
		else
			out *= arr[i] * ", "
		end
	end
	return out
end
macro useit(libs)
	return Meta.parse("using " * toLine(libs))
end
@useit(pkgs)
LoadError: MethodError: no method matching length(::Symbol)
Closest candidates are:
length(!Matched::Core.MethodTable) at reflection.jl:957
length(!Matched::CompositeException) at task.jl:41
length(!Matched::Base.Iterators.Flatten{Tuple{}}) at iterators.jl:1061

Iā€™m sure I could do it with a macro, I was just curious if anyone knew if there is a function for this or a way of calling ā€œusingā€ that makes this work. I just noted that it was strange I could add Packages using a variable, but not easily ā€œusingā€ the Modules.

So much stuff is a function I am confused as to why using would be something special.

2

Thereā€™s no need for a macro hereā€“if you want a function that has the same effect as using at the top level, then itā€™s as simple as:

julia> function my_using(x)
         @eval using $(Symbol(x))
       end
my_using (generic function with 1 method)

julia> pkgs = ["LinearAlgebra", "Random"]
2-element Array{String,1}:
 "LinearAlgebra"
 "Random"

julia> my_using.(pkgs)

using has side-effects (bringing names into scope) which would be very surprising for a function. If you run foo() you do not expect the name bar to suddenly appear in global scope.

By the way, if youā€™re looking to learn more about using macros, I would suggest that you stick to a simple rule of thumb:

Never use Meta.parse, eval, or string concatenation in a macro or in any function used by a macro.

Of course all rules have exceptions, but this is a pretty good guideline to live by when dealing with metaprogramming in Julia. Item by item, the reasons are:

  • You should not need Meta.parse in a macro because a macro already receives a parsed Julia expression. Just use the Expr that the macro received and operate directly on that expression.
  • You should not need eval in a macro because a macro produces code which will eventually be run. If your macro works with the value of some function f(), then it should produce code which calls f() rather than trying to @eval(f()) inside the macro body. The reason is that macro expansion happens when the code is lowered, which is almost never the right time to be calling other functions.
  • You should not need string concatenation because Julia has better tools (like Expr) for representing expressions. Instead of concatenating strings, you can push! into expr.args. This avoids all of the issues like syntax errors or missing escaping of spaces or commas which are all-too-common in lesser macro systems like C/C++.
22 Likes
3

That makes utter sense and is highly appreciated. I suffer from not understanding meta programming yet and am an amateur programmer. I started with cpp and have just picked up the hobby of checking out languages. Julia is really interesting, like a glue language. Anyways, thanks for the well reasoned and thought out answer!

1 Like
4

Can we get this answer turned intoa thing that is really easy to link? I feel like someone has to give this answer about one a week.

3 Likes
5

Just to clarify the bit about avoiding string concatenation: Does this apply only to modifying julia code strings (which would then be parsed), or does this apply to string concatenation in general?

6

Iā€™m pretty sure heā€™s just talking about using it for building string representations of code.

1 Like
7

In principle also applies if you are parsing the result of string manipulation into some structure, even if the parsing result isnā€™t julia code. Case in point, donā€™t use string manipulation to construct a JSON string just to parse it immediately (if you are doing that to pass it to javascript thatā€™s probably the best you can do so thatā€™s fine). Obviously string manipulation is still useful and you should use it when you need a string.

1 Like
8

Excellent writeup, please consider making a PR to the docs, eg as a FAQ.

1 Like
9

Yeah, @Mason is rightā€“I mean avoid concatenating strings to build Julia code. Itā€™s still fine to concatenate "hello" and "world" if your problem requires it :slightly_smiling_face:

10

Also, if you are adding packages in a script, you should really be using a Project.toml and a Manifest.toml along with ] instantiate.

11

It was mainly something I wanted to do for a Pluto project. Not sure if there was a way of using Project or Manifest for that.

12

A Pluto project is just a regular julia Project - all code is stored in src and its has a Project.toml and Manifest.toml :slight_smile:

13

I am a very slow learner about macros. I had a macro that added a seldom-used directory to my LOAD_PATH with eval. I was only calling this thing from the REPL and notebooks. I thought it had to be a macro, but making it a function worked as well.

@Oscar_Smith says, "I feel like someone has to give this answer about one a week.ā€˜ā€™, but some of us need to see it several times before it sinks in.

2 Likes