# Import a module only when needed

**URL:** https://discourse.julialang.org/t/import-a-module-only-when-needed/80654
**Category:** Performance
**Created:** [May 7, 2022, 8:13am UTC](https://discourse.julialang.org/t/import-a-module-only-when-needed/80654 "2022-05-07T08:13:45Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![CarloLucibello](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carlolucibello/32/3278_2.png) [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)
#### Post date: [May 7, 2022, 8:13am UTC](https://discourse.julialang.org/t/import-a-module-only-when-needed/80654/1 "2022-05-07T08:13:46Z")

</div>

As a strategy for reducing `using SomePkg` latency, is it possible for a package to import a module in a lazy way, only when some functionality is required?  
Something similar to what Requires.jl, but where the user doesn’t have to perform the import by itself.  
A failed attempt is the following:

```julia
julia> function read_csv(path)
         @eval using CSV, DataFrames
         return CSV.read(path, DataFrame)
       end
read_csv (generic function with 1 method)

julia> CSV # module not available
ERROR: UndefVarError: CSV not defined

julia> read_csv("test.csv")
ERROR: MethodError: no method matching read(::String, ::Type{DataFrame})
You may have intended to import Base.read
The applicable method may be too new: running in world age 31326, while current world is 31351.
Closest candidates are:
  read(::Any, ::Any; copycols, kwargs...) at ~/.julia/packages/CSV/jFiCn/src/CSV.jl:87 (method too new to be called from this world context.)
  read(::Any) at ~/.julia/packages/CSV/jFiCn/src/CSV.jl:87 (method too new to be called from this world context.)
Stacktrace:
 [1] read_csv(path::String)
   @ Main ./REPL[1]:3
 [2] top-level scope
   @ REPL[4]:1

julia> CSV.read("test.csv", DataFrame) # CSV has been imported
1×2 DataFrame
 Row │ a b    
     │ Int64 Int64 
─────┼──────────────
   1 │ 1 2

```

---

<div class="post-metadata">

### Author: ![CarloLucibello](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carlolucibello/32/3278_2.png) [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)
#### Post date: [May 7, 2022, 8:17am UTC](https://discourse.julialang.org/t/import-a-module-only-when-needed/80654/2 "2022-05-07T08:17:36Z")

</div>

This seems to work instead:

```julia
julia> function read_csv(path)
         @eval begin 
           using CSV, DataFrames
           return CSV.read($path, DataFrame)
         end
       end
read_csv (generic function with 1 method)

julia> @time read_csv("test.csv")
 17.529090 seconds (26.21 M allocations: 1.410 GiB, 5.87% gc time, 90.62% compilation time)
1×2 DataFrame
 Row │ a b    
     │ Int64 Int64 
─────┼──────────────
   1 │ 1 2

julia> @time read_csv("test.csv")
  0.000993 seconds (453 allocations: 37.328 KiB)
1×2 DataFrame
 Row │ a b    
     │ Int64 Int64 
─────┼──────────────
   1 │ 1 2

```

Any counterindications?

---

<div class="post-metadata">

### Author: ![heliosdrm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/heliosdrm/32/3851_2.png) [@heliosdrm](https://discourse.julialang.org/u/heliosdrm)
#### Post date: [May 7, 2022, 1:54pm UTC](https://discourse.julialang.org/t/import-a-module-only-when-needed/80654/3 "2022-05-07T13:54:37Z")

</div>

Calling `eval` (or the macro `@eval`) inside a function is not generally a good idea. The code evaluated that way is executed as a “top-level” expression. That means that: (1) it is not compiled as usual code in a function, so it has poor performance, and (2) it changes the global state of the module.

The last “problem” is actually what you want… or perhaps not… By `using CSV, DataFrames`, you are also exposing all their exported objects on the global scope, and that might cause conflicts. For that kind of things is better let the user be conscious of what packages are being used (as with Requires.jl)

---

<div class="post-metadata">

### Author: ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)
#### Post date: [May 7, 2022, 2:00pm UTC](https://discourse.julialang.org/t/import-a-module-only-when-needed/80654/4 "2022-05-07T14:00:58Z")

</div>

I often do not recommend abusing `eval` but this seems relatively harmless. Just be very aware that your code inside `@eval begin ... end` always run in **global** scope. So avoid doing anything that would pollute it (like creating new variables), and also remember that CSV possibly will not be available for other functions called after unless: (a) they also use `eval`; (b) they employ `invokelatest`; (c) the other functions are called only after the current call stack return to the global scope and continues from there.

---

<div class="post-metadata">

### Author: ![heliosdrm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/heliosdrm/32/3851_2.png) [@heliosdrm](https://discourse.julialang.org/u/heliosdrm)
#### Post date: [May 7, 2022, 2:12pm UTC](https://discourse.julialang.org/t/import-a-module-only-when-needed/80654/5 "2022-05-07T14:12:13Z")

</div>

Another thing to consider: you comment that your purpose of “lazy using” in to reduce latency, but that only happens if those packages are not actually used. Otherwise you are just delaying the “problem”, and perhaps making it worse. As commented in this nice article, one of the important sources of latency is method invalidation, and it is advised that:

> [@](#):
>
> You can minimize some of the costs of invalidation by loading all packages at the outset–if all invalidations happen before you start compiling very much code, then the first compilation already takes the full suite of methods into account.

> **[Analyzing sources of compiler latency in Julia: method invalidations](https://julialang.org/blog/2020/08/invalidations/)**
>
> Julia runs fast, but suffers from latency due to compilation. This post analyzes one source of excess compilation, tools for detecting and eliminating its causes, and the impact this effort has had on latency.

---

<div class="post-metadata">

### Author: ![CarloLucibello](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carlolucibello/32/3278_2.png) [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)
#### Post date: [May 8, 2022, 11:34am UTC](https://discourse.julialang.org/t/import-a-module-only-when-needed/80654/6 "2022-05-08T11:34:04Z")

</div>

My specific use case is for MLDatasets.jl where a single user most likely doesn’t need a large part of the I/O packages imported by the library.

Related issue [using MLDatasets is very slow · Issue #126 · JuliaML/MLDatasets.jl · GitHub](https://github.com/JuliaML/MLDatasets.jl/issues/126). In particular, the comments pointing at FileIO/ImageIO conditionally loading mechanism.
