# "Julian" way to include external scripts/modules without name conflicts

**URL:** <https://discourse.julialang.org/t/julian-way-to-include-external-scripts-modules-without-name-conflicts/97023>\
**Category:** General Usage\
**Tags:** question\
**Created:** [April 3, 2023, 4:55pm UTC](https://discourse.julialang.org/t/julian-way-to-include-external-scripts-modules-without-name-conflicts/97023 "2023-04-03T16:55:58Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Deduction42](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/deduction42/32/9206_2.png) [@Deduction42](https://discourse.julialang.org/u/Deduction42)\
**Post date:** [April 3, 2023, 4:55pm UTC](https://discourse.julialang.org/t/julian-way-to-include-external-scripts-modules-without-name-conflicts/97023/1 "2023-04-03T16:55:58Z")

</div>

I’m trying to manage a set of machine-learning models, each with its own custom-defined data preprocessing pipeline to highlight important features. In a directory, I have one folder per model and within each folder, I have a serialized custom-built model called “model.jls”, and I have an associated preprocessing script in “preprocessing.jl”. The preprocessing file would look like

```julia
function preprocessing!(df::DataFrame)
    df[!,"tag1"] = engineered_feature_1(df)
    df[!,"tag2"] = engineered_feature_2(df)
    return df
end

function engineered_feature_1(df::DataFrame)
    #<some stuff here>
end

function engineered_feature_2(df::DataFrame)
    #<some other stuff here>
end

```

What I want to do is have something like

```julia
mymodel = (
   model = deserialize(joinpath(modelfolder, "model.jls")),
   preprocessing = preprocessing_script(modelname, modelpath)
)

```

Now there would be a considerable number of “preprocessing.jl” files, which all define the “preprocessing function” and would potentiallly also have naming collisions with helper functions. What is the cleanest, most “Julian” way to get the contents from my “preprocessing.jl” file into this named tuple without potential naming conflicts?

---

<div class="post-metadata">

**Author:** ![Deduction42](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/deduction42/32/9206_2.png) [@Deduction42](https://discourse.julialang.org/u/Deduction42)\
**Post date:** [April 3, 2023, 5:21pm UTC](https://discourse.julialang.org/t/julian-way-to-include-external-scripts-modules-without-name-conflicts/97023/2 "2023-04-03T17:21:58Z")

</div>

For example, I could programatically create a module inside the “preprocessing\_script” function

```julia
function preprocessing_script(modelname, modelpath)
    module_name = "Preprocessing_$(modelname)"
    module_command = "
    module $(module_name)
    include(joinpath($(modelpath), \"preprocessing.jl\")
    end
    "
    include_string(module_command)
    return eval(Symbol(module_name)).preprocessing!
end

```

Is this the way I should be doing it? It would be nice to have a way to include the file as a programatically-named module, and extract the main function of interest, while having readable stack traces that you get when you “include” the file.

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [April 3, 2023, 5:52pm UTC](https://discourse.julialang.org/t/julian-way-to-include-external-scripts-modules-without-name-conflicts/97023/3 "2023-04-03T17:52:00Z")

</div>

Something like this might be a better pattern:

```julia
function get_preprocess_function(modelpath)
    # Create an anonymous module
    m = Module()

    # Include the model source file in it
    Base.include(m, joinpath(modelpath, "model.jl"))

    # Return the newly-defined `preprocess!` function
    return m.preprocess!
end

```

---

<div class="post-metadata">

**Author:** ![Deduction42](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/deduction42/32/9206_2.png) [@Deduction42](https://discourse.julialang.org/u/Deduction42)\
**Post date:** [April 3, 2023, 5:55pm UTC](https://discourse.julialang.org/t/julian-way-to-include-external-scripts-modules-without-name-conflicts/97023/4 "2023-04-03T17:55:49Z")

</div>

You can do that?!? This looks exactly like the kind of thing I was wanting to do!

---

<div class="post-metadata">

**Author:** ![jacobusmmsmit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jacobusmmsmit/32/217669_2.png) [@jacobusmmsmit](https://discourse.julialang.org/u/jacobusmmsmit)\
**Post date:** [April 3, 2023, 6:24pm UTC](https://discourse.julialang.org/t/julian-way-to-include-external-scripts-modules-without-name-conflicts/97023/5 "2023-04-03T18:24:37Z")

</div>

This is a bottom of the Julia iceberg tip if I ever saw one. Never would have thought of that, who would ever do `?Base.include`?

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [April 3, 2023, 6:37pm UTC](https://discourse.julialang.org/t/julian-way-to-include-external-scripts-modules-without-name-conflicts/97023/6 "2023-04-03T18:37:53Z")

</div>

I think I saw that here:

[https://docs.julialang.org/en/v1/manual/modules/#Default-top-level-definitions-and-bare-modules](https://docs.julialang.org/en/v1/manual/modules/#Default-top-level-definitions-and-bare-modules)

---

<div class="post-metadata">

**Author:** ![Deduction42](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/deduction42/32/9206_2.png) [@Deduction42](https://discourse.julialang.org/u/Deduction42)\
**Post date:** [April 13, 2023, 10:30pm UTC](https://discourse.julialang.org/t/julian-way-to-include-external-scripts-modules-without-name-conflicts/97023/7 "2023-04-13T22:30:32Z")

</div>

Okay, so this worked when I used it at the top level, but if I use this function inside a function I get

```julia
MethodError: no method matching preprocess!
The applicable method may be too new: running in world age 33212, while current world is 33213

```

---

<div class="post-metadata">

**Author:** ![Deduction42](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/deduction42/32/9206_2.png) [@Deduction42](https://discourse.julialang.org/u/Deduction42)\
**Post date:** [April 13, 2023, 10:57pm UTC](https://discourse.julialang.org/t/julian-way-to-include-external-scripts-modules-without-name-conflicts/97023/8 "2023-04-13T22:57:52Z")

</div>

I guess I can fix this if I use  
`Base.@invokelatest preprocess!(data)`  
where  
`preprocess! = get_preprocess_function`  
At least this is an in-place modification so there is no “unknown-type” output that breaks type inference.