# Trying to develop an extension

**URL:** <https://discourse.julialang.org/t/trying-to-develop-an-extension/106921>\
**Category:** New to Julia\
**Created:** [November 30, 2023, 2:02am UTC](https://discourse.julialang.org/t/trying-to-develop-an-extension/106921 "2023-11-30T02:02:57Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![feribg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/feribg/32/202948_2.png) [@feribg](https://discourse.julialang.org/u/feribg)\
**Post date:** [November 30, 2023, 2:02am UTC](https://discourse.julialang.org/t/trying-to-develop-an-extension/106921/1 "2023-11-30T02:02:57Z")

</div>

I find the process of developing an extension and all the steps extremely puzzling.  
So far i have

these files:  
– Package.jk  
– – Utils.jl

Package is as follows:

```julia
module Package

include("Utils.jl")
export Utils.MarsagliaRng, Utils.next_u32, Utils.rand

end

```

and Utils as follows:

> **[using Datesusing Randommodule Utils# Marsaglia Random Number Generator -...](https://pastebin.com/JvKK9Mns)**
>
> Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

then runtests.jl is

```julia
using TestItemRunner

@run_package_tests

```

and UtilsTests.jl:

```julia
using Test
using Rndmwlk
using Rndmwlk.Utils.MarsagliaRng
using Rndmwlk.Utils.next_u32
using Rndmwlk.Utils.rand

@testitem "MarsagliaRng_nextu32" begin
    # Test for specific seed and expected results
    rng = MarsagliaRng(UInt8.([77, 0, 0, 0])) # Seed = 77
    @test next_u32(rng) == 863531084
    @test next_u32(rng) == 2082884757
    @test next_u32(rng) == 788188464
    @test next_u32(rng) == 951323774
    @test next_u32(rng) == 744852019

    # Reset seed and test again
    rng = MarsagliaRng(UInt8.([77, 0, 0, 0])) # Seed = 77
    @test next_u32(rng) == 863531084
    @test next_u32(rng) == 2082884757
    @test next_u32(rng) == 788188464
    @test next_u32(rng) == 951323774
    @test next_u32(rng) == 744852019

    # Test for seed 1
    rng = MarsagliaRng(UInt8.([1, 0, 0, 0])) # Seed = 1
    @test next_u32(rng) == 1304893148
    @test next_u32(rng) == 840426891
    @test next_u32(rng) == 2480088888
    @test next_u32(rng) == 2020221409
    @test next_u32(rng) == 2766137397
end

@tesitem "Uniform Distribution Random Number Generation Tests" begin
    #Uniform distribution with seed 33
    rng = MarsagliaRng(UInt8.([33, 0, 0, 0])) # Seed = 33
    @test isapprox(rand(rng), 0.68160327; atol=1e-8)
    @test isapprox(rand(rng), 0.37011177; atol=1e-8)
    @test isapprox(rand(rng), 0.00732552; atol=1e-8)
    @test isapprox(rand(rng), 0.53339489; atol=1e-8)
    @test isapprox(rand(rng), 0.79831550; atol=1e-8)

    #Uniform distribution with seed 1
    rng = MarsagliaRng(UInt8.([1, 0, 0, 0])) # Seed = 1
    @test isapprox(rand(rng), 0.30381911; atol=1e-8)
    @test isapprox(rand(rng), 0.19567713; atol=1e-8)
    @test isapprox(rand(rng), 0.57744069; atol=1e-8)
    @test isapprox(rand(rng), 0.47036945; atol=1e-8)
    @test isapprox(rand(rng), 0.64404155; atol=1e-8)

    #Repeating tests for seed 33 to ensure repeatability
    rng = MarsagliaRng(UInt8.([33, 0, 0, 0])) # Seed = 33, repeat to ensure repeatability
    @test isapprox(rand(rng), 0.68160327; atol=1e-8)
    @test isapprox(rand(rng), 0.37011177; atol=1e-8)
    @test isapprox(rand(rng), 0.00732552; atol=1e-8)
    @test isapprox(rand(rng), 0.53339489; atol=1e-8)
    @test isapprox(rand(rng), 0.79831550; atol=1e-8)

end

```

resulting in the most cryptic error message ever:  
`ERROR: LoadError: syntax: "module" at Package.jl:1 expected "end", got "."`

So far its been a few hours trying to understand what’s the differnece between `actIVATE .` and `dev .` how it all plays out with VsCode, VsCode’s test runner vs Julia native and i think i almost have it in some kind of working state but this error makes no sense. Original package name was `rndmwlk` so there might be some refs to it…

Also is there any way to get around having to export every single thing twice or some kind of public private macro? I dont see any value of not having the visibility defined where the symbol is defined but instead having to double export. FYI if i dont export from Package.jl using the FQN in the tests still results in missing symbol…

---

<div class="post-metadata">

**Author:** ![filchristou](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/filchristou/32/26760_2.png) [@filchristou](https://discourse.julialang.org/u/filchristou)\
**Post date:** [November 30, 2023, 7:49am UTC](https://discourse.julialang.org/t/trying-to-develop-an-extension/106921/2 "2023-11-30T07:49:19Z")

</div>

Just some random thoughts here and there.  
Firstly, the correct term for your situation is “subpackage” instead of extension or  
[weak dependencies](https://pkgdocs.julialang.org/v1/creating-packages/#Weak-dependencies).  
Subpackages are not super popular and they could get pretty complex too, so I would suggest you to follow such an architecture only if you are sure about it. Usually flatting everything in a single package works better.

> `ERROR: LoadError: syntax: "module" at Package.jl:1 expected "end", got "."`

Probably you got a syntax error and that there is not a `module ... end` pair.  
I am not sure where the problem is, but I am confident if you investigate your code a bit you will track this done. Is more information in the error message as per the location ? I usually comment large chunks of code to debug such syntax errors.  
Also, keep in mind that `include` doesn’t do any magic and just concatenates text files; so it’s possible that the error lies inside `Utils.jl`

If you want to use some dependencies like `using Dates` in inside a module `Mod`, the the `using Dates` must be inside, i.e. after `module Mod`.

> what’s the differnece between `actIVATE .` and `dev .`

`activate` fires up up a local environment. The default environment is the global.  
`develop` adds a local dependency to the already started environment.

Find more in the docs

```julia
julia> import Pkg
help?> Pkg.activate
...
help?> Pkg.develop
...

```

> Also is there any way to get around having to export every single thing twice

Have a look at Reexport.jl

Finally, I would suggest to get your priorities straight. It seems that your `Package` doesn’t even precompile. So, before going on doing the tests, make sure that at least `Package` precompiles successfully (e.g. with `Pkg.precompile`).  
Also I don’t know why you keep `Rndmwlk` around in the tests, if that was the old name to your package.

---

<div class="post-metadata">

**Author:** ![feribg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/feribg/32/202948_2.png) [@feribg](https://discourse.julialang.org/u/feribg)\
**Post date:** [November 30, 2023, 4:34pm UTC](https://discourse.julialang.org/t/trying-to-develop-an-extension/106921/3 "2023-11-30T16:34:38Z")

</div>

We can ignore the package naming for a second, It’s originally called Rndmwlk.jl, but thats not the main issue.

For example if i open a new julia shell and i do include(“src/Utils.jl”) it works just fine. There’s no syntax warnings in that file (shared in the pastebin) above from the Julia language server.

Is there any better way to include files that actually runs compiler checks and its not just a text append, as well as how come language server and including the file in the repl work just fine like so:

```julia
julia> include("src/Utils.jl")
Main.Utils

julia> next
nextfloat nextind nextpow nextprod
julia> next
nextfloat nextind nextpow nextprod
julia> Utils.next_u32
next_u32 (generic function with 1 method)

julia> 

```

but this fails with compiler error:

```julia
module Rndmwlk

include("Utils.jl")
export Utils.MarsagliaRng, Utils.next_u32, Utils.rand

end

```

```julia
Failed to precompile Rndmwlk [36f4a8c9-f51c-45d8-8f56-ad14a47f3803] to "/home/feras/.julia/compiled/v1.9/Rndmwlk/jl_1nVwUe".
ERROR: LoadError: syntax: "module" at /home/Rndmwlk/src/Rndmwlk.jl:1 expected "end", got "."
Stacktrace:
 [1] top-level scope

```

the fact that this is so cryptic and so trivial makes me wonder what happens if this is the standard way of organizing things and i now have 10 submodules and multiple files flying around, it will be a colossal mess if there’s a typo somewhere, i struggle to believe this is the default way and if it is i think its bad enough to make me less excited about giving Julia a spin frankly ? Hope its not, since the other bits are quite neat.

Utils.jl:

> **[using Datesusing Randommodule Utils# Marsaglia Random Number Generator -...](https://pastebin.com/JvKK9Mns)**
>
> Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

0 issues in that file as reported by JLS:

 ![2023-11-30_11-36](https://global.discourse-cdn.com/julialang/original/3X/2/f/2f105c898dbfb3658c51d6a31aaccc556a1f1154.png)

I also manually concatenated the files like so:

```julia
module Rndmwlk

using Dates
using Random

module Utils

# Marsaglia Random Number Generator struct equivalent
mutable struct MarsagliaRng
    q::Vector{UInt32}
    carry::UInt32
    mwc256_initialized::Bool
    mwc256_seed::Int32
    i::UInt8

    function MarsagliaRng(seed::Vector{UInt8})
        q = zeros(UInt32, 256)
        carry = 362436
        mwc256_initialized = false
        mwc256_seed = reinterpret(Int32, seed)[1]
        i = 255
        new(q, carry, mwc256_initialized, mwc256_seed, i)
    end
end

# Default constructor using system time for seeding
function MarsagliaRng()
    ts_nano = UInt8(nanosecond(now()))
    seed = [ts_nano, ts_nano >>> 8, ts_nano >>> 16, ts_nano >>> 24]
    MarsagliaRng(seed)
end

# Random number generator functions
function next_u32(rng::MarsagliaRng)::UInt32
    a = UInt64(809430660)

    if !rng.mwc256_initialized
        j = UInt32(rng.mwc256_seed)
        c1 = UInt32(69069)
        c2 = UInt32(12345)
        rng.mwc256_initialized = true
        for k in 1:256
            j = (c1 * j + c2) % UInt32
            rng.q[k] = j
        end
    end

    rng.i = (rng.i + 1) % 256
    t = a * UInt64(rng.q[rng.i+1]) + UInt64(rng.carry)
    rng.carry = (t >> 32)
    rng.q[rng.i+1] = t % UInt32

    return rng.q[rng.i+1]
end

function next_u64(rng::MarsagliaRng)::UInt64
    UInt64(next_u32(rng))
end

# Sample function for generating a Float64
function rand(rng::MarsagliaRng)::Float64
    mult = 1.0 / typemax(UInt32)
    mult * next_u32(rng)
end

export MarsagliaRg, next_u32, rand

end

export Utils.MarsagliaRng

end

```

and it seems the error comes from:  
`export Utils.MarsagliaRng` which is valid syntax and the recommended way of doing it. The error message saying error on line 1 is weird, why ? 😅

---

<div class="post-metadata">

**Author:** ![feribg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/feribg/32/202948_2.png) [@feribg](https://discourse.julialang.org/u/feribg)\
**Post date:** [November 30, 2023, 4:54pm UTC](https://discourse.julialang.org/t/trying-to-develop-an-extension/106921/4 "2023-11-30T16:54:33Z")

</div>

Ok thanks ChatGPT:

```julia
Nested Modules: You have a nested module structure where Utils is a module inside the Rndmwlk module. This is perfectly fine in Julia, but it does require careful handling of scope and export statements.

Export Statement: The line export Utils.MarsagliaRng at the end of your outer module Rndmwlk is intended to export MarsagliaRng from the Utils module. However, Julia's module system doesn't directly support exporting a name from a nested module in this way. That's why you are getting the error.

Error Message: The error message syntax: "module" at ... expected "end", got "." is a bit misleading. It suggests a syntax error related to the module keyword, but the real issue is how the export statement is used.

To achieve your intended functionality (i.e., making MarsagliaRng accessible from outside the Rndmwlk module), you need to adjust your approach:

Option 1: Export within the Nested Module: Inside the Utils module, you can use export MarsagliaRng. This will make MarsagliaRng available when you use Utils elsewhere.

Option 2: Qualified Export: In the outer module Rndmwlk, instead of export Utils.MarsagliaRng, you can just use export Utils. This will make the Utils module available outside of Rndmwlk, and you can access MarsagliaRng via Utils.MarsagliaRng.

Option 3: Using Statement: Another approach is to use a using statement. Inside Rndmwlk, after defining Utils, you can add using .Utils: MarsagliaRng. This will bring MarsagliaRng into the scope of Rndmwlk, and then you can use export MarsagliaRng.

```

This is okay now the tests somewhat work but i have to do this every time:

```julia
using Rndmwlk.Utils

@testitem "MarsagliaRng_nextu32" begin
    # Test for specific seed and expected results
    rng = Utils.MarsagliaRng(UInt8.([77, 0, 0, 0])) # Seed = 77

```

and i cant find a way to bring all of Utils in scope a la Python from Utils import \*. the only solution I find is Option 3 recomended by ChatGPT but thats not great as that makes it available for all users of the package. I want to qualify it as Utils.func sometimes and sometimes `using Utils.func`

Is there a way to accomplish that ? For example the recommended:  
`using Rndmwlk.Utils: MarsagliaRng, next_u32, rand` doesn’t work. Function not defined error…

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [November 30, 2023, 6:03pm UTC](https://discourse.julialang.org/t/trying-to-develop-an-extension/106921/5 "2023-11-30T18:03:36Z")

</div>

I think you either want to just `export Utils` or from outside the package do `using Package: Utils`.

# Export example

```julia-repl
julia> module Package
           export Utils
           module Utils
               const tau = 2π
           end
       end
Main.Package

julia> using .Package

julia> Utils.tau
6.283185307179586

```

# using Example

```julia-repl
julia> module PackageB
           module UtilsB
               const tau = 2π
           end
       end
Main.PackageB

julia> using .PackageB

julia> UtilsB
ERROR: UndefVarError: `UtilsB` not defined

julia> using .PackageB: UtilsB

julia> UtilsB.tau
6.283185307179586

```

---

<div class="post-metadata">

**Author:** ![feribg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/feribg/32/202948_2.png) [@feribg](https://discourse.julialang.org/u/feribg)\
**Post date:** [November 30, 2023, 6:55pm UTC](https://discourse.julialang.org/t/trying-to-develop-an-extension/106921/6 "2023-11-30T18:55:07Z")

</div>

My issue is mostly how to:

```julia
using Test
using Rndmwlk.Utils

@testitem "MarsagliaRng_nextu32" begin
    # Test for specific seed and expected results
    rng = Utils.MarsagliaRng(UInt8.([77, 0, 0, 0])) # Seed = 77
    @test Utils.next_u32(rng) == 863531084
    @test Utils.next_u32(rng) == 2082884757

```

Remove the `Utils.next_u32` and make it just `next_u32` in this file ONLY, without having to do using `.Utils.next_u32` in the root module and exporting it as next\_u32 everywhere. So in a sense just a shorthand alias for `Utils.*` for this file only to avoid boilerplate and improve readibility.