How to input files as command line arguments to compiled Julia

General Usage
, , , ,
1

I am building a shippable app with Julia using the PackageCompiler.jl library.

I have followed the example they provided and replicated it properly.

I am now trying to modify the code piece by piece to replace the toy function with my own function. It involves reading a CSV file into a dataframe whose path is taken from the command line. For now, I am just trying to accomplish this task.

I have this code in MyApp.jl:

module MyApp

using Example
using HelloWorldC_jll
using Pkg.Artifacts
using DataFrames, CSV, Statistics  

fooifier_path() = joinpath(artifact"fooifier", "bin", "fooifier" * (Sys.iswindows() ? ".exe" : ""))

function julia_main()
    try
        real_main()
    catch
        Base.invokelatest(Base.display_error, Base.catch_stack())
        return 1
    end
    return 0
end

function real_main()
    # @show ARGS
    # @show Base.PROGRAM_FILE
    # @show DEPOT_PATH
    # @show LOAD_PATH
    # @show pwd()
    # @show Base.active_project()
    # @show Threads.nthreads()
    # @show Sys.BINDIR
    # display(Base.loaded_modules)
    for arg in ARGS
        println(arg)
    end
    println("this part worked!")


    df = CSV.read(string(ARGS[1]), DataFrame);
    println(df)
    return
end

if abspath(PROGRAM_FILE) == @__FILE__
    real_main()
end

I compile the code with:

using PackageCompiler;
create_app("MyApp", "MyAppCompiled")

and it compiles just fine.

I run it with:

julia MyAppCompiled/bin/MyApp <absolute path to csv>

and it works up until the dataframe portion, where I get this error:

it worked!
MyApp(66457,0x115997dc0) malloc: *** error for object 0x1117d9510: pointer being realloc'd was not allocated
MyApp(66457,0x115997dc0) malloc: *** set a breakpoint in malloc_error_break to debug

signal (6): Abort trap: 6
in expression starting at none:0
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 47484171 (Pool: 47468051; Big: 16120); GC: 49
Abort trap: 6

Can anyone help me figure out what I am doing wrong? I know I am putting in the proper path because if I incorrectly type the path I get the “ is not a proper file…” message. I’ve tried calling with relative and absolute paths.

2

Welcome @connormeaton

There are some inconsistencies in your description which make it difficult to check whats wrong:

Your output

it worked!

isn’t reflected in the code. There is a println("this part worked!"), but not clear if it is related.

The loop

    for arg in ARGS
        println(arg)
    end

seems not to generate any output but it should, isn’t it?

A precompiled app isn’t started with

julia MyAppCompiled/bin/MyApp <absolute path to csv>

but rather with something like:

MyAppCompiled/bin/MyApp.exe <absolute path to csv>

Could you provide your latest code example with the latest output after a clean first run?

1 Like
3

I am providing updated code and fresh output, per request. See below. Thank you!
Code:

module MyApp

using Example
using HelloWorldC_jll
using Pkg.Artifacts
using DataFrames, CSV, Statistics  

fooifier_path() = joinpath(artifact"fooifier", "bin", "fooifier" * (Sys.iswindows() ? ".exe" : ""))

function julia_main()
    try
        real_main()
    catch
        Base.invokelatest(Base.display_error, Base.catch_stack())
        return 1
    end
    return 0
end

function real_main()
    # @show ARGS
    # @show Base.PROGRAM_FILE
    # @show DEPOT_PATH
    # @show LOAD_PATH
    # @show pwd()
    # @show Base.active_project()
    # @show Threads.nthreads()
    # @show Sys.BINDIR
    # display(Base.loaded_modules)
    for arg in ARGS
        println(arg)
    end
    println("it worked!")


    df = CSV.read(string(ARGS[1]), DataFrame);
    println(df)
    return
end

if abspath(PROGRAM_FILE) == @__FILE__
    real_main()
end

Command:

$ ~/connor/juliaRppgPacakge$ MyAppCompiled2/bin/MyApp ~/connor/demoVideos/20210701_100807.csv

Output:

/Users/connor//demoVideos/20210701_100807.csv
it worked!
MyApp(28285,0x116fccdc0) malloc: *** error for object 0x10e0513a8: pointer being realloc'd was not allocated
MyApp(28285,0x116fccdc0) malloc: *** set a breakpoint in malloc_error_break to debug

signal (6): Abort trap: 6
in expression starting at none:0
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 45272431 (Pool: 45256526; Big: 15905); GC: 45
Abort trap: 6
4

Thank you for your response. I could not edit my post, so I placed the requested code in the reply below. You were right about some code inconsistencies. Hopefully this helps diagnose the issue. Also, I tried this MyAppCompiled/bin/MyApp.exe <absolute path to csv>, but I have no file with .exe.

5

It’s better not to edit, as the thread stays easier to follow for future readers. (IMO)

Yes, sorry, I am on Windows and somehow assumed you too.

I have one package which can be precompiled and there are some obvious differences. So, before I try to setup something like your code to reproduce the issue, let’s check if it is just those differences (see my next post):

  • in your module the end is missing:
module MyApp
...
end
  • my julia_main looks a bit different, could you try with:
Base.@ccallable function julia_main()::Cint
    try
        real_main()
    catch
        Base.invokelatest(Base.display_error, Base.catch_stack())
        return 1
    end
    return 0
end
1 Like
6

In the meantime I created a new test package.

My above suggestions are not the reason for your problem.

The code is nearly the same as yours:

module MyTestPkg

using DataFrames, CSV

using Pkg.Artifacts

fooifier_path() = joinpath(artifact"fooifier", "bin", "fooifier" * (Sys.iswindows() ? ".exe" : ""))

function julia_main()
    try
        real_main()
    catch
        Base.invokelatest(Base.display_error, Base.catch_stack())
        return 1
    end
    return 0
end

function real_main()
    for arg in ARGS
        println(arg)
    end
    println("it worked!")
    df = CSV.read(string(ARGS[1]), DataFrame);
    println(df)
    return
end

if abspath(PROGRAM_FILE) == @__FILE__
    real_main()
end

end

This is the build:

julia> using MyTestPkg
[ Info: Precompiling MyTestPkg [7b6ef1f2-e27a-4927-afbe-f37c5d2ce363]

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: AMD Ryzen 9 3900X 12-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, znver2)
Environment:
  JULIA_EDITOR = code
  JULIA_NUM_THREADS = 4

julia> using PackageCompiler

julia> create_app("..\\MyTestPkg", "MyTestPkgApp";force=true)
[ Info: PackageCompiler: creating base system image (incremental=false)...
[ Info: PackageCompiler: creating system image object file, this might take a while...

25 dependencies successfully precompiled in 29 seconds
[ Info: PackageCompiler: creating system image object file, this might take a while...

(MyTestPkg) pkg> st
     Project MyTestPkg v0.1.0
      Status `C:\Users\Oli\.julia\dev\MyTestPkg\Project.toml`
  [336ed68f] CSV v0.8.5
  [a93c6f00] DataFrames v1.2.0
  [9b87118b] PackageCompiler v1.2.7
  [44cfe95a] Pkg

This is the output when I run it:

c:\Users\Oli\.julia\dev\MyTestPkg\MyTestPkgApp\bin>.\MyTestPkg.exe test.txt
test.txt
it worked!
2Ă—3 DataFrame
 Row │ x      y      z
     │ Int64  Int64  Int64
─────┼─────────────────────
   1 │     1      2      3
   2 │     4      5      6

Well, the bad news, it just works.

I will check with other Julia versions. Checked with 1.5.3 and 1.6.1, all are working.
You may remove the other packages you are using

using Example
using HelloWorldC_jll

to make sure, that those don’t introduce something weird.

2 Likes
7

Thanks for the reply. I’m confused about this line below:

You do this before the “create_app”) call. I don’t see this anywhere in the docs. How does julia recognize MyTestPkg at this point? What directory are you in?

I’ve tried calling from multiple directory locations and I get this error:

julia> using MyApp
ERROR: ArgumentError: Package MyApp not found in current path:
- Run `import Pkg; Pkg.add("MyApp")` to install the MyApp package.

Stacktrace:
 [1] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:967
8

Also, what does your Manifest.toml file look like? Does this need to be edited from the original example? Thank you.

9

One more piece of information. I changed my code to match what you provided, removed unneeded packages, and reran. The only thing I kept is that the project is still called MyApp.

I was unable to do the first using MyApp call you specify due to the previous comment error I provided.

I am on Julia 1.8.0.

Here is my result:

julia> using PackageCompiler

julia> create_app("MyApp", "MyAppCompiled";force=true)
┌ Warning: The active manifest file is an older format with no julia version entry. Dependencies may have been resolved with a different julia version.
â”” @ ~/connor/MyApp/Manifest.toml:0
  0 dependencies successfully precompiled in 1 seconds (2 already precompiled)
  3 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages
[ Info: PackageCompiler: creating base system image (incremental=false)...



[ Info: PackageCompiler: creating system image object file, this might take a while...
┌ Warning: Inactive precompile statement
│   form = Tuple{typeof(Core.Compiler.ignorelimited), Any}
â”” @ nothing nothing:0
┌ Warning: Inactive precompile statement
│   form = Tuple{typeof(Core.Compiler.abstract_call_known), Core.Compiler.AbstractInterpreter, Any, Union{Nothing, Vector{Any}}, Vector{Any}, Core.Compiler.InferenceState, Int64}
â”” @ nothing nothing:0
┌ Warning: Inactive precompile statement
│   form = Tuple{typeof(Core.Compiler.widenconditional), Any}
â”” @ nothing nothing:0
┌ Warning: Inactive precompile statement
│   form = Tuple{typeof(Core.Compiler.widenwrappedconditional), Any}
â”” @ nothing nothing:0
┌ Warning: Inactive precompile statement
│   form = Tuple{typeof(Core.Compiler._uniontypes), Any, Vector{Any}}
â”” @ nothing nothing:0
┌ Warning: Inactive precompile statement
│   form = Tuple{typeof(Core.Compiler.unwrapva), Any}
â”” @ nothing nothing:0
┌ Warning: Inactive precompile statement
│   form = Tuple{typeof(Core.Compiler.union_count_abstract), Any}
â”” @ nothing nothing:0
┌ Warning: Inactive precompile statement
│   form = Tuple{typeof(Core.Compiler.methods), Any}
â”” @ nothing nothing:0
ld: warning: could not create compact unwind for _julia_Dict_15970: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _julia_Dict_15978: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _julia_Dict_15970.clone_1: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _julia_Dict_15978.clone_1: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _julia_Dict_15970.clone_1.clone_2: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _julia_Dict_15978.clone_1.clone_2: stack subq instruction is too different from dwarf stack size
  0 dependencies successfully precompiled in 0 seconds (2 already precompiled)
  3 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages
┌ Warning: The active manifest file is an older format with no julia version entry. Dependencies may have been resolved with a different julia version.
â”” @ ~/connor/MyApp/Manifest.toml:0
Precompiling project...
  5 dependencies successfully precompiled in 69 seconds
[ Info: PackageCompiler: creating system image object file, this might take a while...
┌ Warning: Package MyApp does not have DataFrames in its dependencies:
│ - If you have MyApp checked out for development and have
│   added DataFrames as a dependency but haven't updated your primary
│   environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with MyApp
│ Loading DataFrames into MyApp from project dependency, future warnings for MyApp are suppressed.
â”” @ nothing nothing:984
ld: warning: could not create compact unwind for _julia_Dict_16356: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _julia_Dict_16364: stack subq instruction is too different from dwarf stack size
/Users/connor/.julia/packages/PackageCompiler/BjVzm/src/embedding_wrapper.c:18:23: error: expected
      identifier or '('
JULIA_DEFINE_FAST_TLS()
                      ^
/Users/connor/.julia/packages/PackageCompiler/BjVzm/src/embedding_wrapper.c:67:19: warning: implicit
      declaration of function 'julia_main' is invalid in C99 [-Wimplicit-function-declaration]
    int retcode = julia_main(ARGS);
                  ^
1 warning and 1 error generated.
ERROR: failed process: Process(`gcc '-DJULIAC_PROGRAM_LIBNAME="MyApp.dylib"' -m64 -march=x86-64 -o MyApp /Users/connor/.julia/packages/PackageCompiler/BjVzm/src/embedding_wrapper.c MyApp.dylib -O2 -Wl,-rpath,@executable_path -Wl,-rpath,@executable_path/../lib -Wl,-rpath,@executable_path/../lib/julia -std=gnu99 -I/usr/local/Cellar/julia/HEAD-5a16805_1/include/julia -fPIC -L/usr/local/Cellar/julia/HEAD-5a16805_1/lib -L/usr/local/Cellar/julia/HEAD-5a16805_1/lib/julia -ljulia -ljulia-internal`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error
   @ ./process.jl:531 [inlined]
 [2] run(::Cmd; wait::Bool)
   @ Base ./process.jl:446
 [3] run
   @ ./process.jl:444 [inlined]
 [4] run_with_env
   @ ~/.julia/packages/PackageCompiler/BjVzm/src/PackageCompiler.jl:104 [inlined]
 [5] create_executable_from_sysimg(; sysimage_path::String, executable_path::String, c_driver_program_path::String)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/BjVzm/src/PackageCompiler.jl:735
 [6] (::PackageCompiler.var"#9#10"{Vector{String}, Vector{String}, Bool, Bool, String, String, String, String, String})()
   @ PackageCompiler ~/.julia/packages/PackageCompiler/BjVzm/src/PackageCompiler.jl:697
 [7] cd(f::PackageCompiler.var"#9#10"{Vector{String}, Vector{String}, Bool, Bool, String, String, String, String, String}, dir::String)
   @ Base.Filesystem ./file.jl:110
 [8] create_app(package_dir::String, app_dir::String; app_name::Nothing, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, audit::Bool, force::Bool, c_driver_program::String, cpu_target::String)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/BjVzm/src/PackageCompiler.jl:671
 [9] top-level scope
   @ REPL[2]:1

Are there any glaring errors I am making? Thank you.

10

I don’t think this is really necessary. I got somewhere a precompile error and starting with this to precompile just worked, so I did it by routine during my checks. If your compile goes well I expect all to be fine.

I didn’t edited the Manifest.toml or the Project.toml. But I had to add a Artifacts.toml with a minimal entry for the artifact you are using.

I created my test project with:

julia> using PkgTemplates

julia> t = Template(;user="oheil")

julia> t("MyTestPkg")

which creates the package in .julia/dev/MyTestPkg with all the needed files (except Artifacts.toml of course). So perhaps this is some cause of your problem, because perhaps some dependencies aren’t properly included in your image. That’s why I have also the activate step, so that all needed dependencies are clean and complete. You have to this before you add the packages you need in your module, e.g.:

(@v1.6) pkg> activate .julia\\dev\\MyTestPkg

(MyTestPkg) pkg> add DataFrames,CSV

This adds the dependencies to the Project.toml.

At least it is called “PackageCompiler”, not “ModuleCompiler”. So, it may be, that you need a proper package (local) and not only a module, but this I don’t know.

Your last post says, that you are not able to create the precompiled application. What has changed? How was your previous one built? If you are moving the target issue, I am not able to help you, as I am always running after new and changed informations which contradict what you wrote before.

So, as it is now difficult to say, at what kind of starting point you really are, I would suggest, that you start with a new package created with PkgTemplate. Activate the new environment and add the minimal set of packages. Copy/Paste the minimal code example you have into the new package and start over.

In the mean time I will try the same for 1.8.

1 Like
11

No, not for me.

1 Like
12

With 1.8 I don’t get a executable with similar errors and warnings like you have.
As this is a developer/unstable build, I see this as futile to try to make this work, just use a stable release.

1 Like
13

Okay, thank you. I am working through your suggestions with PkgTemplate and making progress. I will reinstall stable Julia 1.6 again. Will update. Thanks.

14

Okay, here is what I did, step by step:

$ brew uninstall julia
$ brew install julia
$ julia

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.2 (2021-07-14)
 _/ |\__'_|_|_|\__'_|  |  Built by Homebrew (v1.6.2_1)
|__/                   |


julia> import Pkg
julia> Pkg.add("PkgTemplates")
julia> Pkg.add("CSV")
julia> Pkg.add("DataFrames")
julia> using PkgTemplates

julia> t = Template(;user="connormeaton")
Template:
  authors: ["Connor and contributors"]
  dir: "~/.julia/dev"
  host: "github.com"
  julia: v"1.0.0"
  user: "connormeaton"
  plugins:
    CompatHelper:
      file: "~/.julia/packages/PkgTemplates/2jJdD/templates/github/workflows/CompatHelper.yml"
      destination: "CompatHelper.yml"
      cron: "0 0 * * *"
    Git:
      ignore: String[]
      name: nothing
      email: nothing
      branch: nothing
      ssh: false
      jl: true
      manifest: false
      gpgsign: false
    License:
      path: "~/.julia/packages/PkgTemplates/2jJdD/templates/licenses/MIT"
      destination: "LICENSE"
    ProjectFile:
      version: v"0.1.0"
    Readme:
      file: "~/.julia/packages/PkgTemplates/2jJdD/templates/README.md"
      destination: "README.md"
      inline_badges: false
      badge_order: DataType[Documenter{GitHubActions}, Documenter{GitLabCI}, Documenter{TravisCI}, GitHubActions, GitLabCI, TravisCI, AppVeyor, DroneCI, CirrusCI, Codecov, Coveralls, BlueStyleBadge, ColPracBadge]
      badge_off: DataType[]
    SrcDir:
      file: "~/.julia/packages/PkgTemplates/2jJdD/templates/src/module.jl"
    TagBot:
      file: "~/.julia/packages/PkgTemplates/2jJdD/templates/github/workflows/TagBot.yml"
      destination: "TagBot.yml"
      trigger: "JuliaTagBot"
      token: Secret("GITHUB_TOKEN")
      ssh: Secret("DOCUMENTER_KEY")
      ssh_password: nothing
      changelog: nothing
      changelog_ignore: nothing
      gpg: nothing
      gpg_password: nothing
      registry: nothing
      branches: nothing
      dispatch: nothing
      dispatch_delay: nothing
    Tests:
      file: "~/.julia/packages/PkgTemplates/2jJdD/templates/test/runtests.jl"
      project: false

julia> t("MyTestPkg")
[ Info: Running prehooks
[ Info: Running hooks
  Activating environment at `~/.julia/dev/MyTestPkg/Project.toml`
    Updating registry at `~/.julia/registries/General`
  No Changes to `~/.julia/dev/MyTestPkg/Project.toml`
  No Changes to `~/.julia/dev/MyTestPkg/Manifest.toml`
Precompiling project...
  1 dependency successfully precompiled in 1 seconds
  Activating environment at `~/.julia/environments/v1.6/Project.toml`
[ Info: Running posthooks
[ Info: New package is at /Users/connor/.julia/dev/MyTestPkg

(@v1.6) pkg> activate .julia\\dev\\MyTestPkg
  Activating environment at `~/connor/juliaRppgPacakge/.julia\\dev\\MyTestPkg/Project.toml`

(.julia\\dev\\MyTestPkg) pkg> add DataFrames,CSV
   Resolving package versions...
  No Changes to `~/connor/juliaRppgPacakge/.julia\\dev\\MyTestPkg/Project.toml`
    Updating `~/connor/juliaRppgPacakge/.julia\\dev\\MyTestPkg/Manifest.toml`
  [692b3bcd] - JLLWrappers v1.3.0
  [21216c6a] - Preferences v1.2.2
  [8e850b90] - libblastrampoline_jll v3.0.4+0
  [0dad84c5] ~ ArgTools v1.1.1 ⇒ 
  [f43a241f] ~ Downloads v1.4.0 ⇒ 
  [b27032c2] ~ LibCURL v0.6.2 ⇒ 
  [ca575930] ~ NetworkOptions v1.2.0 ⇒ 
  [44cfe95a] ~ Pkg v1.6.0 ⇒ 
  [fa267f1f] ~ TOML v1.0.0 ⇒ 
  [a4e569a6] ~ Tar v1.9.1 ⇒ 
  [deac9b47] ~ LibCURL_jll v7.73.0+4 ⇒ 
  [29816b5a] ~ LibSSH2_jll v1.9.1+2 ⇒ 
  [c8ffd9c3] ~ MbedTLS_jll v2.24.0+2 ⇒ 
  [14a3606d] ~ MozillaCACerts_jll v2020.7.22 ⇒ 
  [83775a58] ~ Zlib_jll v1.2.12+1 ⇒ 
  [8e850ede] ~ nghttp2_jll v1.41.0+1 ⇒ 
  [3f19e933] ~ p7zip_jll v16.2.1+1 ⇒

This all worked well. Here I come across my first error:

julia> using MyTestPkg
ERROR: ArgumentError: Package MyTestPkg not found in current path:
- Run `import Pkg; Pkg.add("MyTestPkg")` to install the MyTestPkg package.

So I thought moving to the right path would help …

julia> pwd()
"/Users/connor/Testing"

julia> cd("../../.julia/dev/MyTestPkg/")

julia> pwd()
"/Users/connor/.julia/dev/MyTestPkg"

julia> using MyTestPkg
ERROR: ArgumentError: Package MyTestPkg not found in current path:
- Run `import Pkg; Pkg.add("MyTestPkg")` to install the MyTestPkg package.

Stacktrace:
 [1] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:893

As you can see, I still get an error on this command. Alas, I just moved on. Maybe this is the cause of the issue?

julia> Pkg.add("PackageCompiler")
   Resolving package versions...
    Updating `~/connor/juliaRppgPacakge/.julia\\dev\\MyTestPkg/Project.toml`
  [9b87118b] + PackageCompiler v1.2.7
    Updating `~/connor/juliaRppgPacakge/.julia\\dev\\MyTestPkg/Manifest.toml`
  [9b87118b] + PackageCompiler v1.2.7
Precompiling project...
  1 dependency successfully precompiled in 1 seconds (23 already precompiled)

julia> using PackageCompiler
julia> create_app("../MyTestPkg", "MyTestPkgApp";force=true)
┌ Warning: it is not recommended to create an app without a preexisting manifest
â”” @ PackageCompiler ~/.julia/packages/PackageCompiler/BjVzm/src/PackageCompiler.jl:642
  0 dependencies successfully precompiled in 0 seconds
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package
[ Info: PackageCompiler: creating base system image (incremental=false)...
error during bootstrap:
LoadError(at "/usr/local/Cellar/julia/1.6.2_1/share/julia/base/compiler/compiler.jl" line 3: LoadError(at "compiler/bootstrap.jl" line 8: InterruptException()))
jl_typeinf_end at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
unknown function (ip: 0x1116f93ff)
jl_apply_generic at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
unknown function (ip: 0x1116f440e)
jl_toplevel_eval_flex at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_parse_eval_all at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_load_ at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
unknown function (ip: 0x1116c4a54)
jl_apply_generic at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
unknown function (ip: 0x1116c4925)
jl_apply_generic at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
do_call at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
eval_body at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_interpret_toplevel_thunk at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
top-level scope at /usr/local/Cellar/julia/1.6.2_1/share/julia/base/compiler/compiler.jl:125
jl_toplevel_eval_flex at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_toplevel_eval_flex at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_toplevel_eval_in at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
unknown function (ip: 0x1116c3b94)
jl_apply_generic at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
do_call at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
eval_body at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_interpret_toplevel_thunk at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
top-level scope at /usr/local/Cellar/julia/1.6.2_1/share/julia/base/compiler/compiler.jl:3
jl_toplevel_eval_flex at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_parse_eval_all at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_load_ at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_load at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
exec_program at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
true_main at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)
repl_entrypoint at /usr/local/Cellar/julia/1.6.2_1/lib/julia/libjulia-internal.1.6.dylib (unknown line)

[ Info: PackageCompiler: creating system image object file, this might take a while...
  0 dependencies successfully precompiled in 0 seconds
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package
Precompiling project...
  âś— MyTestPkg
  0 dependencies successfully precompiled in 3 seconds

ERROR: The following 1 dependency failed to precompile:

MyTestPkg [b1693a64-f60c-4f39-94a4-785f7c40246b]

Failed to precompile MyTestPkg [b1693a64-f60c-4f39-94a4-785f7c40246b] to /Users/connor/.julia/compiled/v1.6/MyTestPkg/jl_qKnc4T.
ERROR: LoadError: syntax: incomplete: "module" at /Users/connor/.julia/dev/MyTestPkg/src/MyTestPkg.jl:1 requires end
Stacktrace:
 [1] top-level scope
   @ ~/.julia/dev/MyTestPkg/src/MyTestPkg.jl:1
 [2] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
   @ Base /private/var/folders/rg/r1_lqdvn6rj3b3_m2292vmc80000gn/T/jl_4OUU9Y/tmp_sys.so:-1
 [3] _include(mapexpr::Function, mod::Module, _path::String)
   @ Base /private/var/folders/rg/r1_lqdvn6rj3b3_m2292vmc80000gn/T/jl_4OUU9Y/tmp_sys.so:-1
 [4] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
   @ Base /private/var/folders/rg/r1_lqdvn6rj3b3_m2292vmc80000gn/T/jl_4OUU9Y/tmp_sys.so:-1
 [5] top-level scope
   @ none:1
 [6] eval
   @ ./boot.jl:360 [inlined]
in expression starting at /Users/connor/.julia/dev/MyTestPkg/src/MyTestPkg.jl:1
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types /usr/local/Cellar/julia/1.6.2_1/share/julia/stdlib/v1.6/Pkg/src/Types.jl:55
 [2] precompile(ctx::Pkg.Types.Context; internal_call::Bool, strict::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API /usr/local/Cellar/julia/1.6.2_1/share/julia/stdlib/v1.6/Pkg/src/API.jl:1265
 [3] #precompile#196
   @ /usr/local/Cellar/julia/1.6.2_1/share/julia/stdlib/v1.6/Pkg/src/API.jl:919 [inlined]
 [4] top-level scope
   @ none:1
 [5] exec_options(opts::Base.JLOptions)
   @ Base /private/var/folders/rg/r1_lqdvn6rj3b3_m2292vmc80000gn/T/jl_4OUU9Y/tmp_sys.so:-1
 [6] _start()
   @ Base /private/var/folders/rg/r1_lqdvn6rj3b3_m2292vmc80000gn/T/jl_4OUU9Y/tmp_sys.so:-1
ERROR: failed process: Process(`/usr/local/Cellar/julia/1.6.2_1/bin/julia --color=yes --startup-file=no --sysimage=/var/folders/rg/r1_lqdvn6rj3b3_m2292vmc80000gn/T/jl_4OUU9Y/tmp_sys.so --project=/Users/connor/.julia/dev/MyTestPkg -e 'using Pkg; Pkg.precompile(; strict=true)'`, ProcessExited(1)) [1]

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:525 [inlined]
  [2] run(::Cmd; wait::Bool)
    @ Base ./process.jl:440
  [3] run
    @ ./process.jl:438 [inlined]
  [4] do_ensurecompiled
    @ ~/.julia/packages/PackageCompiler/BjVzm/src/PackageCompiler.jl:199 [inlined]
  [5] create_sysimg_object_file(object_file::String, packages::Vector{String}; project::String, base_sysimage::String, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, cpu_target::String, script::Nothing, isapp::Bool)
    @ PackageCompiler ~/.julia/packages/PackageCompiler/BjVzm/src/PackageCompiler.jl:270
  [6] create_sysimage(packages::Symbol; sysimage_path::String, project::String, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, replace_default::Bool, cpu_target::String, script::Nothing, base_sysimage::String, isapp::Bool)
    @ PackageCompiler ~/.julia/packages/PackageCompiler/BjVzm/src/PackageCompiler.jl:452
  [7] (::PackageCompiler.var"#9#10"{Vector{String}, Vector{String}, Bool, Bool, String, String, String, String, String})()
    @ PackageCompiler ~/.julia/packages/PackageCompiler/BjVzm/src/PackageCompiler.jl:682
  [8] cd(f::PackageCompiler.var"#9#10"{Vector{String}, Vector{String}, Bool, Bool, String, String, String, String, String}, dir::String)
    @ Base.Filesystem ./file.jl:106
  [9] create_app(package_dir::String, app_dir::String; app_name::Nothing, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, audit::Bool, force::Bool, c_driver_program::String, cpu_target::String)
    @ PackageCompiler ~/.julia/packages/PackageCompiler/BjVzm/src/PackageCompiler.jl:671
 [10] top-level scope
    @ REPL[33]:1

I ran Pkg.precompile() and tried again, and no difference.

Also, my MyTestPkg.jl code is the same as what you provided.

I think I’ve followed, or tried to follow every step you take from a fresh julia install onwards. Let me know if I am missing anything. Hopefully we can get this package to work and accept file path input. Thank you!

15

Instead of

can you try with

(@v1.6) pkg> activate ~/connor/juliaRppgPacakge/.julia/dev/MyTestPkg

It must be a valid path to the package folder. \\ are the separators on windows, you just need the single /.
If you start in

julia> pwd()
"/Users/connor/Testing"

You may also try with

(@v1.6) pkg> activate ../../.julia/dev/MyTestPkg

After activate autocompletion with TAB should work if these pathes are wrong.

If this is successful it should show as

(MyTestPkg) pkg>

and not as (e.g.)

(.julia\\dev\\MyTestPkg) pkg>

After that step successfull add DataFrames,CSV again, and do a using DataFrames,CSV on the REPL ( I am just unsure which one adds them to Project.toml ).

1 Like
16

Thank you, I was able to get to here:

I then successfully ran:

(MyTestPkg) pkg> add DataFrames,CSV

As suggested, I added the using ... command in the REPL:

julia> using DataFrames, CSV

Then I ran:

julia> using MyTestPkg
ERROR: ArgumentError: Package MyTestPkg not found in current path:
- Run `import Pkg; Pkg.add("MyTestPkg")` to install the MyTestPkg package.

Stacktrace:
 [1] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:893

For reference I am here:

julia> pwd()
"/Users/connor/juliaRppgPacakge"

What am I doing wrong?

17

Hmm, that’s weird
After activate can you check the status?

(MyTestPkg) pkg> st
     Project MyTestPkg v0.1.0
      Status `C:\Users\Oli\.julia\dev\MyTestPkg\Project.toml`
  [336ed68f] CSV v0.8.5
  [a93c6f00] DataFrames v1.2.0
  [9b87118b] PackageCompiler v1.2.7
  [44cfe95a] Pkg
1 Like
18

I get this, which looks right?

(MyTestPkg) pkg> st
      Status `~/Users/connor/juliaRppgPacakge/.julia/dev/MyTestPkg/Project.toml`
  [336ed68f] CSV v0.8.5
  [a93c6f00] DataFrames v1.2.0
  [9b87118b] PackageCompiler v1.2.7
  [44cfe95a] Pkg

Can you describe what you are doing immediately after this step? I am hitting Ctrl+C to get back to Julia REPL and then trying to call using MyTestPkg, where it fails. Are you doing something different at this step?

19

Well, you are missing the line

but I have no idea why.

It seems that in your file

~/Users/connor/juliaRppgPacakge/.julia/dev/MyTestPkg/Project.toml

the following lines (or some of them) at the beginning are missing:

name = "MyTestPkg"
uuid = "7b6ef1f2-e27a-4927-afbe-f37c5d2ce363"
authors = ["oheil <XXX@XXX.de> and contributors"]
version = "0.1.0"

If this is the case, please add them according to my example.

1 Like
20

You can just push backspace to exit pkg mode.

1 Like