Dynamic nested threads pre-1.8

General Usage
1

It’s wonderful that 1.8 provides dynamic scheduling in @threads that can (?) be nested. How can I obtain this functionality using Julia 1.7?

I know that I can do @sync for and then @spawn. What other possibilities exist? At an extreme, would I be able to use parts of the 1.8 multithreading source code, rename the macros and functions in it, and use that until 1.8 is released?

Thanks!

2

Why couldn’t you use 1.8?

2 Likes
4

I think GitHub - JuliaFolds/Folds.jl: A unified interface for sequential, threaded, and distributed fold provides this functionality

1 Like
5

Because other packages that I use are not available for it.

6

I’ll have a look, thanks

7

Folds does not look like what i was hoping for, but thanks.

8

Then you can probably check out the other packages of the organization JuliaFolds · GitHub, from what I understand they provide similar functionalities but with different paradigms

9

Thanks. I’m aware of those packages, but am just looking for something that allows and exploits for generic nested multithreading.

10

But wouldn’t it be easier to bring the packages that you need up to speed?

1 Like
11

I hope not!

12

Sorry: why?

13

There are quite a few and I don’t know in what sense they are not yet compatible with 1.8.

14

Which packages are they? 1.7 to 1.8 shouldn’t have any breaking changes (at least if the packages aren’t using Julia internals), so if packages aren’t working with 1.8, we want to know.

1 Like
15

Thanks @Oscar_Smith . Trying to install CSV, I get

[ Info: Precompiling CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b]
ERROR: LoadError: ccall method definition: argument 1 type doesn't correspond to a C type
Stacktrace:
 [1] top-level scope
   @ ~/.julia/packages/SentinelArrays/iHRpO/src/SentinelArrays.jl:209
 [2] include
   @ ./Base.jl:422 [inlined]
 [3] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
   @ Base ./loading.jl:1400
 [4] top-level scope
   @ stdin:1
in expression starting at /home/joris/.julia/packages/SentinelArrays/iHRpO/src/SentinelArrays.jl:1
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile SentinelArrays [91c51154-3ec4-41a3-a24f-3f23e20d615c] to /home/joris/.julia/compiled/v1.8/SentinelArrays/jl_kfzQoP.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
    @ Base ./loading.jl:1551
  [3] compilecache
    @ ./loading.jl:1495 [inlined]
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1199
  [5] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1084
  [6] macro expansion
    @ ./loading.jl:1064 [inlined]
  [7] macro expansion
    @ ./lock.jl:223 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1028
  [9] include
    @ ./Base.jl:422 [inlined]
 [10] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1400
 [11] top-level scope
    @ stdin:1
in expression starting at /home/joris/.julia/packages/CSV/nofYz/src/CSV.jl:1
in expression starting at stdin:1
ERROR: Failed to precompile CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b] to /home/joris/.julia/compiled/v1.8/CSV/jl_oBVra2.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base ./loading.jl:1551
 [3] compilecache
   @ ./loading.jl:1495 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1199
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1084
 [6] macro expansion
   @ ./loading.jl:1064 [inlined]
 [7] macro expansion
   @ ./lock.jl:223 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1028
16

my version of Julia 1.8 is from March 29, but that’s the latest

17

with a little tinkering, it now appears to be working: thanks @PetrKryslUCSD and @Oscar_Smith !

2 Likes