Omniscape model in Julia

General Usage
1

When running Omniscape model via Julia I got this error massage:

run_omniscape("D:/15_Omniscape/tobler_mean/script/tobler_mean.ini")
ERROR: OutOfMemoryError()
Stacktrace:
  [1] GenericMemory
    @ .\boot.jl:516 [inlined]
  [2] new_as_memoryref
    @ .\boot.jl:535 [inlined]
  [3] Array
    @ .\boot.jl:582 [inlined]
  [4] Array
    @ .\boot.jl:592 [inlined]
  [5] similar
    @ .\array.jl:362 [inlined]
  [6] permutedims(B::Matrix{Float32}, perm::Vector{Int64})
    @ Base .\multidimensional.jl:1633
  [7] (::Omniscape.var"#1#2"{DataType})(raw::ArchGDAL.Dataset)
    @ Omniscape C:\Users\JanZachar\.julia\packages\Omniscape\VSLPS\src\io.jl:46
  [8] read(f::Omniscape.var"#1#2"{DataType}, args::String; kwargs::@Kwargs{})
    @ ArchGDAL C:\Users\JanZachar\.julia\packages\ArchGDAL\xZUPv\src\context.jl:268
  [9] read
    @ C:\Users\JanZachar\.julia\packages\ArchGDAL\xZUPv\src\context.jl:265 [inlined]
 [10] read_raster(path::String, T::Type{Float64})
    @ Omniscape C:\Users\JanZachar\.julia\packages\Omniscape\VSLPS\src\io.jl:16
 [11] run_omniscape(path::String)
    @ Omniscape C:\Users\JanZachar\.julia\packages\Omniscape\VSLPS\src\main.jl:437
 [12] top-level scope
    @ REPL[1]:1

I dont know how to set up detail config for Julia performance.
Thanks for help.

PS: Here is the structire of my .ini file:

[Input]
project_folder = D:/15_Omniscape/tobler_mean
resistance_file = D:/15_Omniscape/tobler_mean/input/resistance_tobler_mean.tif
temp_file = E:/Julia_temp

[Output]
output_dir = D:/15_Omniscape/tobler_mean/output

[Options]
source_from_resistance= true
block_size = 7
radius = 500
project_name = tobler_mean
parallelize = true
parallel_batch_size = 20
calc_flow_potential = true
calc_normalized_current = true
write_raw_currmap = true
write_normalized_currmap = true
write_flow_potential = true
write_as_tif = true
2

Apparently you gave it a problem too big to solve with the computer you have.

I don’t know anything about this package or how it scales with problem size, and it’s quite difficult for anyone to help you without knowing the details of your files. But I would suggest starting with a much smaller problem — maybe one of the examples from the Omniscape.jl tutorials or tests, and then slowly scaling it up to see how big a problem it can handle.

1 Like
3

How much RAM do you have? What is the output of

Sys.free_memory()/1e9

Which OS are you using?

4

Intel(R) Core™ i7-8750H CPU @ 2.20GHz 2.20 GHz, RAM 16GB, input resistance raster: 1,5 GB. I think it woult be necessary to locate temp directory for Julia somewhere else (not on C drive) as it is possible in e.g. in R language, but in Julia I dont know how to do it…

5

Are you using Windows 11? In this case this guide might help: How to manage virtual memory on Windows 11 | Windows Central

And if you use parallel processing, reducing the number of parallel threads also reduces the memory requirements.

6

I don’t think it makes any sense to do these things unless you have some idea of how much memory you need. For all you know, you might be posing a ridiculously large problem that would require petabytes of memory and billions of CPU hours.

You really need to understand the scaling of the algorithms you are using — ideally from first principles knowledge of the underlying algorithms, but at the very least you can do some experiments where you start from a much smaller problem and scale up, observing how the time/memory requirements increase.

7

Thanks all for reply. I will try to perform the whole procedure at first on small fragment of resistance raster to see how it will preform. It seems to be the most appropriate procedure as you pointed out.