GMT error message: Failure to open virtual file

Someone else had this error in 2018 and opened a GMT issue for it: https://github.com/GenericMappingTools/GMT.jl/issues/59 (I’m not sure if it proper precedure to ask for the error to be reopened or to post here.)

Things that are the same:

  • the error message

Things that are different:

  • I installed GMT late last year, so a 2018 installation issue feels unrelated. That being said, I am open to completely uninstalling GMT from my computer and reinstalling it if you can tell me how to make sure I do this correctly.
  • I’m using Julia 1.10.4, but I don’t know how to tell if I am using GMT5 or GMT6. I doubt those are even the relevant numbers anymore. (The error first occurred when I was using Julia 1.8.5 and I was hoping that updating to 1.10.4 would solve it, but it persisted in the same manner as before.) If it matters, I use VS code as my environment and made sure to update both the kernel and environment to 1.10.4. I have also restarted my computer multiple times.
  • The code I am using is running a for loop that seems to successfully run over 10,900 times before it stops and I get the error message. Thus, I don’t know how to share a simple code that will output this same error. When I restart the kernel the code will again run successfully before stopping after several thousand iterations.

Here is the error message in its entirety:

GMT: Failure to open virtual file



Stacktrace:

  [1] error(s::String)

    @ Base .\error.jl:35

  [2] GMTJL_Set_Object(API::Ptr{Nothing}, X::GMT.GMT_RESOURCE, ptr::GMTdataset{Float64, 2}, pad::Int64)

    @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\gmt_main.jl:744

  [3] gmt(::String, ::Matrix{Float64}, ::Vararg{Any})

    @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\gmt_main.jl:160

  [4] common_grd(::Dict{Symbol, Any}, ::String, ::Matrix{Float64}, ::GMTdataset{Float64, 2})

    @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\common_options.jl:3965

  [5] common_grd(::Dict{Symbol, Any}, ::String, ::String, ::String, ::Matrix{Float64}, ::Vararg{Any})

    @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\common_options.jl:3955

  [6] gmtselect(cmd0::String, arg1::Matrix{Float64}, arg2::Nothing, arg3::Nothing, arg4::Nothing; kwargs::@Kwargs{polygon::GMTdataset{Float64, 2}})

    @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\gmtselect.jl:88

  [7] gmtselect (repeats 2 times)

    @ C:\Users\johns\.julia\packages\GMT\bf7CV\src\gmtselect.jl:63 [inlined]

  [8] gmtselect

    @ C:\Users\johns\.julia\packages\GMT\bf7CV\src\gmtselect.jl:92 [inlined]

  [9] inwhichpolygon(D::Vector{GMTdataset{Float64, 2}}, point::Vector{Float64})

    @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\spatial_funs.jl:282

 [10] top-level scope

    @ c:\Users\johns\Documents\GitHub\TC-cell-research\Basins of Attraction.ipynb:73

The only part that is from my code (as far as I can tell) is line 73 in Basins of Attraction.ipynb which is

WhichTri = inwhichpolygon(tri_point,[nPtθ,nPtζ])
  • [nPtθ,nPtζ] is a point in or close to my domain, I doubt this is causing the error
  • inwhichpolygon is a GMT function. If this is causing the error, hopefully you have an idea on how I can change my implementation.
  • tri_point was saved in a different .ipynb file and is created using triplot(data, noplot=true). It is then saved and loaded into Basins of Attraction.ipynb using JLD2 as a .jld2 file. My gut is telling me that my method of saving tri_point is causing the error. If you agree, please tell me how I should be saving it. (Note: the .jld2 file is saved in the same windows folder as both .ipynb files)

Finally, I did test GMT in the command prompt (as suggested in the GMT issue that I linked above) and I did get an error, but I don’t understand what it means. So here is the print out:

     Testing GMT
      Status `C:\Users\johns\AppData\Local\Temp\jl_84WQcX\Project.toml`
  [5752ebe1] GMT v1.15.0
  [aea7be01] PrecompileTools v1.2.1
  [08abe8d2] PrettyTables v2.3.2
  [bd369af6] Tables v1.11.1
⌃ [a7073274] GDAL_jll v301.900.0+0
  [b68b8c3f] GMT_jll v6.5.2+0
  [61579ee1] Ghostscript_jll v9.55.0+4
  [58948b4f] PROJ_jll v901.300.0+1
  [ade2ca70] Dates
  [f43a241f] Downloads v1.6.0
  [37e2e46d] LinearAlgebra
  [de0858da] Printf
  [10745b16] Statistics v1.10.0
  [8dfed614] Test
      Status `C:\Users\johns\AppData\Local\Temp\jl_84WQcX\Manifest.toml`
  [a8cc5b0e] Crayons v4.1.1
  [9a962f9c] DataAPI v1.16.0
  [e2d170a0] DataValueInterfaces v1.0.0
  [5752ebe1] GMT v1.15.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.5.0
  [b964fa9f] LaTeXStrings v1.3.1
  [3da0fdf6] MPIPreferences v0.1.11
  [bac558e1] OrderedCollections v1.6.3
  [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [08abe8d2] PrettyTables v2.3.2
  [189a3867] Reexport v1.2.2
  [892a3eda] StringManipulation v0.3.4
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.11.1
  [8ce61222] Arrow_jll v10.0.0+1
  [0b7ba130] Blosc_jll v1.21.5+0
  [6e34b625] Bzip2_jll v1.0.8+1
  [2e619515] Expat_jll v2.6.2+0
  [f5851436] FFTW_jll v3.3.10+0
⌃ [a7073274] GDAL_jll v301.900.0+0
  [d604d12d] GEOS_jll v3.12.2+0
  [b68b8c3f] GMT_jll v6.5.2+0
  [78b55507] Gettext_jll v0.21.0+0
  [61579ee1] Ghostscript_jll v9.55.0+4
  [7746bdde] Glib_jll v2.80.2+0
  [0951126a] GnuTLS_jll v3.8.4+0
  [0234f1f7] HDF5_jll v1.14.3+3
  [e33a78d0] Hwloc_jll v2.11.0+0
⌅ [a51ab1cf] ICU_jll v69.1.0+0
  [aacddb02] JpegTurbo_jll v3.0.3+0
  [b39eb1a6] Kerberos_krb5_jll v1.19.3+0
  [17f450c3] LAPACK32_jll v3.12.0+0
⌅ [88015f11] LERC_jll v3.0.0+1
  [1d63c593] LLVMOpenMP_jll v15.0.7+0
  [dd4b983a] LZO_jll v2.10.2+0
  [08be9ffa] LibPQ_jll v16.0.0+0
⌅ [e9f186c6] Libffi_jll v3.2.2+1
  [94ce4f54] Libiconv_jll v1.17.0+0
  [4b2f31a3] Libmount_jll v2.40.1+0
⌅ [89763e89] Libtiff_jll v4.5.1+1
  [d3a379c0] LittleCMS_jll v2.16.0+0
  [5ced341a] Lz4_jll v1.9.4+0
  [7cb0a576] MPICH_jll v4.2.1+1
  [f1f71cc9] MPItrampoline_jll v5.4.0+0
  [9237b28f] MicrosoftMPI_jll v10.1.4+2
  [7243133f] NetCDF_jll v400.902.211+1
⌅ [4c82536e] Nettle_jll v3.7.2+0
⌅ [656ef2d0] OpenBLAS32_jll v0.3.24+0
  [643b3616] OpenJpeg_jll v2.5.2+0
⌅ [fe0851c0] OpenMPI_jll v4.1.6+0
  [458c3c95] OpenSSL_jll v3.0.14+0
  [c2071276] P11Kit_jll v0.24.1+0
  [2f80f16e] PCRE_jll v8.45.0+0
  [58948b4f] PROJ_jll v901.300.0+1
  [76ed43ae] SQLite_jll v3.45.3+0
⌅ [e0b8ae26] Thrift_jll v0.16.0+0
  [02c8fc9c] XML2_jll v2.13.1+0
  [ffd25f8a] XZ_jll v5.4.6+0
  [3161d3a3] Zstd_jll v1.5.6+0
⌅ [28df3c45] boost_jll v1.76.0+1
  [477f73a3] libaec_jll v1.1.2+0
  [06c338fa] libgeotiff_jll v100.701.300+0
  [b53b4c65] libpng_jll v1.6.43+1
  [337d8026] libzip_jll v1.10.1+0
⌅ [fe1e1685] snappy_jll v1.1.10+0
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [b77e0a4c] InteractiveUtils
  [4af54fe1] LazyArtifacts
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.10.0
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [6462fe0b] Sockets
  [2f01184e] SparseArrays v1.10.0
  [10745b16] Statistics v1.10.0
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v1.1.1+0
  [781609d7] GMP_jll v6.2.1+6
  [deac9b47] LibCURL_jll v8.4.0+0
  [e37daf67] LibGit2_jll v1.6.4+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.2+1
  [14a3606d] MozillaCACerts_jll v2023.1.10
  [4536629a] OpenBLAS_jll v0.3.23+4
  [efcefdf7] PCRE2_jll v10.42.0+1
  [bea87d4a] SuiteSparse_jll v7.2.1+1
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.8.0+1
  [8e850ede] nghttp2_jll v1.52.0+1
  [3f19e933] p7zip_jll v17.4.0+2
        Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading.
Precompiling project...
  69 dependencies successfully precompiled in 54 seconds. 7 already precompiled.
     Testing Running tests...
                Entering: test_proj4.jl
Rel. 9.3.0, September 1st, 2023
Located at: C:\Users\johns\AppData\Local/proj;C:\Users\johns\.julia\artifacts\1a92056bb5245200776914024d9a9ec6e44ccd4e/share/proj;/workspace/destdir/share/proj
GMT [WARNING]: GMT_COMPATIBILITY: Expects values from 6 to 6; reset to 6.
                Entering: test_gd_ext.jl
                Entering: test_gdal.jl
┌ Info: GDAL 3.9.0, released 2024/05/07
│ 207 GDAL drivers found
└ 75 OGR drivers found
ERROR 1: Zero positional arguments expected
ERROR 1: Zero positional arguments expected
[ Info: 1...
[ Info: 2...
ERROR 1: Zero positional arguments expected
Warning 1: OGR_G_Area() called against non-surface geometry type.
┌ Warning: Format not supported. Only TIF, PNG or netCDF are allowed. Resorting to TIF
└ @ GMT.Gdal C:\Users\johns\.julia\packages\GMT\bf7CV\src\gdal_tools.jl:502
[ Info: 3...
ERROR 10: Pointer 'hBand' is NULL in 'GDALGetRasterDataType'.

[ Info: 4...
[ Info: 5...
Warning 1: Non closed ring detected. To avoid accepting it, set the OGR_GEOMETRY_ACCEPT_UNCLOSED_RING configuration option to NO
[ Info: 6...
[ Info: 7...
ERROR 6: Incompatible geometry for operation
ERROR 6: Incompatible geometry for operation
[ Info: 8...
Geometry: POLYGON ((1198554.34 648493.09,1198553.65476738 64 ... .09))[ Info: 9...
[ Info: 10...
Layer: SELECT (100)
     Field 0 (HEX(GeomFromText('POINT(10 20)'))): [String], ...
[ Info: 11...
        CUBES
grdblend [NOTICE]: Remote data courtesy of GMT data server oceania [http://oceania.generic-mapping-tools.org]

grdblend [NOTICE]: SRTM15 Earth Relief v2.6 at 05x05 arc minutes reduced by Gaussian Cartesian filtering (26.2 km fullwidth) [Tozer et al., 2019].
grdblend [NOTICE]:   -> Download 180x180 degree grid tile (earth_relief_05m_g): S90E000
        WMS
serverURL:      http://tiles.maps.eox.at/wms?
OnlineResource: http://tiles.maps.eox.at/?
version:        1.1.1
request:        GetMap
layernames:     ["s2cloudless-2018_3857", "coastline", "s2cloudless-2020", "osm_3857", "s2cloudless_3857", "terrain_3857", "overlay_base_bright_3857", "hydrography", "graticules", "overlay_3857", "bluemarble", "coastline_black", "s2cloudless-2017_3857", "streets_3857", "overlay_bright", "s2cloudless-2019_3857", "overlay", "blackmarble_3857", "hydrography_3857", "s2cloudless-2017", "streets", "s2cloudless-2018", "s2cloudless-2019", "osm", "blackmarble", "terrain-light_3857", "s2cloudless-2020_3857_512", "bluemarble_3857", "overlay_base_3857", "s2cloudless", "terrain", "overlay_bright_3857", "coastline_3857", "s2cloudless-2020_3857", "magnetic_graticules", "terrain-light", "overlay_base_bright", "overlay_base"]

layer:  38 Layers. Use layer[k] to see the contents of layer k
name:   s2cloudless-2018_3857
title:  Sentinel-2 cloudless layer for 2018 by EOX - 3857
srs:    EPSG:900913
crs:
bbox:   (-2.0037508342789e7, 2.0037508342789e7, -2.0037508342789e7, 2.0037508342789e7)
imgformat:      image/jpeg
tilesize:       256
overviewcount:  18
resolution:     0.5971642834779395
tiled:  true
transparent:    false
WMS:http://tiles.maps.eox.at/?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=coastline&SRS=EPSG:4326&BBOX=-180,-90,180,90&FORMAT=image/png&TILESIZE=256&OVERVIEWCOUNT=17&MINRESOLUTION=0.00000536441803&TILED=true
                Entering: test_imgfuns.jl
        IMGFUNS
                Entering: test_imgtiles.jl
        TILES MOSAIC
                Entering: test_makecpts.jl
        MAKECPT
                MAKECPT - 0
                MAKECPT - 1
                MAKECPT - 2
                Entering: test_avatars.jl
        PLOT
┌ Warning: option *ms* is ignored when either 'S' or 'symbol' options are used
└ @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\psxy.jl:690
┌ Warning: markerline overrides markeredgecolor
└ @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\psxy.jl:706
┌ Warning: option *ms* is ignored when either 'S' or 'symbol' options are used
└ @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\psxy.jl:690
┌ Warning: option *marker* is ignored when either 'S' or 'symbol' options are used
└ @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\psxy.jl:692
arg1 = GMTdataset{Float64, 2}
arg2 = GMTcpt
makecpt [ERROR]: Option -G: Unable to parse 0.25
GMT: Error During Test at C:\Users\johns\.julia\packages\GMT\bf7CV\test\runtests.jl:5
  Got exception outside of a @test
  LoadError: Something went wrong when calling the module. GMT error number = 72
  Stacktrace:
    [1] error(s::String)
      @ Base .\error.jl:35
    [2] gmt(::String)
      @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\gmt_main.jl:169
    [3] fish_bg(d::Dict{Symbol, Any}, cmd::Vector{String})
      @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\psxy.jl:528
    [4] common_plot_xyz(cmd0::String, arg1::GMTdataset{Float64, 2}, caller::String, first::Bool, is3D::Bool, kwargs::Pair{Symbol, String})
      @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\psxy.jl:308
    [5] plot(arg1::Matrix{Float64}; first::Bool, kw::@Kwargs{bg::String})
      @ GMT C:\Users\johns\.julia\packages\GMT\bf7CV\src\plot.jl:122
    [6] top-level scope
      @ C:\Users\johns\.julia\packages\GMT\bf7CV\test\test_avatars.jl:61
    [7] include(fname::String)
      @ Base.MainInclude .\client.jl:489
    [8] macro expansion
      @ C:\Users\johns\.julia\packages\GMT\bf7CV\test\runtests.jl:51 [inlined]
    [9] macro expansion
      @ C:\Users\johns\AppData\Local\Programs\julia-1.10.4\share\julia\stdlib\v1.10\Test\src\Test.jl:1577 [inlined]
   [10] top-level scope
      @ C:\Users\johns\.julia\packages\GMT\bf7CV\test\runtests.jl:9
   [11] include(fname::String)
      @ Base.MainInclude .\client.jl:489
   [12] top-level scope
      @ none:6
   [13] eval
      @ .\boot.jl:385 [inlined]
   [14] exec_options(opts::Base.JLOptions)
      @ Base .\client.jl:291
   [15] _start()
      @ Base .\client.jl:552
  in expression starting at C:\Users\johns\.julia\packages\GMT\bf7CV\test\test_avatars.jl:61
Test Summary: | Pass  Error  Total   Time
GMT           |   94      1     95  55.1s
  PROJ4       |    3             3   6.6s
  GDALext     |   44            44   2.6s
  GDAL        |   30            30   8.1s
  IMGFUNS     |               None   1.3s
  MOSAIC      |               None   5.0s
  MAKECPT     |    3             3   0.6s
ERROR: LoadError: Some tests did not pass: 94 passed, 0 failed, 1 errored, 0 broken.
in expression starting at C:\Users\johns\.julia\packages\GMT\bf7CV\test\runtests.jl:5
ERROR: Package GMT errored during testing

If it matters, my larger code is trying to find the basins of attraction of a dynamical system that has been simplified to a mix between mesh points and an analytic process. To acheive this, I need to find points within the triangularization a lot. (I’m basically coloring the triangles based on their attraction.) I am working towards simplifying the process so that I don’t need to do as many iterations, but I will still have to do several thousand iterations, so I would really love to know why this error is happenning without needing to work around it.

Thank you for your willingness to help me work though this issue.

There seem to be several issues in this error.

First, your GMT version is no doubt 6.5+ (that’s what gets installed by ] add GMT).

Now, by the error description it smells a memory leak in the GMT lib, which is not easy to track.

BUT, by crazy coincidence I happen to to be working on the inwhichpolygon function and speed it up by a factor of ~100 when calling it with many query points, and it also now uses a pure Julia function instead of calling the GMT C lib. So, if that was the cause of error it should hopefully avoid it now. Please try the GMT.jl master version to see if fixes it.

Regarding the issue of saving the triangulation file. Don’t know if JLD2 causes some issue, but you can try to use gmtwrite to save the triangulation and gmtread to read it back.

Thank you for putting a name to what the error might be.

I’ve been trying some of your suggestions throughout the week and it didn’t seem to be working, but I really sat down to deal with it today, and the error hasn’t appeared yet. So, I’m not convinced it is fixed, but I’m glad that I’m not currently getting an error.

As a clarifying question: when you suggest trying the master version, does that mean updating the version I have, uninstalling and reinstalling, or something else. I tried the first two earlier this week.

Thanks again!

From you message I can see that you are not using the master version, so whatever caused the error is still there. To install the master version of a package we add the `#master to package name. Thst is, in this case

] add GMT#master

The GMT point-in-polygon routine is very fast but calling the GMT lib thousands of times is not. The code on master avoids that so for (multi)thousands of points it is 2 orders of magnitude faster. And hopefully avoids the error.

The failing tests on Windows are due to