Setting ENV for a shared library in __init__

General Usage
1

In this GDAL.jl PR I want to set an environment variable PROJ_LIB that one of the underlying shared libraries uses to find resources. If I set this environment globally, in the shell, or in startup.jl with ENV["PROJ_LIB"] = ... the shared library picks it up as expected.

However I don’t want to ask users to do this, so I wanted to set ENV["PROJ_LIB"] = ... in the module’s __init__ function. In this case the shared library does not pick it up and I wonder why. Julia itself does pick it up:

module A
__init__() = ENV["ABC"] = "2"
end

using Main.A
ENV["ABC"]  # => "2"

Any ideas for why this happens or how I can make it work? I find it particularly strange that it works from startup.jl but not from __init__, also when I set ENV before check_deps(), which is presumably the moment it loads the libraries.

1 Like
2

Poking around inside the current libproj, I came across proj_context_set_search_paths (legacy api pj_set_searchpath). I didn’t get to the bottom of your problem, but this should be better than using the environment variable.

For even more powerful integration with proj’s resource discovery there’s also proj_context_set_file_finder for custom resource search, and pj_ctx_set_fileapi to define how resources are loaded.

3

Is this environment variable read at the time when the external library is opened? If that’s the case, I guess __init__ is too late to do this. I remember that I couldn’t change dlopen option if the library is ccalled from the module (but I’m not super sure about this).

4

It seems indeed that __init__ is too late. Recently the __init__ docs were written. It is called after all statements in the module have been executed. Does having functions with ccall in there cause the library to be dlopened already? Seems like it. I believe in general libraries load in their copy of environment variables at that time, such that setting it later is useless. Because it does look like PROJ just does a runtime getenv.

If this is the case that might be worth clarification in the __init__ docs.

Yeah indeed. Currently for GDAL_DATA we do both methods, CPLSetConfigOption and ENV. ENV was added at the time for parallel processes. Though a quick test now shows that this works with CPLSetConfigOption only, and not at all with ENV only (which is consistent with my assumptions in the paragraph above).

I’ll focus then on trying to set it directly in PROJ with one of the functions you mentioned. Though I have not been successful so far, which is why I came back to ENV. I tried proj_context_set_search_paths and proj_context_set_file_finder. It’s a bit tricky to have a clear mental model of what is happening, since PROJ is only loaded indirectly by GDAL, so not sure how to work with the contexts.

5

Oh right. Perhaps GDAL is overriding those settings itself or using a custom proj context?

6

I tried looking into this a little, and found ogr_proj_p.h and ogr_proj_p.cpp.

So there is a private header that provides OSRGetProjTLSContext. Not sure why this is private or how to make it public, because I can’t seem to ccall it in libgdal. The cpp file also has OSRSetPROJSearchPaths. Perhaps I need to patch the GDALBuilder in some way? I would think this would need to be exposed in the C API.

7

Ah great, turns out OSRSetPROJSearchPaths is part of the GDAL C API after all, and it works if I call this function. Thanks for the help both!

2 Likes