How to Best Store and Access Credentials in Julia?

General Usage
, , ,
1

Hi all,

I was recently stumped by something:

Suppose I have some credentials I need to store to access a SQL DB. How should I best store these credentials for Julia to easily access?

My current pattern is to use a combo of DataFrames.jl and CSV.jl like so:

  1. Create a credentials.csv like so:
user pass
lama 1234
  1. Using CSV.jl and DataFrames.jl, read it into a DataFrame:
using CSV
using DataFrames

f = CSV.File("credentials.csv") |> DataFrame;
  1. Parse and assign variables:
USERNAME = f[1, :user]
PASSWORD = f[1, :pass]

Is there a better way of doing this? Any better suggestions? Thank you!

~ tcp :deciduous_tree:

2 Likes
2

Usually environment variables are used for this, perhaps with an .env file that you source:

export DB_USER=fredrik
export DB_PASSWORD=*******

and then access as

db_user = ENV["DB_USER"]
db_password = ENV["DB_PASSWORD"]

Edit: Looks like Discourse automatically edits the post and insert *s instead of my password. I typed DB_PASSWORD=hunter2 but it was edited.

7 Likes
3

I believe the correct answer is: it depends (on the platform).
For example if you run in GCP: Secret Manager  |  Google Cloud

1 Like
4

Maybe, one could make a function that would interactively ask the credentials if those env-variables are empty.

Does anyone know easy cyber secure solution?

5

I really like this solution! Thanks @fredrikekre - when you say source, how would I do that? Would it be as simple as include("credentials.env") or how do you imagine it?

I like the function idea @Tero_Frondelius - I will prototype something and post back later.

6

I was referring to the shell command source:

$ cat .env 
export DB_USER=fredrik

$ source .env 

$ julia -E 'ENV["DB_USER"]'
"fredrik"
2 Likes
7

Oh gotcha! Do you happen to know of an analogous method I could do this for within Julia?

8

You can just save credentials in a Julia file and add it with include

1 Like
9

Do not save credentials in a file within your source tree. I an no expert.
If you upload to Github/Gitlab you have a security breach - I believe Github scans repositories for private keys which are uploaded in files.
A recent big security problem was triggered by keeping credentials in a file.

2 Likes
10

That’s true, so I usually generate two files: secrets.jl and secrets_template.jl You put secrets.jl in .gitignore and write all necessary constants in secrets_template.jl without values of course. This way you always know how to deploy your script.

2 Likes
11

@Skoffer - how do you happen to generate these two files? I ideally would like to take in user input, write that to a Julia file that could then be ran and export these user defined values whenever they run a package I am creating.

12

It looks like you have a slightly different scenario in mind. You can see an example of what I said for example here. I.e. it is assumed, that the user should do the following

  1. Clone your application
  2. Copy secrets_template.jl to secrets.jl
  3. Edit file secrets.jl by hand.

Of course, it means, that the user is advanced enough, to understand what is .gitignore, how relative paths are working and can without errors edit julia files. If it is a problem, then I would recommend to use environment variables, it is much easier to explain and it is less error prone.

You can try to find middle ground, by using .env file, so less advanced users can just setup environment variables on os level, and more advanced users can write all values in .env file and get a more flexible approach. To avoid issues with file sourcing, you can add something like this to the beginning of your application

function load_dotenv(filename = ".env")
    isfile(filename) || return
    for line in eachline(filename)
        var, val = strip.(split(line, "="))
        ENV[var] = val
    end
end

load_dotenv()

This way .env will override any environment variables and will do nothing if the file does not exist. As a side note, it could be good to revive DotEnv.jl which was written specifically for this purposes.

Of course, you still can simplify users life by creating something like gen_dotenv.jl with the content

function gen_dotenv()
    print("User: ")
    user = readline()
    pass = read(Base.getpass("Password"), String)
    open(".env", "w") do io
        println(io, "USER = ", user)
        print(io, "PASSWORD = ", pass)
    end
end

gen_dotenv()

and explain to a user, that he should run this script at the root of the application.

4 Likes
13

I was going to advise using DotEnv.jl when I saw this comment. Is it dead? I was under the impression that it simply did not get updated because there wasn’t much to do anyway…

In any case, if DotEnv.jl is indeed dead, I do think it would be important to revive it. I know about several people relying on it in production environments.

14

Ah, my bad, it uses old REQUIRE instead of Project.toml, which is why I decided that it is dead. Maybe cosmetic fresh up is all that is needed.

1 Like