New Package: Journal.jl

Specific Domains Data
1

I would like to announce a new logging package we have developed and are using in a number of internal projects:

https://github.com/joshbode/Journal.jl

Journal.jl is currently unregistered but development has stabilised.
Any feedback would be appreciated :slight_smile:

To install it, simply run:

julia> Pkg.clone("https://github.com/joshbode/Journal.jl")

Journal.jl is an extensible hierarchical logging framework for Julia with
multiple output targets, including:

  • Streams: Console, File, etc
  • Google Datastore (via GoogleCloud.jl)
  • Arbitrary webhook APIs (e.g. Slack) with ability to authenticate via custom
    methods

Loggers can be configured hierarchically, with child loggers set to log at different
levels or to different targets.

Data stored by Journal.jl can also be read back later from a specific store.

Additional features (undocumented but finished) include defining metrics to be applied to stored log data.

Basic Usage

Journal.jl is generally configured via YAML. The YAML format specifies:

  • stores with associated:
    • store type: e.g. io, datastore, webhook or some custom registered type
    • plus configuration relevant to the specific store type
  • loggers with associated:
    • log level: (DEBUG < INFO < WARN < ERROR)
    • target stores: referencing a store definition
    • dependent children: referencing child loggers that are to be passed the same messages as the parent

Here is a simple configuration file:

# journal.yml
stores:
  console:
    type: io
  file:
    type: io
    file: [journal.log, w+]
    format: "$timestamp: $level: $name: topic=$topic; message=$message; value=$value"
loggers:
  screen:
    level: DEBUG
    stores: [console]
    children: [disk]
  disk:
    level: INFO
    stores: [file]

Journal can now be set up from the configuration file:

using Journal
Journal.config("journal.yml")

Use the loggers:

# use default "root" logger (screen)
Journal.info("Is this thing on?")

# specify topic (overrides line func[file:line])
Journal.info("Helllloooooo"; topic="greeting")

# attach a value to the message
Journal.info("Testing, Testing"; value=[1, 2, 3], topic="mic_check")
Journal.warn("Check"; value=[1, 2], topic="mic_check")

# override the timestamp
Journal.info("A long time ago in a galaxy far far away..."; timestamp=DateTime("1977-05-25"), topic="star wars")

# add custom tags
Journal.info("Exterminate"; topic="threat", species="dalek", source="Davros")

# log to a specific logger
logger = getlogger(:screen)
Journal.debug(logger, "Can you hear me?")  # note: not stored to "disk" logger since DEBUG < INFO

# or using a do block
getlogger(:disk) do logger
    Journal.warn(logger, "Don't touch that!")
    Journal.error(logger, "ZAP")
end

Journal can also read back log data:

using DataTables
using Base.Dates

store = getstore(:file)
records = read(store)
table = DataTable(records)

# apply a filter to the data
mic_checks = read(store; filter=Dict(:topic => "mic_check"))

# apply a timestamp filter [start, finish]
recent = read(store; start=now(UTC) - Day(1), finish=now(UTC))

See https://github.com/joshbode/Journal.jl/blob/master/README.md for more detail.

6 Likes