Plots in the terminal (with sixel)

affans · November 15, 2022, 10:06pm

Does Sixel depend on the plotting library or the terminal itself or both? Does the WIndows Terminal support sixel? Does Kitty terminal support Sixel?

jheinen · November 16, 2022, 7:47am

Normally it should work like this:

GKSwstype=iterm julia
julia> using GR
julia> surface(peaks())

This works both with GR (see Retina screenshot) and Plots, but in Plots the resolution is extremely poor!?

t-bltg · November 16, 2022, 8:36am

Plots also supports an experimental sixel mode when ImageInTerminal is loaded:

julia> ENV["PLOTS_IMAGE_IN_TERMINAL"] = true;  # experimental for now
julia> using ImageInTerminal, Plots
julia> pyplot();  # or whatever backend that can save to `png` (except `UnicodePlots` which is built in for terminal plots)
julia> plot(1:2)

Here on mlterm, linux.

photor · November 16, 2022, 8:42am

Does this work in windows terminal?

t-bltg · November 16, 2022, 8:44am

I do not know that.
The not exhaustive list of tested terminals, which support sixels, are listed in GitHub - JuliaIO/Sixel.jl: The Julia wrapper of libsixel.

piechologist · November 16, 2022, 10:42am

Exactly, that’s where default(display_type=:inline) shines.

Using GR.jl or GRUtils.jl rather than Plots.jl reduces the TTFP but there are downsides for my setup (macOS, iTerm):

GR.jl plots are always positioned at the upper left corner of the terminal window and mess up the scroll back.
Plots.jl allows switching back and forth between default(display_type=:inline) → terminal and default(display_type=:gui) → GKSQT. No need to set an ENV variable before loading the plotting package.
Additionally, GR.jl can’t consume Date/Time/DateTime vectors for the x axis out of the box.

Do you have any idea why GKSQT is launched when plotting inline? Is that a Plots.jl issue?

There used to be a GKSTerm.app that kept the plot history so you could go back. Is that app still around?

piechologist · November 16, 2022, 10:53am

Thank you. I tried that but just got pages of errors instead of a figure (macOS 12.6.1, iTerm 3.4.17, Julia 1.8.3, Plots.jl 1.36.1 with GR backend, and a fresh ImageInTerminal.jl, everything’s up-to-date).

Why would I need ImageInTerminal.jl if there’s default(display_type=:inline)? Is it intended for Linux or Windows? It pulls in many additional dependencies and adds to TTFP.

EDIT: mentioned GR backend

t-bltg · November 16, 2022, 10:57am

What error ?

Why would I need ImageInTerminal.jl if there’s default(display_type=:inline)?

Not all backends support this: :inline is ~~specific to~~ currently only used by GR.
By using ImageInTerminal, we first encode a png (a feature that all backends support now), encode it in sixels and display them in the terminal: this is a backend independent approach.

Here on linux + mlterm, the ImageInTerminal approach works flawlessly with the GR backend.

piechologist · November 16, 2022, 11:15am

Got it

Error resolved: now I see there’s one more package missing. I added ImageIO.jl manually and then it works as expected.

Just for the record:

I started with a fresh environment:

Status `~/.julia/environments/v1.8/Project.toml`
  [d8c32880] ImageInTerminal v0.5.2
  [91a5bcdd] Plots v1.36.1

Did:

ENV["PLOTS_IMAGE_IN_TERMINAL"] = true
using ImageInTerminal, Plots
plot(1:2)

Got:

Errors encountered while load FileIO.Stream{FileIO.DataFormat{:PNG}, IOBuffer, Nothing}(IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=false, append=true, size=12904, maxsize=Inf, ptr=1, mark=-1), nothing).
All errors:
===========================================
ArgumentError: Package ImageIO [82e4d734-157c-48bb-816b-45c225c6df19] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

===========================================
ArgumentError: Package QuartzImageIO [dca85d43-d64c-5e67-8c65-017450d5d020] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

===========================================
ArgumentError: Package ImageMagick [6218d12a-5da1-5696-b52f-db25d2ecc6d1] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

===========================================

Fatal error:
Error showing value of type Plots.Plot{Plots.GRBackend}:
ERROR: ArgumentError: Package ImageIO [82e4d734-157c-48bb-816b-45c225c6df19] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
  [1] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1306
  [2] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1200
  [3] macro expansion
    @ ./lock.jl:223 [inlined]
  [4] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1195
  [5] #34
    @ ~/.julia/packages/FileIO/aP78L/src/loadsave.jl:203 [inlined]
  [6] lock(f::FileIO.var"#34#35"{Base.PkgId}, l::ReentrantLock)
    @ Base ./lock.jl:185
  [7] action(::Symbol, ::Vector{Union{Base.PkgId, Module}}, ::FileIO.Formatted; options::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ FileIO ~/.julia/packages/FileIO/aP78L/src/loadsave.jl:203
  [8] action
    @ ~/.julia/packages/FileIO/aP78L/src/loadsave.jl:196 [inlined]
  [9] #load#15
    @ ~/.julia/packages/FileIO/aP78L/src/loadsave.jl:120 [inlined]
 [10] load
    @ ~/.julia/packages/FileIO/aP78L/src/loadsave.jl:116 [inlined]
 [11] display(d::ImageInTerminal.TerminalGraphicDisplay{Base.TTY, Base.TTY}, #unused#::MIME{Symbol("image/png")}, bytes::Vector{UInt8})
    @ ImageInTerminal ~/.julia/packages/ImageInTerminal/VHb5T/src/display.jl:11
 [12] display(#unused#::Plots.PlotsDisplay, plt::Plots.Plot{Plots.GRBackend})
    @ Plots ~/.julia/packages/Plots/gzYVM/src/init.jl:131
 [13] display(x::Any)
    @ Base.Multimedia ./multimedia.jl:328
 [14] #invokelatest#2
    @ ./essentials.jl:729 [inlined]
 [15] invokelatest
    @ ./essentials.jl:726 [inlined]
 [16] print_response(errio::IO, response::Any, show_value::Bool, have_color::Bool, specialdisplay::Union{Nothing, AbstractDisplay})
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:296
 [17] (::REPL.var"#45#46"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:278
 [18] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:521
 [19] print_response(repl::REPL.AbstractREPL, response::Any, show_value::Bool, have_color::Bool)
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:276
 [20] (::REPL.var"#do_respond#66"{Bool, Bool, REPL.var"#77#87"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::Any, ok::Bool)
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:857
 [21] #invokelatest#2
    @ ./essentials.jl:729 [inlined]
 [22] invokelatest
    @ ./essentials.jl:726 [inlined]
 [23] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
    @ REPL.LineEdit /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/LineEdit.jl:2510
 [24] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:1248
 [25] (::REPL.var"#49#54"{REPL.LineEditREPL, REPL.REPLBackendRef})()
    @ REPL ./task.jl:484
Stacktrace:
  [1] handle_error(e::ArgumentError, q::Base.PkgId, bt::Vector{Union{Ptr{Nothing}, Base.InterpreterIP}})
    @ FileIO ~/.julia/packages/FileIO/aP78L/src/error_handling.jl:61
  [2] handle_exceptions(exceptions::Vector{Tuple{Any, Union{Base.PkgId, Module}, Vector}}, action::String)
    @ FileIO ~/.julia/packages/FileIO/aP78L/src/error_handling.jl:56
  [3] action(::Symbol, ::Vector{Union{Base.PkgId, Module}}, ::FileIO.Formatted; options::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ FileIO ~/.julia/packages/FileIO/aP78L/src/loadsave.jl:228
  [4] action
    @ ~/.julia/packages/FileIO/aP78L/src/loadsave.jl:196 [inlined]
  [5] #load#15
    @ ~/.julia/packages/FileIO/aP78L/src/loadsave.jl:120 [inlined]
  [6] load
    @ ~/.julia/packages/FileIO/aP78L/src/loadsave.jl:116 [inlined]
  [7] display(d::ImageInTerminal.TerminalGraphicDisplay{Base.TTY, Base.TTY}, #unused#::MIME{Symbol("image/png")}, bytes::Vector{UInt8})
    @ ImageInTerminal ~/.julia/packages/ImageInTerminal/VHb5T/src/display.jl:11
  [8] display(#unused#::Plots.PlotsDisplay, plt::Plots.Plot{Plots.GRBackend})
    @ Plots ~/.julia/packages/Plots/gzYVM/src/init.jl:131
  [9] display(x::Any)
    @ Base.Multimedia ./multimedia.jl:328
 [10] #invokelatest#2
    @ ./essentials.jl:729 [inlined]
 [11] invokelatest
    @ ./essentials.jl:726 [inlined]
 [12] print_response(errio::IO, response::Any, show_value::Bool, have_color::Bool, specialdisplay::Union{Nothing, AbstractDisplay})
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:296
 [13] (::REPL.var"#45#46"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:278
 [14] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:521
 [15] print_response(repl::REPL.AbstractREPL, response::Any, show_value::Bool, have_color::Bool)
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:276
 [16] (::REPL.var"#do_respond#66"{Bool, Bool, REPL.var"#77#87"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::Any, ok::Bool)
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:857
 [17] #invokelatest#2
    @ ./essentials.jl:729 [inlined]
 [18] invokelatest
    @ ./essentials.jl:726 [inlined]
 [19] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
    @ REPL.LineEdit /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/LineEdit.jl:2510
 [20] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
    @ REPL /Applications/julia-1.8.3/share/julia/stdlib/v1.8/REPL/src/REPL.jl:1248
 [21] (::REPL.var"#49#54"{REPL.LineEditREPL, REPL.REPLBackendRef})()
    @ REPL ./task.jl:484

Okay, I see there’s one more package missing. I added ImageIO.jl

t-bltg · November 16, 2022, 11:19am

Nice that you got it working: optional dependencies are a bit of a pain in julia (but they are being worked on: Support in code loading and precompilation for weak dependencies by KristofferC · Pull Request #47040 · JuliaLang/julia · GitHub).

jheinen · November 16, 2022, 11:53am

using GR; inline("item", false) allows scroll back
inline("") switches back to GUI
NYI

For Plots, launching of gksqt might be suppressed setting GKSwstype=nul before starting Julia.

GKSTerm.app for macOS is still there:

GKSwstype=quartz julia ...

piechologist · November 16, 2022, 12:37pm

That does the trick

devel-chm · November 16, 2022, 2:37pm

Regarding sixel output, the gnuplot program and thus
Gnuplot.jl and Gaston.jl support that output format.

xterm can display sixel graphics if compiled with the
appropriate flags ( --enable-sixel-graphics ).

NOTE: Removed outdated libsixel information.

Please see Plots in the terminal (with sixel) - #34 by t-bltg
and thanks to t-bltg for the correction!

t-bltg · November 16, 2022, 3:00pm

Note that saitoha/libsixel is unmaintained and has been forked at libsixel/libsixel (see issue).

mbaz · November 16, 2022, 3:28pm

As an alternative to xterm, wezterm is a modern terminal that supports sixel. It has a few rough edges but it’s in active development. kitty doesn’t support sixel, but there is GitHub - simonschoelly/KittyTerminalImages.jl: A package that allows Julia to display images in the kitty terminal editor

mbaz · November 17, 2022, 3:28pm

I just discovered a nice trick with sixel and gnuplot: in-terminal animations. The following code is from Gaston’s development branch (still not published), but should be easy to adapt to Gaston current or to Gnuplot.jl. Note the animate in the term definition.

using Gaston

Gaston.config.term = "sixelgd truecolor size 320, 200 animate"
f = Figure()
phase = 0:0.05:2π
for i in range(0, 2π, length=40)
    plot(f, sin.(phase.+i)) |> display
    sleep(0.1)
end

Peek2022-11-17-10-25

Not very practical since it messes up the terminal, but cool anyway.

photor · November 17, 2022, 3:32pm

would be cleaner if it can be put at the upper right corner of the term

mbaz · November 17, 2022, 3:54pm

Yeah – I wonder if Gnuplot’s developers would be open to feature requests, now that interest in sixel is increasing and more terminals support it.

TheLateKronos · May 3, 2023, 8:09am

This is amazing. I only have one problem.
My terminal background is black. The text, frame, and ticks, are all black. Is it possible to change this, allowing plotting on dark backgrounds? The current look is like this:

I have tried mucking around with gr3.setbackgroundcolor, giving arguments like all zeros, all 256, all 0.5, but nothing produced a visible change.

jheinen · May 3, 2023, 8:25am

In dark mode you have to add GR.usecolorscheme() after importing GR:

using GR
usecolorscheme(2)
...