How to display images in Pluto?

I’ve seen a couple of threads discussion a bit of these but I couldn’t make it work for me in GMT.jl. There I have

		if (isdefined(Main, :IJulia) && Main.IJulia.inited)		# From Jupyter?
			if (fname == "") display("image/png", read(out))

and I’ve seen that GR start to do it similar for Pluto

isdefined(Main, :PlutoRunner) && Main.PlutoRunner isa Module

but then I’m lost on how the displaying the images is done. Tried this

		elseif isdefined(Main, :PlutoRunner) && Main.PlutoRunner isa Module
			display("image/png", read(out))

but it errors with

MethodError: no method matching display(::MIME{Symbol("image/png")}, ::Vector{UInt8})

Closest candidates are:

display(::MIME, ::Any) at multimedia.jl:338

display(!Matched::AbstractString, ::Any) at multimedia.jl:217

display(!Matched::TextDisplay, ::Any) at multimedia.jl:243

...

    display(::MIME{Symbol("image/png")}, ::Any)@multimedia.jl:349
    display(::String, ::Any)@multimedia.jl:217
    showfig(::Dict{Symbol, Any}, ::String, ::String, ::String, ::Bool, ::String)@common_options.jl:2742
...

How can this be made to work?

1 Like

If you are simply trying to display an image in a Pluto notebook, then you can find examples in the PlutoUI.jl sample notebook (you can open it from the “welcome to Pluto.jl” homepage that pops up when you start Pluto). It has examples for downloadable images and for local files.

1 Like

Not exactly. I want that, like in IJulia, I can do plot(..., show=true) and get the image displayed in the notebook.

Tried to add PlutoUI.LocalResource(out) but got the obvious error UndefVarError: PlutoUI not defined and I certainly do not want to make Pluto a dependency of the package.

1 Like

I find Julia’s display system very difficult to understand. Here’s what I have learned making Gaston work on notebooks.

  • You don’t really need a display method; you need show methods, one per MIME type.
  • The notebook will cause your show methods to be called. Each notebook has an ordered list of preferred MIME types. Some, like Jupyter, will call every show method available and then display the one it likes better.
  • You can define showable methods to indicate to the notebook which MIME types you support.
  • You need to wrap your plots in a type, so that multiple dispatch works and your methods (and not others) end up being called. I’ll assume that your plot function has return type GMTPlot.

Let’s say you want to (only) produce PNG plots in Pluto. First define showable:

function showable(mime::Type{T}, p::GMTPlot) where T <: MIME
    if mime == MIME"image/png"
        return true
     end
     return false
end

Now the show method. This is based on what I do in Gaston. It is essential that the actual plot data is written to io.

function show(io::IO, mime::MIME"image/png", p::GMTPlot)
    tmpfile = tempname()
    ... # save p as PNG with filename tmpfile
    write(io, read(tmpfile))
    rm(tmpfile, force=true)
end

And that should be it! I hope this points you in the right direction.

6 Likes

Thanks, I see some light but … still fuzzy :slight_smile:

GMT actually can produce a PNG directly so perhaps I would only need a show method that would do a write(io, read(tmpfile))

However, some other package (GADM) screwed me an now when I try to load Pluto, I’m have an error. Need to fid out how to fix this first.

Downloaded artifact: MbedTLS
ERROR: InitError: could not load library "C:\Users\joaqu\.julia\artifacts\766d976def66e8367dd690d05cfe422f883d43ba\bin\libmbedtls.dll"
The specified procedure could not be found.

I know how it feels :rofl:

PlutoUI is a separate light-weight) package, not part of Pluto.jl.

1 Like

Still in (dark) shadows.

First lots of troubles because Pluto seems to be un-usable on Windows due to a problem MbedTLS which I managed to workaround with a local Win build with Visual Studio.

Then I tried this, but now no error no nothing. Since what I need to pass is a file name I had to create a dump type to wrap it so a new show method could be defined.

The @show(w) line get displayed but the println(wp.fname) in the show method is not, showing it doesn’t get called.

		elseif isdefined(Main, :PlutoRunner) && Main.PlutoRunner isa Module
			w = WrapperPluto(out)
@show(w)
			show(w)

and

function Base.:showable(mime::Type{T}, wp::WrapperPluto) where T <: MIME
	return true
end
function Base.:show(io::IO, ::MIME"image/png", wp::WrapperPluto)
	println(wp.fname)
	write(io, read(wp.fname))
end

… and I finally managed to make it call that show method, the write(io, read(wp.fname)) displayed on the Julia REPL instead of the Pluto notebook.

Time to give up, I guess.

Here is a MWE of a type that can be displayed as image:

In one cell (or in your package):

begin
struct Wow
filename
end

function Base.show(io::IO, ::MIME"image/png", w::Wow)
write(io, read(w.filename))
end
end

in another cell:

Wow("~/cat.png")

Note that display is implicit in Pluto. Pluto users do not call show or display or @show – you see objects by having that object be the result of a cell. Internally, Pluto calls Base.show(internal_io_buffer, richest_possible_mime, cell_result) after running a cell, and relays that to the browser.


You generally don’t want any more code than this MWE: do not define a Base.showable method unless the set of possible mime types for your struct type is dynamic.

In particular, the definition Base.showable(...) = true in your last comment is definitely not what you want. This means that it is displayable as LaTeX, HTML, JPG, etc., and that it’s up to Pluto/Jupyter to choose a favorite (Pluto likes HTML).

3 Likes

Thanks, it’s getting closer. In fact it’s close to what I had already tried except that I was trying with show and display

Unfortunately I want. While it works if I add the equivalent to Wow("~/cat.png") line (GMT.WrapperPluto("C:\\TMP\\GMTjl_tmp.png"), that it’s no solution. What I need is a way that

plot(...., show=true)

works out of the box without the user having to call the wrapper type with the file name that he is not even supposed to know all the times.

Basically, how do I run the equivalent of Wow("~/cat.png") from the code and not from a Pluto cell?

1 Like

To answer your question:

function plot(args...)
    return Wow("~/cat.png")
end

Why do you want to create a separate wrapper function? A package like Plots.jl defines the svg,png,etc show methods for their Plots.Plot type. Plots.plot and Plots.scatter return a Plots.Plot.

Again, display is implicit in Pluto. This is also true for every other Julia environment, but they support additional, explicit, ways of showing things to the user. This is confusing, which is why Pluto does not support explicit display. So to support all Julia environments, including Pluto, make your plot function return an object on which Base.show can be called. Try it yourself – the Wow MWE above works in Pluto, Juno, VS Code, Jupyter, Franklin, etc. (This means that you also don’t need to check whether you are running inside Pluto.)

1 Like

I am not creating any wrapper function. GMT.jl has many plot producing modules and one one of them is also called plot and shares many of the Plots plot module. What I am trying to achieve is that, like in Jupyter, users can do plot(....) and see the result in the notebook. Unfortunately, it’s not working.

See lines. I think they should reproduce what you are explaining me (but I’m likely wrong). I checked that return WrapperPluto(out) is executed but nothing gets displayed in the notebook.

https://github.com/GenericMappingTools/GMT.jl/blob/master/src/gmt_main.jl#L116

https://github.com/GenericMappingTools/GMT.jl/blob/master/src/common_options.jl#L2741
https://github.com/GenericMappingTools/GMT.jl/blob/master/src/common_options.jl#L2751

Victory. It works now.

It happens that the above return WrapperPluto(out) line was not making it way all way down to Julia (it was in a function called by other functions). Now that I made survive down to the root, it works.

Thanks a lot for the help.

How can I display a .tif image? I tried the MWE by @fonsp using Base.show(io::IO, ::MIME"image/tif", w::Wow), and it only displays the path string when I call it with Wow(img_path).

Answering my own question … just plot the image.

using Images
using Plots

img_path = "path/to/image.tif"
img_data = load(img_path)
plot(img_data)

What also works is loading both FileIO.jl and Images.jl:

begin
    using FileIO: load
    # Images.jl contains a `show` method for PNG images
    using Images: Images
end
load(download("https://julialang.org/assets/infra/revise.png"))

image

3 Likes