How does Pluto decide what to show graphically?

rpmuller · March 26, 2021, 5:37pm

I’m having fun writing all my code in Pluto, and I’d like to add some graphical bells and whistles. In Python/Jupyter, I’ve done this by overloading repr_html or repr_svg. Is there something similar in Julia? I see as_html and as_svg in the PlutoUI tutorial, and I also see a lot of references to overloading show with a specific MIME type in the PlutoUI/Images.jl module. Can someone give me pointers on where to start? If this is something where I should leverage existing code rather than writing this myself, please also let me know. I fear I’ve reached the end of my productive googling.

rpmuller · March 26, 2021, 5:59pm

Discovered the following doctest in the PlutoUI source code:
Show(MIME"text/html"(), "I can be <b>rendered</b> as <em>HTML</em>!")

Tried the analogous
Show(MIME"image/svg+xml"(), txtsvg)

with txtsvg equal to a simple SVG string, but it didn’t work .

rpmuller · March 26, 2021, 6:32pm

Also found Compose.jl, which looks great, but might be more complex than what I’m looking for.

lungben · March 26, 2021, 6:39pm

cc @fonsp

rikh · March 26, 2021, 9:07pm

If I understand correctly, this is a typical case where multiple dispatch is used. Basically, each library defines it’s own

function show(io::IO, m::MIME"some_mime", x::T)

where some_mime can be MIME"text/html" and the type T is some object defined by the library. Then, Pluto will just call

show(io, m, object)

and Julia figures out which show method needs to be called exactly. (Julia will try to do this by the most specific method possible or fall back to printing it as a string if no specific method for some type is known.)

For example, in Gadfly:

function show(io::IO, m::MIME"image/svg+xml", p::Plot)
    buf = IOBuffer()
    svg = SVG(buf, Compose.default_graphic_width,
              Compose.default_graphic_height, false)
    draw(svg, p)
    show(io, m, svg)
end

You can find many more examples in other libraries.

Note that the type, here, is important. If you passed a simple SVG string as a literal String, then Julia will print it as a string.

To answer this question, what kind of objects do you want to show?

rpmuller · March 26, 2021, 11:08pm

Thank you very much for your response. To answer what kind of objects I want to show, there are several, but I’m initially interested in being able to make graphical descriptions of quantum circuits, the kind of thing that Yao does here. I realize this already exists, but I’m trying to understand Julia and Pluto a little bit more by rewriting some of the utilities I’ve written in python in the past, and I had a cute little routine that used matplotlib to plot out quantum circuits in just a few hundred loc.

Like I said, I did that one using matplotlib, but I could have just as easy done this in pure SVG. I had another little python script here that overloaded repr_svg to draw svg figures in Jupyter notebooks. I know that the ProfileSVG module uses SVG to visualize the FlameGraphs, so I thought that could be a good approach, but I got lost in looking through the source to see how they manage to display their SVG files.

I see that Yao uses Compose.jl, which looks beautiful, and may be the best way to go, but before I use someone else’s solution, I’d like to understand the mechanics a little bit better. It was a real lightbulb moment for me when I realized that all of those cool Jupyter notebook plugins just overloaded repr_svg or repr_html, and it was fun to have that kind of fluency with the Python/Jupyter code.

For the record, what I tried to do with the SVG string was to define the SVG string:

txtsvg = """<svg height="100" width="100">
  <circle cx="50" cy="50" r="40" stroke="black" stroke-width="3" fill="red" />
</svg>"""

and to then call

Show(MIME"image/svg+xml"(),txtsvg)

but that didn’t work. I also don’t understand why there are function parens after the MIME"image/svg+xml"() string literal.

rpmuller · March 26, 2021, 11:52pm

…and after some more googling around, this section of the manual on custom pretty printing is what I needed to read.

What I hadn’t understood is that I need to define not only show(io,obj) but other show types as well, for example show(io,::MIME"text/plain",obj) and show(io,::MIME"text/html",obj).

The latter appears to be called to display objects in Pluto notebooks. How do I control when the MIME"image/svg+xml" is displayed?

For example, does Pluto default by trying to show the object as an image, and then fall back to showing it as html?

For reasons I don’t yet understand, when my objects lacked a MIME"text/html method, Pluto didn’t call the normal show(io,obj), but fell back to a 3-argument method of AbstractArray.

rikh · March 27, 2021, 8:49am

As far as I understand, the MIME types state how the browser shows the output data. So, for an object of type T, it doesn’t make sense to define two different output MIME types.

See the documentation of show via ?show. It depends on the type of the third parameter (the object). The behavior is not really Pluto specific. Pluto just calls show(io, mime, x) and Julia calls the most applicable show method.

rpmuller:

For the record, what I tried to do with the SVG string was to define the SVG string:
txtsvg = """<svg height="100" width="100">
  <circle cx="50" cy="50" r="40" stroke="black" stroke-width="3" fill="red" />
</svg>"""
and to then call
Show(MIME"image/svg+xml"(),txtsvg)

The type of textsvg is a string, so show(io, mime, obj::String) will be called. Instead, you could define your own type

struct MySVG
   x::String
end

and your own show method

function show(io::IO, m::MIME"image/svg+xml", obj::MySVG)
    [...]
end

rpmuller · March 27, 2021, 4:53pm

Thanks again for your responses. I think I have the basics now, and I’ll play around defining some types to see how to draw them. Thanks!

rpmuller · March 27, 2021, 7:28pm

Sorry, I thought I had this, but I’m still being dense.

You say:

rikh:

The type of textsvg is a string, so show(io, mime, obj::String) will be called. Instead, you could define your own type
struct MySVG
   x::String
end
and your own show method
function show(io::IO, m::MIME"image/svg+xml", obj::MySVG)
    [...]
end

I’m just being stupid, but everything I put inside the show() block comes out as a string. I can write a proper svg xml file, and can test that the file is written correctly. But the only way I know how to include this is to cast it to a string and then write/show/print it as a string, which doesn’t work for the reasons you cite above the section I quoted.

E.g., if I have

function Base.show(io::IO, m::MIME"image/svg+xml", obj::MySVG)
    show(io,m,obj.x)
end

and

MySVG("""<?xml version="1.0" encoding="utf-8"?>
<svg width="40" height="40" xmlns="http://www.w3.org/2000/svg">
<g style="fill-opacity:1.0; stroke:black; stroke-width:1;">
  <line x1="5" y1="5" x2="25" y2="25"/>
</g>
</svg>
""")

it doesn’t work. Any hints as to what I’m doing wrong?

Thanks in advance for your help.

rpmuller · March 27, 2021, 7:36pm

Nevermind. Turns out that

write(io,obj.x)

in the show() works. print also works here, but show doesn’t. I think the difference is that when I call show() I’m asking the interpreter to reinterpret this, and it sees and handles a string. Whereas if I just write the text to io, it ?? does the correct thing???

rikh · March 27, 2021, 8:40pm

I’m happy to hear that you got further Unfortunately, I’m unsure what is now the unexpected behavior with the show. Could you post a MWE (Please read: make it easier to help you)?

rpmuller · March 28, 2021, 12:01am

Sure thing. Following the code you posted…

struct MySVG
   x::String
end

function Base.show(io::IO, m::MIME"image/svg+xml", obj::MySVG)
    write(io,obj.x) # also works with print(io,obj.x), but not show(io,obj.x)
end

MySVG("""<?xml version="1.0" encoding="utf-8"?>
<svg width="40" height="40" xmlns="http://www.w3.org/2000/svg">
 <g style="fill-opacity:1.0; stroke:black; stroke-width:1;">
  <line x1="5" y1="5" x2="25" y2="25"/>
 </g>
</svg>
""")

See the comment after the write command on the 5th line of code: This displays an SVG image in Pluto if I use write(io,obj.x) or print(io,obj.x), but not if I use show(io,obj.x). The best guess I can make as to why this happens is that if I call show, the interpreter goes through the whole cycle again, decides it’s handling a string, and calls something else, whereas with print/write it just prints straight to io.

Does that make sense?

Mattriks · March 28, 2021, 12:19am

julia>?show

show(io::IO, mime, x)
...
it is only necessary to define a new `show` method for T, via: 
`show(io, ::MIME"mime", x::T) = ...`, where `mime` is a MIME-type string and
 the function body calls `write` (or similar) ...

You could also make use of Compose’s types/functions. Note Compose will add it’s own svg header to the file.

using Compose
img = SVG(40mm, 40mm)
# img = SVG("test.svg", 40mm, 40mm)
write(img.out, """<g style="fill-opacity:1.0; stroke:black; stroke-width:1;">
  <line x1="0" y1="0" x2="40" y2="40"/></g></svg>""")
img

rpmuller · March 28, 2021, 1:02am

Thanks!

L_Adam · August 29, 2021, 2:31pm

as you’ve figured it out, just implement show(io::IO, m::MIME"image/svg+xml", c) . It’s simpley

using Compose 

struct DrawConfig
	c::Compose.Context
	width
	height
end 

import Base

function Base.show(io::IO, m::MIME"image/svg+xml", d::DrawConfig)
	backend = SVG(io, d.width, d.height)
	draw(backend, d.c)
end 

DrawConfig(compose(compose(context(), rectangle()), fill("tomato")), 4cm, 4cm)

done, and pluto just call show for you

the good thing about that is you never needs to touch svg. Until you want native interaction from browser, that you have to inject some attributes and more likely, js code script element, this could be done by combining Compose, specically draw part, with some xml package, like EzXML.jl