Can someone summarize state-of-the-art workflows for generating presentable finished documents?

Hello. I apologize if this question isn’t well-formed for the forums.

I am wondering what the state of the art is for mixed plot/results-and-text documents intended for display as HTML (web) or PDF.

In the old days, before I ever used Julia, I had two workflows that I would have used to answer this question relative to R:

  1. Write your document in R Sweave / eventually in knitr, .Rmd, compile it to HTML, post to web
  2. Write your document in R Sweave / eventually in knitr .Rmd, compile to TeX then pdflatex to PDF.

Since then I’ve been overwhelmed by the apparent explosion in options. I think this is really cool; it seems like more things offer more support for more other embedded things. E.g., a Jupyter notebook can have Markdown and LaTeX and even both embedded R and Julia. The Pluto.jl platform has built-in interactivity. And this is only scratching the surface.

What I am wondering is if someone far more knowledgeable about (a) the breadth of these solutions and (b) the state of the art can offer a somewhat opinionated summary of the current state of the art workflow for various tasks.

For example, suppose my main desire is to write a blog post that is primarily words for a non-technical audience, but include a couple of Julia-generated graphs. A valid claim would be “don’t try to embed anything, just generate your graphs outside the post and write the post in text, copy the images in.” Another might be “Don’t be silly, such-and-such workflow is the perfect workflow for this,” e.g. write it in some Julia analogue of R markdown (of which I’m unaware) and only in a couple places will I have to fence off some Julia that generates the graph.

Suppose separately my main desire is to write a PDF that is primarily math-heavy. One valid claim would be “write it in TeX and name your included images intelligently, write the Julia script to generate those images”; another would be “here is the right workflow for generating TeX->PDF outputs from a mixed Julia-and-TeX file.”

I hope this question makes sense. I’m open to pushback on it; essentially my challenge here is that I think I’ve failed to keep up with all the developments in integrated document, online, notebook formats, etc.

Thank you!

6 Likes

Not trying to answer your whole question. But just so you know, R Markdown is actually a misnomer, since it supports about 40 languages including Julia.

4 Likes

Indeed I did not know that. Cool–thanks!

This is one of the reasons why all my Julia blog posts are written in RMarkdown. :blush:

3 Likes

And may I follow up by asking–do you have a certain tooling that you like around this, e.g. do you typically write documents in VSCode and knit to preview in HTML, or do you use one of those notebooks-embedded-in-VSCode formats, or … ?

1 Like

If I have to document a simulation or the results of running julia and the math behind it, I will sometimes use a library I wrote to programmatically create Latex output, including math , figures and tables etc to generate either a slide deck or even a full report.

I can even write the report in Lyx and then simply include the latex output from my simulations.
There are similar capabilities for matlab and python, e.g.

pyt2tex, etc.
pytexit — pytexit 0.4.0 documentation .

It’s not too hard to do it on your own. I’d release it but unfortunately I don’t control it’s license.

I use neovim together with tmux to be able to send code to the console. Then i have a Hugo serve running which compiles on every change and it shows up in my browser. For PDF the workflow is the same except i use a different output and viewer.

This is what I do, mainly because otherwise

  1. rebuilding a whole site would take enormous amounts of time,
  2. I would have to be very organized about manifests etc if I wanted the code to work a couple of years later. With the graphs generated, I always have a backup.
1 Like

Just to add a few more to the discussion that don’t seem to have been linked directly, there’s:

I’ve probably missed off a couple, but there’s a lot of options, and since generating documents is a pretty broad area, it’s great that we have so many options.

7 Likes

This is a matter of opinion and perhaps also where you’re at in your workflow, but if you’re mainly interested in PDF output and are already proficient in LaTeX, have your own scripts written over several years, occasionally use TikZ and PGFPlots, as I do, here’s my opinion: I prefer to generate figures with Julia and import them with \includegraphics, for a variety of reasons: I find it more convenient for reusing the images from article to slides to lecture notes, I prefer to not have nice figures I’m likely to reuse buried in a document I’m not going to reuse: I often find myself searching through old documents looking for the code for that one figure I want to reproduce, but can’t quite remember where exactly it is. If I keep figures in their own files (as I do now), it’s a lot easier to navigate (I can quickly preview pdf figures in my file manager). Also with this workflow you can produce a graph with R, another with Python, another with Julia and import them all into the same document: It’s much harder to compile one markdown or sweave document that calls R, Python, Julia at once. Also I find that plotting packages evolve faster than LaTeX and some markdown documents I made four or five years ago now don’t compile and I’d have to go over all 200 pages of them to figure out which part is holding everything back.

However if you’re looking for interactivity with students or coauthors, Pluto is absolutely the way to go.

2 Likes

You can put tikz/PGFPlots figures in separate files that you can reuse. A nice thing is that they will adapt to your new document, be it report or presentation, with fonts, linestyles, aspect ratio, etc. And you can make minor modifications to it that would be impossible with an image file.

I normally keep a pdf version of the figure next to the tex/tikz file, so it’s easy to know what it contains.

4 Likes

Very good and useful
Thank you

Indeed! You’re right that if you want consistency in the fonts and other sizing features, it’s better that way. You have to load every package used by every figure in your main document. You’d also need a script to generate a separate PDF simultaneously. I’ve also done it that way. I just find my preambles get too big and if I need conflicting features (like line-spacing or whatever) I need to figure out whether to add to the preamble or to the local figure-file. But yes it does work pretty well that way. By the way (message to the OP), do look into PGFPlotsX.jl, a great package.

Tikzexternalize does this for you, while saving you from recompiling all figures on each build of the document, quite handy.

4 Likes

Sounds and looks awesome.