Franklin.jl -- How to Extract Properties of Specific Posts/Pages

General Usage
1

I’ve started using the Franklin package. I generally understand how it works, but am struggling with how to extract specific information about a post or page when writing an “hfun_” function for utils.jl. For example, if I have several posts in a folder, and each of them have specific page variables defined in them (date is the one I’m thinking of), how do I extract those page variables when looping through the posts’ filenames (or the posts themselves if that’s necessary)?

More generally, where does Franklin.jl store information about a site after the site is processed? For example, if I use Franklin.serve() to bring the site to life, how do I go about using Franklin’s functions to access information about the site and it’s components? I’m struggling with writing hfun_ functions because I’m not sure how to see what each line of the function is doing by using the REPL.

I’ve spent quite a bit of time with the documents but this is still confounding me. Any help is appreciated. Thanks much everyone.

1 Like
2

Hello again! I don’t know Franklin inside out, but I can make it do a few things.

Say you have this in your index.md file.

{{ postlist }}

And you have this function in your utils.jl file:

function hfun_postlist()
    list = readdir("pages")
    filter!(f -> endswith(f, ".md"), list)
    posts = []
    for (k, i) in enumerate(list)
        open(joinpath("pages", i)) do f
            r = read(f, String)
            # scruffy regex time
            pubdate = match(r"@def published = \"(.*?)\"", r).captures[1]
            title = match(r"@def title = \"(.*?)\"", r).captures[1]
            pushfirst!(posts, (title, DateTime(first(split(pubdate)))))
        end
    end
    io = IOBuffer()
    sort!(posts, lt = (a, b) -> a < b)
    for d in posts
        write(io, "<p><b>$(first(d))</b> $(last(d)) </p>")
    end 
    return String(take!(io))
end

When Franklin builds your site, and this function runs, the index.html page will contain (among other things) a bolded list of your post titles and publication dates, sorted alphabetically by title, stored in paragraph tags. Eg on my Franklin site you would get this:

Screenshot 2023-07-08 at 09.17.37
Screenshot 2023-07-08 at 09.17.37590×652 48.6 KB

(Obviously you’d be adding links and formatting the dates etc.)

As to the more general question - where is information stored - the __site folder contains all the files in the built site. I suppose you can run any scripts you like to access this outside of Franklin, but I think Franklin’s job is done when the site has been built - it is “a simple, customisable static site generator”.

3

Without regex and using the tools from Franklin, it should be the pagevar function, see

where several variables are extracted build a similar list, and one of the example files is https://raw.githubusercontent.com/kellertuer/kellertuer.github.io/master/projects/graph-laplace.md and the result is Ronny Bergmann | Projects corresponding markdown file is just {{projectslist}}.

3 Likes
4

Yes, I should have started using pagevar by now :slight_smile:

1 Like
5

indeed for the use case mentioned the replies above are correct, pagevar is what you want, there’s an example here: Franklin FAQ , you can see the source code here: https://github.com/tlienart/Franklin.jl/blob/5629fc51904f454b299c020f60061ca0701cafcc/demos/utils.jl#L41-L56

indeed all content is generated in __site but if you want access to the state at the end of the generation process then there’s essentially two things you can look at (but that are not exposed and not really meant for the user to use, see also note about new version further down):

Franklin.GLOBAL_VARS
Franklin.LOCAL_VARS

if you do serve(..., cleanup=false) these object will remain and you can inspect them.

  • GLOBAL_VARS contains all the global variables as well as a bunch of general state variables that Franklin uses site-wide
  • LOCAL_VARS is a temporary container for everything that’s used on a single page (local variables).

you should not use those directly, using globvar, locvar or pagevar is the expected way to go to retrieve respectively a global variable, a local one (from the page on which it’s defined) or a local one from another page.

A note on the future

(I’m writing it for the sake of indicating that I’m well aware of the clunkiness of some of these things and that serious work has gone into re-designing it – if you’re curious you can test it out at Xranklin.jl).

In the future there’s one container for the global context (~ the GLOBAL_VARS above) and a list of local containers for each of the individual contexts associated with each of the pages (~ the LOCAL_VARS but keeping all of them). These containers are actually exposed to the user so you can do whatever you want with them and while the globvar, locvar, pagevar api will remain, you can also directly interact with those containers.

If you’re wondering why this wasn’t the case in the first place, there’s a slightly tricky graph dependency problem with page variables. As soon as you allow pages to depend on other ones (e.g. query their page variables) then the order in which you process pages matters and, potentially, you may have to re-process some to be sure to have all the information.

Initially there wasn’t the possibility to query across pages, this was added (not very well), it kinda worked, then serious bugs started appearing, which required the introduction of the clunky @delay macro which made sure that some pages were processed twice to ensure that dependencies were resolved, long story short it’s messy. One of the main improvement of the next version is to fix that stuff (and remove this @delay thing).

3 Likes
6

Thank you all. With further noodling and tlienart’s response, I believe I answered my ambiguous question.

I was hoping that there would be a way to run individual Franklin.jl internal functions in the REPL using contents of the project folder before running serve(), so (for example) there would be a way to test an hfun_postlist function and its internal steps as I was creating it. I tried this several times using simple functions (e.g., pulling a title set as a page variable), but always an errors were thrown.

For example, I set a “title” variable in a post having a relative path of "“posts/2022-07-11-musique-bretonne.md”. I then ran Franklin.pagevar("posts/2022-07-11-musique-bretonne.md", "title" in the REPL, which according to the docstring (below) should have returned a string representing the title. It throws this error:

Stacktrace:
 [1] pagevar(rpath::String, name::String; default::Nothing)
   @ Franklin ~/.julia/packages/Franklin/J8Nru/src/utils/vars.jl:265
 [2] pagevar(rpath::String, name::String)
   @ Franklin ~/.julia/packages/Franklin/J8Nru/src/utils/vars.jl:245
 [3] top-level scope
   @ ~/Franklin/SSS/scratch.jl:36

When I look at the Franklin code referenced in the error, I see it contains references to variables that I assume are created and defined when Franklin builds the site. That being the case, I wouldn’t be able to use the Franklin pagevar function as a “freestanding” function in the REPL. If so, it seems the only way to test any “hfun_…” function would be to put it in “utils.jl”, use the {{hfun_...}} format somewhere on a page, build the site, and see if it returns the correct result. This seems to be the case given tlienart’s response.

By the way, bravo tlienart for your work on Franklin! I will check out the Xranklin.jl site to see what’s coming.

I’m an amateur trying to keep my brain spry in retirement, so I’m sorry to take up the time/energy of “real” coders. Know that your help is greatly appreciated.

1 Like
7

Hello Steve,

You’re almost there, it didn’t work “just” in the REPL because the state is destroyed after the server shuts down unless you set cleanup=false (you can try some of this by doing serve(single=true, cleanup=false) which will do a single pass to build the whole website, and not destroy the state afterwards, then you can inspect Franklin.GLOBAL_VARS in the REPL for instance; however this is not what I recommend you do for your use case).

If you want to test a hfun_*, the easiest (in my opinion) is, as you said, to have a dummy page that calls it and gradually increase the complexity. So for instance you’d have a page test.md on which you’d only have

{{myfunction}}

and in utils.jl you’d have

function hfun_myfunction()
    result = begin
        try
            Franklin.pagevar("posts/2022-07-11-musique-bretonne.md", "title")
        catch e
            string(e)
        end
    end
    return result
end

then you can call serve() see what happens, modify the function hfun_myfunction watching the results on page /test/ etc. Basically a similar process as what you’d do in the REPL except that here the interaction is with the browser. And once you’re happy with what hfun_myfunction does you can remove the try-catch.

2 Likes