I agree, but it is my impression that a lot of new users eventually learn a bit of git workflow and start making contributions, including docs fixes, for subsequent packages they encounter. At least I get quite a few tiny, but incredibly useful docs PRs.
I don’t think it is simply about ability/willingness. Investing in documentation only makes sense at some later point in a project’s lifecycle. A lot of Julia packages are in a stage where the author wrote something for his/her own needs, and is experimenting with various approaches before even deciding if it the whole idea is worth pursuing, but at the same time made the code publicly available.
This is one of those situations where the glass being half empty or full depends on one’s perspective: one can complain that such projects are unpolished and underdocumented, or just appreciate that at least something is available and understand that it is a gift of someone’s time and effort already.
8 Likes
I think that reducing the barrier for contributions is a (the ?) major topic here.
I often wish I could make comments directly in a (documentation) page (ideally in a WYSIWIG way like you can comment a pdf or a google doc) while I am reading it. These comments can be either typos, or indication that a given sentence is really difficult to understand, or that a given term is not (yet) defined, or an entire copy/pasted Julia MWE.
Obviously, PRs are greats but they are only natural and easy for (experienced) package developers. End-users will rarely take the time for this. Git hub “issues” are far easier but are not WYSIWIG and you have to take additional time to specify what and where you find unclear/wrong in a documentation.
This would require to develop an appropriate dynamic web app for documentation but this large piece of work could be used by all packages and amortized by the community (as well as the server infrastructure). If you consider a tool similar to shared google documents, where you can “accept” comments, it could be also a good way to merge this kind of instantaneous PRs via a simple click.
This involves more work from the package maintainer than proper PRs but I guess that a lot of them would be happy to have more feed-back anyway.
7 Likes
As pointed out here countless times before, simple changes to the docs can be made very easily directly using Github with a point-and-click interface. Documenter.jl has an “Edit on Github” link on top of every page, which takes you there to the relevant markdown sources, which you can edit (and preview, etc). Technically it will make a PR, but it does not require any more technical expertise than operating a microwave oven.
FWIW I find the following two themes repeating in these discussions:
-
It rarely ever occurs to people who complain about docs that they are written other Julia users just like them. The corollary is that effort expended on complaining about docs can be allocated to improving the docs instead. Everyone can contribute. You don’t need a license.
-
The majority of people who complain about the technical difficulty of improving the docs have not actually devoted even a very minor amount of effort to finding out if this is the case. If they had, they would have just discovered that it is very easy.
10 Likes
A lot of this assumes users read docs, think that something is missing (hopefully create an issue), and make a PR fixing it. Maybe we should be focusing on getting people through the first two steps of this before we worry about educating everyone on documentation PRs. This would at least help keep track of documentation errors/insufficiencies that need to be addressed.
I think the best approach may be to encourage users to file an issue anytime they find difficulties with a package, and use that to focus people on the documentation. I know it sometimes comes across as impatient when asking if someone read the docs when they file an issue, but if we never ask if they read it then we don’t know if the documentation is helpful.
2 Likes
Thank you for mentioning the “Edit on Github” that I have missed/skipped a large (countless) number of time while reading the docs. I have learned something.
The fact that I have missed this functionality countless times should probably be interpreted as a personal disorder. Nevertheless, the fact that it has been pointed out countless times to a countless number of different readers may indicate an ergonomic issue. I personally do not consider this (very interesting) functionality to be as accessible as a right-click comment on a google doc.
Anyway I apologize for my ignorance that seems to exasperates you.
3 Likes
Keep in mind that the time of most package maintainers is a scarce resource. There is of course always demand for getting support without reading the docs, or making any kind of effort whatsoever, but most open source projects provide this as a paid service.
While most packages are happy to provide support in issues, at least checking the docs (if they exist) is the minimum courtesy that most open source projects implicitly expect before opening one. It is of course fine if something was missed or unclear in the docs (which is an opportunity for improving them), it’s the intent that counts.
1 Like
I completely agree and the caveat to that statement is that people actually tried to resolve an issue themselves. Perhaps we need to be using issue templates that say some of this as politely as possible. I often don’t have time to really say much more than “have you looked at …”. I’m afraid it may come off as rude but most of the time I would really like to know if someone has because then that documentation isn’t working. If someone hasn’t read the appropriate documentation were they not able to find it? If not then there needs to be better directions to the proper documentation.
1 Like
I agree. I have fallen into your cycle myself.
I started looking into Julia a couple of weeks ago after working quite a bit with QT Creator and other systems. I found the Julia documentation sparse in good, progressive documentation.
And, in some places, items where not documented where I expected to find them. Specific example: I wanted to do a for loop from a to b by step_c.
I know how to do that in C/C++ — “for (i =start; I <= end; I = I + step)”–(simplified for clarity, not good code). And I tried that–no go obviously.
I looked in the Manual/Control Flow/Repeated Evaluation: Loops section and found a number of examples for “for”, but none had the basic format of “for start:step:end”. I finally found a sample in another discussion group.
I like a lot of the things I see in Julia that really impress me and, do things the way I would do them if I was to design a language.
I commend the authors for every thing they have accomplished. I would like to see Julia succeed and wonder how we could go about making improvements in the documentation.
1 Like
Myself, as the majority, has suffer about the problem with non-friendly documentation in some on the packages I work (the official Julia documentation is great, by the way). In my opinion, the edit option in Github is very useful. However, sometimes the improvement should implies new sections in the documentation, and it is not so easy to do that, because it is not so easy to add a new section in the documentation (because it usually would require a new markdown file). Maybe some sections in the documentation to be more complete with the users (as a Example Section or similar) could allow them to improve them more easily (with the edit option in github).
I agree. I have fallen into your cycle myself.
I have too, of course, even though it’s also true that I’ve submitted a lot of documentation PRs for packages I don’t maintain. You certainly don’t have to (can’t) fix everything. To me this is mostly about distributing the expertise & workload in a way that accelerates the improvement of Julia as much as possible. And improving documentation—including harnessing the special gift that only newcomers can bring, which is their perspective as a newcomer—is a huge part of that.
And, in some places, items where not documented where I expected to find them.
When you have a lot to convey, putting the right information in the right spot is a hard (probably ill-defined) problem, though I don’t mean to imply that we can’t do better. To those who might be motivated to pitch in, let me post my new favorite link on writing documentation, shared with me by @c42f: https://documentation.divio.com/. It’s certainly a model I will follow the next time I sit down for a major documentation project.
8 Likes
I have gotten use to Qt’s documentation and it comes closer to combining example sections with reference sections but it is still a struggle.
One documentation method that I used in the past was IBM’s Keyword In Context. The index of a manual gave all references to a word in manual used in specific contexts.
The manuals where usually structured in a recursive style of Introduction/Body/Technical Details.
When I wanted to look up something, I looked in the index for the keyword and go to references where that keyword was used in a particular context.
Made for very quick look ups.
1 Like
This is a pretty big issue. Sometimes I worry the community may inadvertently reward packages which are under documented out of utility to a seriously limitted population of users(1 out of a billion people can understand them) and not value those with good docs very highly.
I found this to be a gap in learning/getting established in Julia. Sometimes, I found myself writing my own versions of a package rather then figuring out how to use something pre-existing. Because I’ve wasted a lot of time learning someone elses code to find out that it was not complete, or did not do what I needed it to do. Which creates a weird kind of technical debt…
Some people (not common) make incomplete packages to stake a ‘claim’ to an area. Then someone else realizes “hey this is not useful and I don’t even understand the footing its on. So I’ll just make it my way” then we end up with three packages containing the same algorithms which may or may not be documented correctly.
I don’t mean to blow this out of proportion - Docs for packages are markedly improved now days! But, this still exists, and it’s not a “code smell” it’s an “infrastructure/community smell”. I think we need to kind of take steps back and ensure our community isn’t supportive of “solo heros”, “flag planting”, “exclusivity of contributions”, and “in crowd vs others” to grow past some of this. Otherwise we won’t round out as a technology and could fall into some of the traps(inevitable maybe) most communities like this have.
The community overall is good - but we could lose out on serious technical contributors if we aren’t careful to avoid these issues. A sober look on how we treat others contributions, the contributors themselves, and our own contributions, could probably do us all some good.
7 Likes
I’m currently redesigning a lot of documentation after reading that divio link. It’s very useful.
Perhaps we should have a documentation style guide like Invenia’s BlueStyle guide. It would at least be a place to start improving quality of documentation across the community as a whole.
5 Likes
What if we had a documentation coverage github badge option?
How might that work? I can think of a few KPI’s for this - but I’d be curious what other’s think.
2 Likes
I think it’s an interesting idea, but what would it mean exactly. Does it mean what percent of code coverage is due to documentation based tests or does it mean that it adheres to a specific standard?
2 Likes
So how I potentially see this working is the following…
# API User Score
1. Hook into Documenter.jl to find all documented functions: fn_documented
2. get all Functions that are exported, as fn_exp
3. Find cardinality of the intersection of 1&2
4. Score = ( intersection / |fn_exp|)
# API Contributor Score
1. Hook into Documenter.jl to find all documented functions: fn_documented
2. get all Functions that are exported, as fn_exp
3. Find cardinality of the intersection of 1&2
3. Find cardinality of the union of 1&2
4. Score = ( intersection / union)
# Example's/tutorials score
1. get all Functions that are exported, as fn_exp
2. For each function
2.A. Hook into Documenter.jl to find all code-blocks.
2.B. counts = does codeblock contain a call to fn_exp? 1 : 0
3. Score = counts / fn_exp
Surely we could get more complicated than this too, but this seems feasible.
1 Like
@anon92994695, an interesting thought! We have the discussion here again and again in intervals. There seems to be a real need and this should not always be put down with the same arguments (do a PR, get involved, etc.). I had expressed the idea of KPI’s some time ago and the resonance speaks for itself…
1 Like
I think it might be better to start with documentation style guides and instructions first. It’s difficult to create a scoring system for something that might mean something different to each of us.
1 Like
I think good documentation is inherently hard thing to achieve. Not because of lack of trying or attention. But because fundamental mechanism of how humans can maintain focus and be good at something.
I’m working as full time game designer. And learned in my career that good writer is far rarer than good programmer or good artist. And writing is not necessary cognitively more demanding skill than programming. But It’s nature makes writing is much harder to learn and hone as you write.
For anyone to learn anything, one needs a feedback. And feedback from others are rare and far between. One need a feedback from oneself, whether one did a good job or not. When one is coding, he can set goal all by himself, and measure his achievement quite objectively. And he can keep doing this until his achievement satisfices his own criteria of a goal. And feel good about himself.
But when one writes technical documentation or creative writing. One cannot do this. Did he produced many pages? well yeah, but was it good documentation? one can’t know it by oneself.
And that’s why writing documentation is not engaging task. And harder to draw energy for writing than coding. Because one cannot feel satisfaction like passing test script with a writing.
6 Likes
Nice! yes I think KPI’s or statistic based figures of merit could help us all become more aware of the holes in the ecosystem. As an aside - It may be great to have the fastest package for doing X_Y_Z in the world, but if it’s constantly broken, and all of the nonbenchmarked parts of the codebase change syntax every 2 weeks - well a user might want to know that before selecting it as a tool for a project. Especially if it’s their introduction to Julia. KPI’s can offer glimpses of that.
Not to say we have a lot of false promises here, but I think if we honestly look at some of our deficits from a user’s perspective, we may change what we do with our contributions and how we maintain them.
We have been encouraged to cut 1.0 releases (probably so people on the outside felt Julia’s ecosystem was more mature), but what if we were encouraged to package maintainers to get >80% on their documentation coverage (should be pretty easy with doc strings and a single example)? Which would make the greater impact long term? I think the docs, but I may be wrong.
1 Like