AI-Generated Documentation(Deepwiki) + Julia

Offtopic
, , ,
1

AI-Generated Documentation :
( Deepwiki - powered by Devin, an AI coding assistant )

+LLVM: https://deepwiki.com/llvm/llvm-project

see more ( search for .jl )

I believe this is only the first wave of AI-generated documentation:

  • Could we create documentation that is even more specific to Julia?
  • Is there another option, preferably open-source?
  • Could we generate beginner-focused tutorials from the tests to help new programmers get started with Julia?
2 Likes
2

I tried to generate the docs for the StreamSampling.jl package, and while the general description is okay (but I have similar and hopefully higher quality human-made docs for that), the details are really wrong e.g.

Screenshot from 2025-04-26 23-06-39
Screenshot from 2025-04-26 23-06-39672Ă—532 25.8 KB

is totally nonsense, I wouldn’t take this “docs” seriously

2 Likes
3

Can it answer specific questions about the package correctly?

IMO it makes sense that the diagram is nonsense because it’s already quite unclear what “memory efficiency” and “time efficiency” mean. It can’t actually run code and do measurements, so it comes up with bullshit, probably because the prompt forces it to draw some kind of diagram. Does it get the facts wrong as well? Like if you ask how to do particular things using your package, does it “hallucinate” nonexistent API or pass incorrect arguments to functions etc?

I asked how to use variational inference in Turing, and at the first glance it looks fine.

4

yes, I think that it should be able to suggest code which respects the general API most of the time. But consider that the package is small so it’s not that hard in this case. Actually I don’t really understand its usage given that with many state-of-the-art LLMs you can already ask them to base their response by considering the documentation and the repository of the library. So for packages which already have decent docs I don’t get the value proposition at all. Moreover, many details which were not already in the real docs in the case of my library are wrong.