Julia is a good language to complement AI for several reasons. For one, it is actually a language that AI seems to perform well at. However, there are several other reasons.
First, there are many reasons why giving access to a coding environment would be beneficial.
- Memory: While improving rapidly, memory is not a strong point for LLM.
- Factual information: While also improving rapidly, LLMs suffer from hallucinations as well.
- Calculations: LLMs suck at calculations. They currently struggle with answering questions like how many "r"s are there in the word strawberry or the standard deviation of a few numbers even though they have Ph. Ph.D.-level knowledge in many areas.
The solution? Give LLM access to an REPL, it could store information for a later time as it wants, and the LLM could store information in it. If the LLM needs to count how many "r"s are there in the word “strawberry” or the standard deviation of a few numbers, it could simply defer to code. Giving API access to Google search, a database, and so on would also allow the AI to find and process factual information. All of these could be done with a single unified system. No need to develop one system for searching this database and one system for that. Just have the LLMs learn the proper API.
However, why Julia specifically? Other than that LLMs seem to be good at Julia due to its consistency in design, there are many reasons as well.
- Supports for problem-solving: Whether the AI needs to augment its thinking with linear programming, differential equation solving, etc, Julia can help. With rich scientific libraries and performance, Julia is a good choice. This could augment the LLM capability in physics, chemistry, biology, etc. Given a logic puzzle? Instead of having an LLM struggle with it, the LLM could import the JuMP library and solve it.
- Speed: Julia is a language that is easy to make it fast. This would be very useful for tasks that require number crunching, state space search, and so on.
- Multiple accessible levels of intermediate representations: An AI might actually be better at manipulating AST than manipulating code in a human-readable format. With Julia, there are many levels of IR to choose from that an AI could be designed to generate.
There could be many more reasons. The thing is that Julia is a good language for an LLM to have, perhaps as one of the languages the LLM could choose alongside Python and so on.
But now, the issue?
Imagine an AI takes 10 seconds to generate code in a step in the chain of thought. However, the code compiles for a minute. And this is not just Julia but many languages, though Julia might be hit hard.
Currently, I’ve tried measuring one of my old C++ code. It takes 1 second to compile, and around 2-3 seconds for an AI to generate. The AI could be expected to be faster, especially if the paradigm shifts from having an AI generate one good answer to having an AI generate a chain/tree/whatever of thought. An AI might generate 10 intermediate steps for a single response, and each step might be optimized to take 1/10 as long. Here comes the big problem, compilation speed. It might take 2 seconds to generate all the code, but 10 seconds waiting for all the code to compile.
And since the AI-generated code is expected to perhaps solve complex, compute-intensive problems, giving up on optimization might not be the answer. This means if we do nothing, the lack of a fast optimizing compiler could become a huge bottleneck in AI.
And unfortunately, Julia would be hit pretty hard by it.