Run the Gemma 2 9B LLM in Julia and without a GPU?

findmyway · April 2, 2025, 11:01am

Gemma3 implemented in pure Julia without GPU :

github.com/JuliaGenAI/Master-Deep-Generative-Models-in-Julia

04_gemma/gemma3.jl

main

using FileIO
using ImageCore: channelview
using SafeTensors: load_sharded_safetensors

import HuggingFaceTokenizers as HFT
using NNlib: batched_mul, gelu_tanh, conv, DenseConvDims, softmax, meanpool, PoolDims
using Statistics: mean, var
using LinearAlgebra: triu, tril
using ProgressMeter

#####
# Processor
#####
struct Processor
    sliding_window::Int
    boi_token::String
    eoi_token::String
    image_token::String
    mm_tokens_per_image::Int

This file has been truncated. show original

Topic		Replies	Views
LLaMA in Julia? Offtopic	13	3773	August 7, 2023
Gradient of llama2 computed by Zygote seems to be incorrect Machine Learning	6	778	September 13, 2023
Sequence language models in Julia Machine Learning	6	429	August 18, 2025
[ANN] Julia LLM Leaderboard - Help us make it more relevant for every day problems! Package Announcements announcement , generative-ai , prompting	22	3748	April 5, 2024
LLM AI just for Julia? A proposal: Julia plus science LLM? General Usage machine-learning	4	1675	June 24, 2023

Run the Gemma 2 9B LLM in Julia and without a GPU?

Related topics