Run the Gemma 2 9B LLM in Julia and without a GPU?

findmyway · April 2, 2025, 11:01am

Gemma3 implemented in pure Julia without GPU :

github.com/JuliaGenAI/Master-Deep-Generative-Models-in-Julia

04_gemma/gemma3.jl

main

using FileIO
using ImageCore: channelview
using SafeTensors: load_sharded_safetensors

import HuggingFaceTokenizers as HFT
using NNlib: batched_mul, gelu_tanh, conv, DenseConvDims, softmax, meanpool, PoolDims
using Statistics: mean, var
using LinearAlgebra: triu, tril
using ProgressMeter

#####
# Processor
#####
struct Processor
    sliding_window::Int
    boi_token::String
    eoi_token::String
    image_token::String
    mm_tokens_per_image::Int

This file has been truncated. show original

Topic		Replies	Views
LLaMA in Julia? Offtopic	13	3728	August 7, 2023
Gradient of llama2 computed by Zygote seems to be incorrect Machine Learning	6	769	September 13, 2023
OpenAL GPT Machine Learning question	0	802	March 22, 2019
Generate independent LLVM code from Julia General Usage question , llvm	0	690	April 5, 2017
On Machine Learning and Programming Languages Machine Learning	48	8907	January 25, 2018

Run the Gemma 2 9B LLM in Julia and without a GPU?

Related topics