Run the Gemma 2 9B LLM in Julia and without a GPU?

findmyway · April 2, 2025, 11:01am

Gemma3 implemented in pure Julia without GPU :

github.com/JuliaGenAI/Master-Deep-Generative-Models-in-Julia

04_gemma/gemma3.jl

main

using FileIO
using ImageCore: channelview
using SafeTensors: load_sharded_safetensors

import HuggingFaceTokenizers as HFT
using NNlib: batched_mul, gelu_tanh, conv, DenseConvDims, softmax, meanpool, PoolDims
using Statistics: mean, var
using LinearAlgebra: triu, tril
using ProgressMeter

#####
# Processor
#####
struct Processor
    sliding_window::Int
    boi_token::String
    eoi_token::String
    image_token::String
    mm_tokens_per_image::Int

This file has been truncated. show original

Topic		Replies	Views
LLaMA in Julia? Offtopic	13	3612	August 7, 2023
Community Interest Check: LLMs from Scratch in Pure Julia Offtopic package	43	2604	January 31, 2025
LLM AI just for Julia? A proposal: Julia plus science LLM? General Usage machine-learning	4	1602	June 24, 2023
A new LLM benchmark for Julia programming Tooling generative-ai	0	198	May 21, 2025
[ANN] Julia LLM Leaderboard - Help us make it more relevant for every day problems! Package Announcements announcement , generative-ai , prompting	22	3471	April 5, 2024

Run the Gemma 2 9B LLM in Julia and without a GPU?

Related topics