# How to load models from HuggingFace with Transformers.jl

**URL:** <https://discourse.julialang.org/t/how-to-load-models-from-huggingface-with-transformers-jl/100573>\
**Category:** Machine Learning\
**Created:** [June 19, 2023, 7:29pm UTC](https://discourse.julialang.org/t/how-to-load-models-from-huggingface-with-transformers-jl/100573 "2023-06-19T19:29:56Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [June 19, 2023, 7:29pm UTC](https://discourse.julialang.org/t/how-to-load-models-from-huggingface-with-transformers-jl/100573/1 "2023-06-19T19:29:56Z")

</div>

Hi All,

I quite like the Transformers.jl. I do not aspire to train or finetune LLMs, but I am more interested in learning prompts. I am a bit lost with loading models from the HuggingFace.  
Here are few questions:

1. Why are models named with symbols? e.g. `:gpt2` ?
2. How do I find the names of models? For example “gpt2:lmheadmodel”?
3. How do I find the model is supported? Is there any tutorial, how to add unsupported model?
4. Has anyone tried to run the model on multiple GPUs, if it does not fit one?

Let me give you a concrete example.  
Let’s say that I would load this model [EleutherAI/gpt-neo-125m · Hugging Face](https://huggingface.co/EleutherAI/gpt-neo-125m) .

I can load the tokenizer as

```julia
hgf"EleutherAI/gpt-neo-125m:tokenizer"

```

but I do not know, which heads the model offer.  
I figured out that I can probably load everything as

```julia
hgf"EleutherAI/gpt-neo-125m"

```

which seems to return a tuple `(tokenizer, model)` and “ditch” the naming.

Nevertheless, questions 1,2, and 4 remains. The lib is great. So much hard work and so nice.

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [June 20, 2023, 9:57am UTC](https://discourse.julialang.org/t/how-to-load-models-from-huggingface-with-transformers-jl/100573/2 "2023-06-20T09:57:16Z")

</div>

The loading of model using `hgf"EleutherAI/gpt-neo-125m"` is not complete, as it is missing a `cls` head used to compute logits of tokens. The problem is obviously in loading, since `hgf"gpt2"` loads the tokenizer and the model, but does not load `cls` layer which computes logits. Contrary, if I load the gpt2 model as `hgf"gpt2:lmheadmodel"`, then the model contains a `cls` layers computing logits.

But this does not work

```julia
hgf"EleutherAI/gpt-neo-125m:lmheadmodel"
ERROR: Model gpt_neo doesn't support this kind of task: lmheadmodel

```