Assistance with Transformers.jl

Specific Domains Machine Learning
1

Hi

I would like to create an implementation of the Transformer NLP architecture, more specifically I am trying to implement a BERT Summarizer. I have been using the Transformer.jl (v 0.1.7) package and have followed the documentation tutorial, however I am unable to get the example to run for a following reasons:

  1. The line in the tutorial
    enable_gpu(true) # make todevice work on gpu

returns “CUDA not functional”, even though when it is (as running CUDA.functional() returns true). I have made sure that libcuda.so is visible.

  1. The code
    vocab = Vocabulary(labels, unksym)

which references Transformers.Basic.Vocabulary which does not exist. I have scoured the Transformer.jl source code and cannot find the Vocabulary function.

I am assuming that the documentation that is available might not be up to date with the current version of the package, hence the need for assistance. Any feedback or advice would be greatly appreciated.

Thanks
Stantin

1 Like