I would like to create an implementation of the Transformer NLP architecture, more specifically I am trying to implement a BERT Summarizer. I have been using the Transformer.jl (v 0.1.7) package and have followed the documentation tutorial, however I am unable to get the example to run for a following reasons:
- The line in the tutorial
enable_gpu(true) # make
todevicework on gpu
returns “CUDA not functional”, even though when it is (as running CUDA.functional() returns true). I have made sure that libcuda.so is visible.
- The code
vocab = Vocabulary(labels, unksym)
which references Transformers.Basic.Vocabulary which does not exist. I have scoured the Transformer.jl source code and cannot find the Vocabulary function.
I am assuming that the documentation that is available might not be up to date with the current version of the package, hence the need for assistance. Any feedback or advice would be greatly appreciated.