The problem here stems basically from a problem with cuTENSOR and Tensoroperations. At some point, cuTENSOR 2.0 got released, which broke a lot of things, including the TensorOperations wrapper. It took some time to figure out this update, and sadly the update for this, while implemented in the master version, has not gotten released yet, as this will be bundled with some breaking changes on the TensorOperations side as well.
Depending on your specific needs, I would advise to have a look and see if you can either downgrade CUDA (or add this dependency last), and maybe have a look at GitHub - lkdvos/TensorOperationsTBLIS.jl: Julia wrapper for TBLIS with TensorOperations.jl. I am not sure how the compatibility requirements for your specific case will play out, but it might require just a little more time from our side (for which I apologise!) before we get TensorOperations v5 functional. (At that point, I will include a release of TensorOperationsTBLIS compatible with this update as well)
1 Like