In my experience Optim can be suitable for small problems but I have had difficulty getting it to work on large constrained optimization problems. There is some discussion of my experience here, AC Optimal Power Flow in Various Nonlinear Optimization Frameworks
There are a few pure-Julia MOI compatible solvers out there but there is a good reason that JuMP primarily calls a wide variety of non-Julia solvers. These solvers distill decades of R&D in optimization algorithms, and it is not easy to get similar performance rolling your own.