If this is your primary desire and you want to use a backend like Ipopt as a solver, NonConvex.jl came to my mind as the Julia framework that has similar design goals. I reviewed a bunch of them here, Survey of Non-Linear Optimization Modeling Layers in Julia
That said, if you want maximum performance and scalability you might want to reconsider the pros and cons of using JuMP. See this discussion, AC Optimal Power Flow in Various Nonlinear Optimization Frameworks - #35 by ccoffrin