I suspect that some of the type returned by the setup are not inferrable. Try a function barrier before you enter the loop and see whether this recovers the performance of solve!:
function inner(integrator)
x = 0.0
for i in integrator
x = i.u
end
end
function integrate(alg=OrdinaryDiffEq.RK4())
tspan = (0.0, 1e3)
u0 = 1.0
prob = ODEProblem{false}(diffeq, u0, tspan)
integrator = OrdinaryDiffEq.init(prob, alg, save_everystep = false, maxiters = 1e4, abstol=1e-8, reltol=1e-8)
inner(integrator)
end
@btime integrate()