Thank you!
So what I was missing was the kwarg noise_rate_prototype which is an argument which is of the same type as the output of g.
r = 1
kp = 0.5
kr = 0.5
A = [0 1
0 0]
B = [0,1]
u(x) = -kp*x[1] - kr*r
f(x,p,t) = A*x + B*u(x)
g(u,p,t) = 0.1*[[kp,1] [kp,0.5]]
prob = SDEProblem(f,g,[0,0], (0,10), noise_rate_prototype = zeros(2, 2))
plot(solve(prob))