You do need to map data to an aesthetic, before using a scale for that aesthetic. See the Gadfly Tutorial (perhaps we could add “Steps to build a plot”).
x = ["A", "B"]
p = plot(x=x, y=[81.7, 13.0], Geom.bar(position=:dodge),
color=x,
Scale.color_discrete_manual("yellow3", "steelblue")
)