The Effect of spiral and bar structure on the Milky Way velocity distribution