Use use both but not mixed. I'll first use the medium and then sprinkle a little fine on top. No real reason, I just think it looks a little better that way
I use the fine, although I have used medium as a sub-ballast if the area required a bit of fill. Sidings get cinders, or cinders with dirt and weeds, or, in some cases, just dirt and weeds.