For much of the paint, you can go to a local hardware or art supply store. A quart of latex paint in an appropriate tone (which is NOT always brown) is a great start--find a sample of dirt you like (or a photo of landscape like that which you wish to model), bring it to the local hardware/paint store, and compare it to the paint sample swatches there. Have them mix up a gallon of dirt-colored paint to your specifications. Hobby paints are far too expensive to use painting scenery!
You'll be able to find lots of other handy hobby stuff at the hardware store--Hydrocal, sandpaper, Durham's water putty, foam sheets (if you're doing foam scenery), lumber of course, wire, etcetera.
I also recommend getting some flat black, flat white and primer gray spray paint--they can be good for rough-priming structures before applying paint (Paint sticks better to primer than to plastic.)
Here are the paints I burn through the most:
Grimy Black (good for everything!)
Roof Brown
Railroad Tie Brown (great for painting track and ties, and otherwise coloring wood)
CP Gray
Boxcar Red (good for brick as well as boxcars)
Aged Concrete
Aged White
Engine Black
Reefer White
Caboose Red
Rust (me, I like rust. Also good for brick.)
Polly S does a couple of good "basic paint set" sets--get a couple, one with basic railroad/rolling stock colors, one for weathering (which will include most of the above) and get an extra bottle of Grimy Black.
In a pinch, though, you can make good use of art-supply store (or big-box department store) craft paints to paint structures and I'd guess even rolling stock. My latest couple of structure projects have been painted primarily with hardware-store spray paint.
Get a couple colors of Woodland Scenics ground foam--the fine stuff to represent grass, the clumpy stuff to represent small bushes. Colors will depend on where you're modeling--New England in springtime will require different colors than the Southwest in high summer. If you want trees, get some lichen and some Woodland Scenics tree kits--make sure you have some gray paint (not brown) to paint those trees!
Chalk? I assume you mean for weathering. If you have an art-supply store nearby you can use artist's chalks/colored charcoal--otherwise if you want to get into chalks, order a complete set.
so am ordering everything online. If I can get a (more or less) complete list of colors and only make one order it will help save shipping costs. I've found a couple of good sites to order from, just need to know what to get.