The cost per hour for N (a different metric than cost per layout) is probably a lot higher due to lack of kits. In HO, you can reduce your cost per hour substantially by buying readily available kits for your rolling stock (they are still available!). The more complex can take several evenings to several weeks to assemble, decal, and weather. So even if a Labelle car kit costs $20 to buy, plus another $10 for trucks, couplers, extra details, and decals, the cost per hour is down around $3 or less per hour.
You may be able to buy 3 RTR cars in N for the same $30, but they will cost you $10 per hour or more for your modeling time. With a $40/month model railroading budget, I run out of $ before I run out of time if I buy RTR.
The "downside" for me (if it is one) is that I will have substantially less rolling stock than the next guy. Given that 20-30 cars is the maximum my layouts will handle, kit building comes out better again.
When it comes to structures, N and HO are both fairly equal for $$/hour from what I can see. N structure kits cost less, but have less detail and therefore take less time to assemble. However, a given space will absorb more N structures than HO.
my thoughts, your choices