Lumber is accumulated by chopping down (dark green) trees, e.g. a whole tree will get you around 8+ pieces.
In some of the more recent games, Stone has been available for building with. For example, in Harvest Moon: Tree of Tranquility, material stone can be accumulated by smashing boulders. Each boulder gives about 4 pieces.