The Gem of the Goddess is made by collecting all nine Goddess Jewels. The jewels can be found in the Spring Mine on certain floors throughout the mine. If you're playing Friends of Mineral Town, the Gem of the Goddess is necessary in order to marry Harvest Goddess.
The gem's main purpose is to increase your character's stamina. The Gem can be equipped in your active tool slot. When equipped, your character's stamina will gradually increase when your character is outside walking around. Time must be passing, and it must be equipped as your current tool. If you have the Truth Gem, you can see how much stamina you have regained. The Gem of the Goddess is the only way in game to recover all of your stamina in a day.