Tools that are able to be upgraded are strengthened in two different ways: they can "level up", which happens just by using them a certain number of times, and they can be "upgraded", which requires you to purchase the upgrade from the General store or the Blacksmith's Shop. "Leveling up" and "Upgrading" result in two different types of benefits.
Generally speaking, leveling up a tool makes the tool more powerful. It can hit harder or hit more at once. To use the higher-level use, hold down A when using the tool and let it charge up. With each charge, you'll see an animation and hear a sound. Don't charge up beyond your tool's level, though, or your character will fail and lose stamina. For example, if your tool is Level 2, release A once you've seen one "power-up" animation.
Upgrading makes the tool easier to use. In most cases, the only benefit to upgrading is that the tool consumes less stamina when used. Exceptions are noted below. Also note that the tool will use the same amount of stamina no matter how much you charge it up, so other than saving a couple seconds there is no reason to use a lower-level strike when you could charge the tool up more.