When you do work to lift the object, the amount of work you do BECOMES the object's gravitational potential energy. It GETS its potential energy from the work you do to lift it. They're equal. You lose it, and the object gains it. Energy is not created or destroyed. It's just transferred from you to the object.
Later, when you DROP the object, GRAVITY does the same amount of work on it, to pull it to the ground. Again, no energy is created or destroyed. Every time a force acts to move anything, the energy to do it comes from somewhere, and the energy goes somewhere.