There are several differences, however a big property that distinguishes the 2 is that potential energy cannot be transferred, whereas kinetic energy can.
We can see this for example in a bow, when you pull it back it has elastic potential energy because of the strain in the bow, however you can't transfer this potential energy to another object, whereas the flying arrow can transfer it's KE to the target it hits as several things including thermal and sound energy.