Answer:
This is somewhat of a confusing question.
Zero potential energy is usually used to refers to the zero potential energy of objects at infinity. Thus, bringing an object from infinity requires negative work because work is done by the object. One could also choose the pivot point of the pendulum as zero potential. Then any objects above the pivot point would be at a positive potential, and objects below the pivot point would have a negative potential (the object does work moving to a point of lower potential).
Basically, work done "on" an object requires positive potential energy input while work done "by" the object requires negative energy input.
Those are the basic considerations in solving potential energy problems.