The work done will be negative.
Why?
We must remember that work is the result of applying a force upon an object and causing a motion.
Let's answer the question using an example.
If we pick up a book from a table, and we move it, putting it on the floor, we have at least two forces acting. We are applying a force upon the book that goes upward, the gravity force is acting downwards and the displacement is against the force applied by you. In consequence, the work done by the force is negative.
Remember, if the applied force and the displacement have the same direction, the work will be positive, if the applied force and the displacement have different directions, the work done will be negative.
Have a nice day!