This statement is, in fact, false. As Hamlet up and leaves his mothers room, he stops and gets the body of polonius and takes it with him. He does not leave it behind. Incidentally, when Hamlet realizes that he has murdered Polonius, he feels no guilt at all.