I believe the answer is: excluding members of political factions that were not loyal to him
At that time, andrew jackson perceive the chance of being re-elected to the second term was really low and his current cabinet members somehow also pessimistic about his chance. Because of this, he fire many members of the cabinet (excluding the postmaster general) and keep the one that loyal to him.