They provide evidence of genes coding for structures that are present not because they are useful, but because they were once useful to an ancestor.
One example is the plantaris muscle. This is a long, thin muscle in the human foot and calf that serves no significanl purpose in humans ... in fact, it is regularly removed by doctors harvesting muscle tissue to use in reconstructive heart surgery, because its removal leaves no loss of function in walking or balance.
However this same muscle is found more fully developed in the feet and calves of other primates. It is the muscle used for grasping with the feet.