Answer:
The total ways the textbooks can be arranged in the 3 boxes is , which is equivalent to . If all of the math textbooks are put into the box that can hold  textbooks, there are  ways for the other textbooks to be arranged. If all of the math textbooks are put into the box that can hold  textbooks, there are  ways to choose the other book in that box, times  ways for the other books to be arranged. If all of the math textbooks are put into the box with the capability of holding  textbooks, there are  ways to choose the other 2 textbooks in that box, times  ways to arrange the other 7 textbooks. , , and , so the total number of ways the math textbooks can all be placed into the same box is . So, the probability of this occurring is . If the numerator and denominator are both divided by , we have . Simplifying the numerator yields , and dividing both numerator and denominator by  results in . This fraction cannot be simplified any further, so  and . Therefore, .