Final answer:
The relative error in the computed solution to Ax=b is approximately 2E-16. The computed solution has 4 significant figures.
Step-by-step explanation:
To estimate the relative error in the computed solution to Ax=b, we can use the formula: relative error = condition number * (relative error in A) + (relative error in b). In this case, the condition number is approximately 1.7E3 and the relative error in b is 9E-9. Since the elements of A are known to 8 significant figures, the relative error in A is 8E-8. Using the formula, the relative error in the computed solution is approximately 2E-16, so the correct answer is (b).
To determine the number of significant figures in the computed solution, we look at the relative error. The relative error in the computed solution is approximately 2E-16, which means that we have about 16 correct decimal places. Since there are 8 significant figures in the original matrix A, the computed solution will have 8 - 16 = -8 significant figures. However, we can only have positive significant figures, so the correct answer is (c) 4 significant figures.