Question

4.9 (Confusion matrix) In a classification problem with K classes, the cost matrix is K×K matrix C=[c

ij ] in which c ij is the cost when one example belongs to class i but is predicted to be in class j. Similarly, a confusion matrix is a K×K matrix A=[a ij ], in which a ijis the number of class i examples that are classified as belonging to class j. Let the confusion matrix be computed based on a test set with N examples. We often normalize the confusion matrix to obtain
A^ , by a^ij =a ij/∑ k=1Kaik . Hence, the sum of all elements in any row of A^equals 1 . We call A^ the normalized confusion matrix. (a) Prove that the total cost for the test set equals tr(C TA). (b) In an imbalanced classification problem, do you prefer the confusion matrix or the normalized one? Why?

Answer 1

The total cost for the test set is proven by the matrix trace operation tr(CTA), and in imbalanced classification problems, the normalized confusion matrix is preferred as it offers a fair comparison across classes with different sample sizes.

Step-by-step explanation:

The question addresses the concepts of a confusion matrix and a cost matrix within the context of a multi-class classification problem. In part (a), the proof involves matrix multiplication and the trace of a matrix. The trace operation, denoted as tr(), is the sum of the elements on the main diagonal of a matrix.

The total cost is calculated by multiplying the transpose of the cost matrix (C) with the confusion matrix (A), which represents the frequency of predictions, and then taking the trace of the resulting matrix. The formula tr(CTA) gives the summation of the costs weighted by their respective frequencies of occurrence.

For part (b), in an imbalanced classification problem, there is usually a preference toward the normalized confusion matrix because it provides insights into classification performance relative to the presence of classes with different sample sizes.

A normalized matrix helps to compare the performance across classes more fairly, since each row sums to 1, showing the distribution of predictions for a given true class.