Question

We have seen that the inductive bias of an SVC guarantees that when the dataset is linearly separable, SVC will return a hyperplane that is at exactly the same distance from the two classes. But what about logistic regression? Can we guarantee that it can also be at least partially fair? The answer is no. We can demonstrate how logistic regression can be 'unfair' by constructing a dataset with the properties that: a. the data set is linearly separable, and b. the optimal logistic regression model corresponds to a hyperplane that nearly 'touches' one of the two classes - that is, it has a very big margin with respect to one of the two classes, and a very small margin with respect to the other class. Demonstrate your answer as follows: Q3-1. Plot the data points, as we did above for the Iris data set. This will show that your data set is linearly separable. Q3-2. Calculate the optimal logistic neuron weights using the function LogisticRegressionGD from Question 1. Q3-3. Plot the decision regions to demonstrate how the learned separation line is unfair. Hint: Try small datasets. Note: It's best to use fresh variables for your dataset, since the previous values of X,y

Answer 1

The question addresses the difference in inductive bias between SVC and logistic regression, highlighting how logistic regression may create 'unfair' decision boundaries in linearly separable datasets.

Step-by-step explanation:

The student's question concerns the difference between the inductive biases of Support Vector Machines (SVM) with a linear kernel, specifically Support Vector Classifier (SVC), and logistic regression when it comes to the fairness of the decision boundary in a linearly separable dataset.

While SVC aims to maximize the margin between classes, logistic regression does not have this mechanism, which means it can produce a decision boundary very close to one of the classes. To demonstrate this, the student is asked to plot a linearly separable dataset, calculate the optimal logistic regression weights, and plot the decision regions to show the potential 'unfairness' in the learned model.