(Machine Learning) Suppose you have a 2-class classification problem, where each class is Gaussian. Let θ = {α, µ1, Σ1, µ2, Σ2} denote the set of model parameters. Suppose the class probability p(y|θ) is modeled via the Bernoulli distribution, i. e. p(y|θ) = αy(1 −α)1−y, and the probability of the data p(x|y, θ) is modeled as p(x|y, θ) = (x µy, Σy).
1) Recover the parameters of the model from the maximum likelihood approach ( for α, for µ’s, and for Σ’s). * Assume the data are i. i.d. and N is the number of data samples. Show all derivations.
2) Next, suppose you want to make a classification decision by assigning proper label y to a given data point x. You decide the label based on Bayes optimal decision y = arg maxyˆ={0,1} p(yˆ|x). Prove that the decision boundary is linear when covariances Σ1 and Σ2 are equal and otherwise the boundary is quadratic.
3) Illustrate on 2d example the decision boundary for the case when covariances are not equal clearly indicating which class is more concentrated around its mean