acemate | Machine_Learning_I_Exam_WS_14

Section MAIN-dd0444ba-2553-4ed1-9d9d-c4fe9ab8f9d9

Mixed

Multi Choice

15 P

Es gibt nur eine richtige Antwort. Falsche Antworten geben 0 Punkte genauso wie keine Antwort.

a

3 P

The Bayes error for classification is:

Select one answer

b

3 P

Independent Component Analysis can be achieved by:

Select one answer

c

3 P

The K-means algorithm:

Select one answer

d

3 P

A biased estimator is sometimes used to:

Select one answer

e

3 P

Which is False? The Restricted Boltzmann machine is:

Select one or multiple answers

Section MAIN-d6cc762f-db24-4666-9c07-f994243f7f26

Mixed

Models and Datasets

15 P

a

7 P

Sketch a two-dimensional two-class supervised dataset that Fishers linear discriminant and the linear hard-margin SVM would learn separating bounderies with different directions. Your example must be stereotyping for the two classification techniques.

Your answer:

b

7 P

Sketch a two-dimensional unsupervised dataset for which K-means (K=3) may get stuck at a local optimum. In your drawing show (using square markers) the position of the centroids for the local minimum and (with circels) the position of the global minimum. Your example must be stereotyping in order to show the difference between the local and global minimum.

Your answer:

Section MAIN-6afbbac4-8754-42e1-a08b-53612b0510ae

Mixed

Kernels and Feature Maps

25 P

In order for a kernel to be positive semid-definite (PSD) it must satisfy $\sum_{i=1}^{n} \sum_{j=1}^{n} c_i c_j k(x_i, x_j) \ge 0$ for all sequences of data points and coefficients.

a

10 P

Show that if $k_1$ and $k_2$ are PSD kernels, then $k_3 = \alpha k_1 + \beta k_2$ with $\alpha, \beta \ge 0$ is also a PSD kernel.

Your answer:

b

5 P

Give an example showing that the positive semi-definiteness is not guaranteed if $\alpha < 0$ or $\beta < 0$

Your answer:

c

10 P

A feature map associated to a kernel $k$ must satisfy: $\langle \phi(x_i), \phi(x_j) \rangle = k(x_i, x_j) \forall i, j \in X$ Find the feature map of $k_3 = \alpha k_1 + \beta k_2$ and show that it fulfills the equation above.

Your answer:

Section MAIN-6b5f0dbf-b253-44e3-bd00-65b58852a908

Mixed

Lagrance multipliers

20 P

We consider a discrete probability distribution $p(i)$ , $i = \{1..10\}$ . We can represent such probability distribution as a vector $p$ indexed by $i$ subject to the constraints $p_i \ge 0$ and $\sum_{i=1}^{10} p_i = 1$ . We would like to find analytically the probability distribution $p$ with maximum entropy. The Entropy is given by $H(p) = -\sum_{i=1}^{10} p_i log p_i$ and is a concave function

a

5 P

Write down the Lagrangian function associated to this constrained optimization problem.

Your answer:

b

10 P

Show using the Lagrangian method that the optimal probability distribution is uniform with $p(i) = 0.1$ for all $i$ .

Your answer:

c

5 P

Explain briefly why the same Lagrange method cannot be used to find probability distribution with minimum entropy

Your answer:

Section MAIN-5af17f8f-c333-4f31-a190-4a2e7cc41348

Mixed

Quadratic Programming

25 P

We consider the regularized regression task this is solved by optimizing $\min_{W} \sum_{k=1}^{N} (y_k - w_1 x_k)^2$ subject to $||w||_\infty \leq 1$ where $w$ is the solution and $(x, y)_k$ is a dataset of $N$ input-output pairs.

a

10 P

Show that the optimization problem can be rewritten as: $\min_W w^T X^T X w - 2y^T X w$ subject to $w_i \leq 1$ and $w_i \geq -1$

Your answer:

b

15 P

You have at your disposal a quadratic solver quadprog(Q,1,A,b) that is solving $\min_V v^T Q v + 1^T v$ subject to $Av \leq b$ Write down the code that builds the numpy array Q,1,A,b from the data X and y.

Your answer: