Classification algorithms

• Many classification algorithms available
• Going to use probabilistic generative models
  • Not used today since it's less generalizable
  • Not applicable to autonomous systems? How?

Discrete Classification

Linear Algebra Review

Let $x\in \mathbb{R}^D$ and $y \in \mathbb{R}^D$

Let $\alpha(x) = y^Tx$

$\dfrac{\partial \alpha(x)}{\alpha x} = y$

So, a derivative of a scalar with respect to a vector has the same dimension as the vector.

Take $A\in \mathbb{R}^{m\times n}$

$\alpha(x) = x^TAy$

$\nabla_A \alpha(x) \in \mathbb{R}^{m\times n}$

$\alpha(x)=x^TAx$, $x\in\mathbb{R}^D$, $A\in\mathbb{R}^{D\times D}$

$\nabla_x \alpha(x) \in \mathbb{R}^D$

$x^TAx = \sum_i\sum_j x_i a_{ij} x_j$

$\dfrac{\partial x^TAx}{\partial x_1} = 2a_{11}x_1 + \sum_{j=2}a_{1j} x_j + \sum_{i=2}a_{i1}x_i$

$=\sum_{j=1}a_{1j} x_j + \sum_{i=1}a_{i1}x_i = (Ax)_1 +(A^Tx)_1$

$\nabla_x \alpha(x) = (A+A^T)x$