Logistic Regression

Sigmoid/Logistic function

$$g(z) = {1 \over 1+e^{-z}}$$

Hypothesis

$$h_\theta(x) = g(\theta \cdot x) = {1 \over 1 + e^{-\theta \cdot x}}$$

• $y=1$ if $h_\theta(x) \ge 0.5$ (i.e. $\theta \cdot x \ge 0$)
• $y=0$ if $h_\theta(x) < 0.5$ (i.e. $\theta \cdot x < 0$)

Logistic Regression Cost Function

$$J(\theta) = - {1 \over m} [\sum_{i=1}^m y_i \log h_\theta(x_i) + (1-y_i) \log(1-h_\theta(x_i))]$$

Gradient Descent

$$\min_\theta J(\theta)$$

Repeat: simultaneously update all $\theta_j$

$$\theta_j:=\theta_j - \alpha {\partial \over \partial \theta_j} J(\theta)$$ $$\theta_j:=\theta_j - \alpha \sum_{i=1}^m (h_\theta(x_i) - y_i)x_i$$

Regularization

$$J(\theta) = - {1 \over m} [\sum_{i=1}^m y_i \log h_{\theta(x_i)} + (1-y_i) \log(1-h_\theta(x_i))] + {\lambda \over 2m} \sum_{i=1}^n \theta_j^2$$