1 files changed, 53 insertions, 0 deletions
diff --git a/SI/Resource/Data Science/Machine Learning/Contents/Optimization.md b/SI/Resource/Data Science/Machine Learning/Contents/Optimization.md
new file mode 100644
index 0000000..2283442
--- /dev/null
+++ b/SI/Resource/Data Science/Machine Learning/Contents/Optimization.md
@@ -0,0 +1,53 @@
+---
+id: 2023-12-17
+aliases: December 17, 2023
+tags:
+- link-note
+- Data-Science
+- Machine-Learning
+- Optimization
+---
+
+# Optimization
+
+## Math
+
+### Partial Differentiation/Derivative
+
+- Differentiate about a specific variable
+- Consider others as constants
+- $\dfrac{\partial y}{\partial x}$
+- e.g., $f(x,y) = x^2 + xy + 3$
+
+### Chain Rule
+
+- $\dfrac{dy}{dx} = \dfrac{dy}{du}*\dfrac{du}{dx}$
+- e.g., $y = ln(u), u = 2x + 4$
+
+## Loss Function
+
+### Mean Squared Error (MSE)
+
+- $L = \frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y_i})^2$
+
+## Parameter Calculation
+
+### Least Square Method (LSM)
+
+- Minimize error of data
+- a: slope (coefficient)
+- b: intercept
+- $L = \sum_{i=1}^{N} (y_i - (ax_i + b))^2$
+
+#### Method 1.
+
+- $0 = \dfrac{\partial L}{\partial a} = \sum_{i=1}^{N} 2(y_i - (ax_i + b))(-x_i) = 2(a\sum_{i=1}^{N} x_i^2 + b\sum_{i=1}^{N} x_i - \sum_{i=1}^{N} x_iy_i)$
+- $0 = \dfrac{\partial L}{\partial b} = \sum_{i=1}^{N} 2(y_i - (ax_i + b))(-1) = 2(a\sum_{i=1}^{N} x_i + b\sum_{i=1}^{N}1 - \sum_{i=1}^{N} y_i)$
+- $a^* = \dfrac{\sum_{i=1}^{N}(x-\bar{x})(y-\bar{y})}{\sum_{i=1}^{N}(x-\bar{x})^2}$
+- $b^* = \bar{y} - a^*\bar{x}$
+
+#### Method 2.
+
+- Partial differentiation with respect to matrix $||Y - WX||^2$
+- $-2X^T(Y-WX) = 0$
+- $W = (X^TX)^{-1}X^TY$
+\ No newline at end of file