init

1 files changed, 47 insertions, 0 deletions

diff --git a/SI/Resource/Data Science/Machine Learning/Contents/Logistic Regression.md b/SI/Resource/Data Science/Machine Learning/Contents/Logistic Regression.md
new file mode 100644
index 0000000..24b714e
--- /dev/null
+++ b/SI/Resource/Data Science/Machine Learning/Contents/Logistic Regression.md
@@ -0,0 +1,47 @@
+---
+id: 2023-12-18
+aliases: December 18, 2023
+tags:
+- link-note 
+- Data-Science
+- Machine-Learning 
+- Logistic-Regression 
+---
+
+# Logistic Regression
+
+$$Y = \begin{cases} 1  & \text{if korean} \\ 2   & \text{if american} \\ 3   & \text{if japanese} \end{cases} \qquad\qquad\qquad Y = \begin{cases} 1  & \text{if american} \\ 2   & \text{if korean} \\ 3   & \text{if japanese} \end{cases}$$
+
+- In general regression, the results vary depending on the order (size) of the labels
+- A different loss function or model is needed
+- The logistic regression model is a regression model in the form of a logistic function.
+- The predicted value changes depending on the value of wX.
+	1. If $w^{T}X > 0$: classified as 1.
+	2. If $w^{T} X< 0$: classified as 0.
+- How should the loss function be defined to find the optimal value of the parameter *w*?
+
+## Odds
+
+- The odds ratio represents how many times higher the probability of success (y=1) is compared to the probability of failure (y=0)
+- $odds = \dfrac{p(y=1|x)}{1-p(y=1|x)}$
+
+## Logit
+
+- The function form of taking the logarithm of odds
+- When the range of input probability (p) is [0,1], it outputs [$-\infty$, $+\infty$]
+- $logit(p) = log(odds) = log\dfrac{p(y=1|x)}{1-p(y=1|x)}$
+
+## Logistic Function
+
+- The inverse function of the logit transformation
+- $logit(p) = log(odds) = log\dfrac{p(y=1|x)}{1-p(y=1|x)} = w_{0}+ w_1x_{1}+ \dots + w_Dx_{D}= w^TX$
+- $p(y = 1|x) = \dfrac{e^{w^{T}X}}{1 + e^{w^{T}X}} = \dfrac{1}{1 + e^{-w^TX}}$
+- Therefore, the logistic function is a combination of linear regression and the sigmoid function
+
+## Bayes' Theorem
+
+- $P(w|X) = \dfrac{P(X|w)P(w)}{P(X)} \propto P(X|w)P(w)$
+- **[[Posterior]]** probability, $P(w|X)$: The probability distribution of a hypothesis given the data (reliability).
+- **Likelihood** probability, $P(X|W)$: The distribution of given data assuming a hypothesis is known, albeit not well understood.
+- **Prior** probability, $P(w)$: The probability of a hypothesis known in general before looking at the data.
+- There are two methods to estimate the hypothesis (model parameters) using these probabilities: [[Maximum Likelihood Estimation]] (**MLE**) and [[Maximum A Posteriori]] (**MAP**)
\ No newline at end of file