PowerPoint...
Transcript of PowerPoint...
![Page 1: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/1.jpg)
Information Security & IoT Lab
조동근
2017.11.08
![Page 2: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/2.jpg)
목 차
목차
2
1. Review
3. Support Vector Machine(SVM)
2. Regularization
1-1 Hypothesis, Cost function, Gradient Descent1-2 Variable, Placeholder, Session1-3 Example
2-1 Overfitting Problem2-2 Regularization2-3 Example
3-1 SVM3-2 Example
![Page 3: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/3.jpg)
3
1-1 Hypothesis, Cost function, Gradient Descent1-2 Variable, Placeholder, Session1-3 Example
![Page 4: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/4.jpg)
1-1 Hypothesis, Cost function, Gradient Descent
Neural Network 정리 Hypothesis : 임의의 입력에 대해 예측 및 분류를 수행하는 가설 함수
Cost function : 이 함수는 선택된 w, b가 적절한 가설인지 여부를 결정
Gradient Descent : 최적의 가설 함수를 찾기 위해 cost function 최소값을 찾는 방법
그 외 : activation function(Softmax, ReLu, sigmoid, tahn), mini batch, epoch …
4
ℎ𝑤 𝑥 = 𝑤 ∙ 𝑥 + 𝑏
𝑤𝑗 ≔ 𝑤𝑗 - α𝜕
𝜕𝑤𝑗𝐽 𝜃
𝐽 𝑤, 𝑏 =1
2𝑛
𝑥
(ℎ(𝑥) − 𝑦)2
𝐽 𝑤, 𝑏 = −1
𝑛
𝑥
[ℎ(𝑥) ln 𝑎 + 1 − 𝑦 ln 1 − ℎ(𝑥) ]
hypothesis
Cost Function
Gradient Descent
![Page 5: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/5.jpg)
1-2 Variable, Placeholder, Session
Tensorflow 정리
5
Tf.Variable : weight 와 bias 변수 선언 시 사용
Tf.placeholder : feed_dict와 함께 사용되며 세션을 수행할 때 데이터를 입력함
Tf.Session : tensorflow 그래프를 생성하여 코드를 수행
그 외 : tensor, node, edge, operation, tensorboard...
![Page 6: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/6.jpg)
1-3. Example
예제 소스(Linear Regression을 Tensorflow로 구현)
6
![Page 7: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/7.jpg)
1-3. Example
예제 소스(Linear Regression을 Tensorflow로 구현)
7
출력 결과 :
![Page 8: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/8.jpg)
8
2-1 Overfitting Problem2-2 Regularization2-3 Example
![Page 9: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/9.jpg)
2-1 Overfitting Problem
9
regularization
Layer 개수와 node의 개수가 많은 Neural network 아키텍처는 잠재적으로
overfitting 문제를 갖고 있음
Regularization은 overfitting 문제를 완화
validation data를 이용
Overfitting check
cross validation check 를 사용
Training, Validation, Test data 비율을 6:2:2로 나눔
Overfit
fit
Overfitting Problem
![Page 10: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/10.jpg)
2-2 Regularization
10
Formula
L2 regularization
𝐶 = 𝐶0 +𝜆
2𝑛
𝑤
𝑤2
𝜕𝐶
𝜕𝑤𝑗=𝜕𝐶0𝜕𝑤𝑗
+𝜆
𝑛𝑤𝑗
𝑤𝑗 ≔ 𝑤𝑗 − 𝜂𝜕𝐶
𝜕𝑤𝑗
𝑤𝑗 ≔ 𝑤𝑗 − 𝜂𝜕𝐶0𝜕𝑤𝑗
+𝜆
𝑛𝑤𝑗
𝑤𝑗 ≔ 1 − 𝜂𝜆
𝑛𝑤𝑗 − 𝜂
𝜕𝐶0𝜕𝑤𝑗
적용 결과
Regularization
![Page 11: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/11.jpg)
2-2 Regularization
11
Regularization Parameter 계산 방법
𝐶 =1
2𝑛σ𝑥(ℎ(𝑥) − 𝑦)2 +
𝜆
2𝑛σ𝑤𝑤
2 에서 실험적으로 𝜆를 계산
![Page 12: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/12.jpg)
2-2 Regularization
12
Regularization Parameter 계산 방법
![Page 13: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/13.jpg)
2-2 Regularization
13
Regularization Parameter 계산 방법
![Page 14: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/14.jpg)
2-3 example
14
예제 소스
![Page 15: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/15.jpg)
2-3 example
15
예제 소스
![Page 16: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/16.jpg)
2-3 example
16
예제 소스
Regularization 적용
![Page 17: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/17.jpg)
2-3 example
17
예제 소스 결과
𝜆 = 0 (variance) 𝜆 = 1
𝜆 = 10 𝜆 = 100 (bias)
![Page 18: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/18.jpg)
18
3-1 SVM3-2 Example
![Page 19: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/19.jpg)
3-1 SVM
19
SVM SVM은 Margin을 최대화하여 일반화 능력의 극대화를 꾀함
Margin은2
| 𝑤 |으로 표현되며 이를 최대화 함
𝐽 𝑤 =| 𝑤 |
2형태로 변환하고, 𝑡𝑖 𝑤
𝑇𝑥𝑖 + 𝑏 − 1 ≥ 0 의 제한 조건을 가짐
라그랑제 승수법을 도입하여 w 대신 라그랑제 승수를 구하는 문제로 전환
Hyper Plane
Support Vector
Margin
![Page 20: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/20.jpg)
3-2 example
20
예제 소스 https://github.com/nfmcclure/tensorflow_cookbook
![Page 21: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/21.jpg)
3-2 example
21
예제 소스
![Page 22: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/22.jpg)
3-2 example
22
예제 소스
![Page 23: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/23.jpg)
3-2 example
23
예제 소스
![Page 24: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/24.jpg)
3-2 example
24
예제 소스
![Page 25: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/25.jpg)
3-2 example
25
예제 소스
![Page 26: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/26.jpg)
3-2 example
26
예제 소스
![Page 27: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/27.jpg)
3-2 example
27
예제 소스 결과
![Page 28: PowerPoint 프레젠테이션infosec.pusan.ac.kr/wp-content/uploads/2017/11/regularization-and-svm.pdf · Neural Network 정리 Hypothesis: 임의의입력에대해예측및분류를수행하는가설함수](https://reader033.fdocuments.net/reader033/viewer/2022041716/5e4bdfd76c2b7220c5592ce3/html5/thumbnails/28.jpg)
Thank you!