Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2...
Transcript of Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2...
![Page 1: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/1.jpg)
Boosted Decision Trees
![Page 2: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/2.jpg)
Previously on CSCI 4622
2
Ensemble Learning: Build a bunch of slightly different models. Predict by majority vote.
Bagging: Train a bunch of decision trees on bootstrapped resamples of training data.
Random Forests: Train a bunch of decision trees on bootstrapped resamples of training data. On each node, consider only a random subset of features to split on.
A single tree is very sensitive to training data. An average over many trees is less sensitive.
Bagging can sometimes create correlated trees, especially if there’s a handful of strong features
Random Forests helps to de-correlate trees by leaving strong features out in some splits
![Page 3: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/3.jpg)
Boosting Intuition
3
Boosting is another ensemble model, but with a slight twist
“How is it that a committee of blockheads can somehow arrive at highly reasoned decisions, despite the weak judgment of the individual members? How can the shaky separate views of a
panel of dolts be combined into a single opinion that is very likely to be correct?
Schapire and Freund, Boosting: Foundations and Algorithms”
"
I
![Page 4: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/4.jpg)
Boosting Intuition
4
Boosting is another ensemble model, but with a slight twist
“How is it that a committee of blockheads can somehow arrive at highly reasoned decisions, despite the weak judgment of the individual members? How can the shaky separate views of a
panel of dolts be combined into a single opinion that is very likely to be correct?
Schapire and Freund, Boosting: Foundations and Algorithms”
Idea: Build a sequence of dumb models
Modify the training data along the way to focus on difficult to classify examples
Predict based on weighted majority vote of all dumb models
o What do we mean by dumb? o How do we promote difficult examples? o Which models get more say in the vote?
![Page 5: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/5.jpg)
Dumb Models
5
What do we mean by dumb?
Each model in our sequence will be a weak learner
A model is a weak learner if its training error is just under 50%
Most common weak learner is a Decision Stump – a Decision Tree with a single split
Other common weak learners include slightly deeper Decision Trees and perceptrons
h(x)<latexit sha1_base64="DpZ0qqzuWcfNwUlQjJ+pzy3FXxM=">AAAB93icbVBNT8JAEJ3iF9YPqh69bAQTvJCWC3oj8eIREyskQMh22cKG7bbZ3Rqx4Zd48aDGq3/Fm//GBXpQ8CWTvLw3k5l5QcKZ0q77bRU2Nre2d4q79t7+wWHJOTq+V3EqCfVJzGPZCbCinAnqa6Y57SSS4ijgtB1Mrud++4FKxWJxp6cJ7Ud4JFjICNZGGjilij2uZr0gRI+zC7uCBk7ZrbkLoHXi5aQMOVoD56s3jEkaUaEJx0p1PTfR/QxLzQinM7uXKppgMsEj2jVU4IiqfrY4fIbOjTJEYSxNCY0W6u+JDEdKTaPAdEZYj9WqNxf/87qpDi/7GRNJqqkgy0VhypGO0TwFNGSSEs2nhmAimbkVkTGWmGiTlW1C8FZfXid+vXZV827r5WYjT6MIp3AGVfCgAU24gRb4QCCFZ3iFN+vJerHerY9la8HKZ07gD6zPHx5mkPE=</latexit><latexit sha1_base64="DpZ0qqzuWcfNwUlQjJ+pzy3FXxM=">AAAB93icbVBNT8JAEJ3iF9YPqh69bAQTvJCWC3oj8eIREyskQMh22cKG7bbZ3Rqx4Zd48aDGq3/Fm//GBXpQ8CWTvLw3k5l5QcKZ0q77bRU2Nre2d4q79t7+wWHJOTq+V3EqCfVJzGPZCbCinAnqa6Y57SSS4ijgtB1Mrud++4FKxWJxp6cJ7Ud4JFjICNZGGjilij2uZr0gRI+zC7uCBk7ZrbkLoHXi5aQMOVoD56s3jEkaUaEJx0p1PTfR/QxLzQinM7uXKppgMsEj2jVU4IiqfrY4fIbOjTJEYSxNCY0W6u+JDEdKTaPAdEZYj9WqNxf/87qpDi/7GRNJqqkgy0VhypGO0TwFNGSSEs2nhmAimbkVkTGWmGiTlW1C8FZfXid+vXZV827r5WYjT6MIp3AGVfCgAU24gRb4QCCFZ3iFN+vJerHerY9la8HKZ07gD6zPHx5mkPE=</latexit><latexit sha1_base64="DpZ0qqzuWcfNwUlQjJ+pzy3FXxM=">AAAB93icbVBNT8JAEJ3iF9YPqh69bAQTvJCWC3oj8eIREyskQMh22cKG7bbZ3Rqx4Zd48aDGq3/Fm//GBXpQ8CWTvLw3k5l5QcKZ0q77bRU2Nre2d4q79t7+wWHJOTq+V3EqCfVJzGPZCbCinAnqa6Y57SSS4ijgtB1Mrud++4FKxWJxp6cJ7Ud4JFjICNZGGjilij2uZr0gRI+zC7uCBk7ZrbkLoHXi5aQMOVoD56s3jEkaUaEJx0p1PTfR/QxLzQinM7uXKppgMsEj2jVU4IiqfrY4fIbOjTJEYSxNCY0W6u+JDEdKTaPAdEZYj9WqNxf/87qpDi/7GRNJqqkgy0VhypGO0TwFNGSSEs2nhmAimbkVkTGWmGiTlW1C8FZfXid+vXZV827r5WYjT6MIp3AGVfCgAU24gRb4QCCFZ3iFN+vJerHerY9la8HKZ07gD6zPHx5mkPE=</latexit><latexit sha1_base64="DpZ0qqzuWcfNwUlQjJ+pzy3FXxM=">AAAB93icbVBNT8JAEJ3iF9YPqh69bAQTvJCWC3oj8eIREyskQMh22cKG7bbZ3Rqx4Zd48aDGq3/Fm//GBXpQ8CWTvLw3k5l5QcKZ0q77bRU2Nre2d4q79t7+wWHJOTq+V3EqCfVJzGPZCbCinAnqa6Y57SSS4ijgtB1Mrud++4FKxWJxp6cJ7Ud4JFjICNZGGjilij2uZr0gRI+zC7uCBk7ZrbkLoHXi5aQMOVoD56s3jEkaUaEJx0p1PTfR/QxLzQinM7uXKppgMsEj2jVU4IiqfrY4fIbOjTJEYSxNCY0W6u+JDEdKTaPAdEZYj9WqNxf/87qpDi/7GRNJqqkgy0VhypGO0TwFNGSSEs2nhmAimbkVkTGWmGiTlW1C8FZfXid+vXZV827r5WYjT6MIp3AGVfCgAU24gRb4QCCFZ3iFN+vJerHerY9la8HKZ07gD6zPHx5mkPE=</latexit>
err =1
m
mX
i=1
I(yi 6= h(xi)) =1
2� �
<latexit sha1_base64="N46xc2aFCg/Ztbtuk1eTBKVbj1k=">AAACUnicbVJNbxMxFPSGr3YpNNAjF4sUKT0Q7eZSOFSqxIXeitTQSnFYeb1vE1Pbu9hvUVfW/keE1EP/CBcOrZPmQFtGsjSaee/ZHjuvlXSYJFdR79HjJ0+fbWzGz7devNzuv3r91VWNFTARlarsWc4dKGlgghIVnNUWuM4VnObnn5b+6U+wTlbmBNsaZprPjSyl4BikrP99N2YIF2i1B2s7ekBZUVoufNp53bFCulrx1mGrgLlGZ14epN03TY+GbSYpM/CDLoae5SW96DK5t3dnwLij7ymbc615vEuz/iAZJSvQhyRdkwFZ4zjr/2ZFJRoNBoXizk3TpMaZ5xalUNDFrHFQc3HO5zAN1HANbuZXmXT0XVAKWlY2LIN0pf7b4bl2rtV5qNQcF+6+txT/500bLD/MvDR1g2DE7UZloyhWdBkwLaQFgaoNhAsrw1mpWPCQCIZniEMI6f0rPyST8ejjKP0yHhzur9PYIG/IWzIkKdknh+QzOSYTIsgv8odcRyS6jP72wi+5Le1F654dcge9rRt+/rHz</latexit><latexit sha1_base64="N46xc2aFCg/Ztbtuk1eTBKVbj1k=">AAACUnicbVJNbxMxFPSGr3YpNNAjF4sUKT0Q7eZSOFSqxIXeitTQSnFYeb1vE1Pbu9hvUVfW/keE1EP/CBcOrZPmQFtGsjSaee/ZHjuvlXSYJFdR79HjJ0+fbWzGz7devNzuv3r91VWNFTARlarsWc4dKGlgghIVnNUWuM4VnObnn5b+6U+wTlbmBNsaZprPjSyl4BikrP99N2YIF2i1B2s7ekBZUVoufNp53bFCulrx1mGrgLlGZ14epN03TY+GbSYpM/CDLoae5SW96DK5t3dnwLij7ymbc615vEuz/iAZJSvQhyRdkwFZ4zjr/2ZFJRoNBoXizk3TpMaZ5xalUNDFrHFQc3HO5zAN1HANbuZXmXT0XVAKWlY2LIN0pf7b4bl2rtV5qNQcF+6+txT/500bLD/MvDR1g2DE7UZloyhWdBkwLaQFgaoNhAsrw1mpWPCQCIZniEMI6f0rPyST8ejjKP0yHhzur9PYIG/IWzIkKdknh+QzOSYTIsgv8odcRyS6jP72wi+5Le1F654dcge9rRt+/rHz</latexit><latexit sha1_base64="N46xc2aFCg/Ztbtuk1eTBKVbj1k=">AAACUnicbVJNbxMxFPSGr3YpNNAjF4sUKT0Q7eZSOFSqxIXeitTQSnFYeb1vE1Pbu9hvUVfW/keE1EP/CBcOrZPmQFtGsjSaee/ZHjuvlXSYJFdR79HjJ0+fbWzGz7devNzuv3r91VWNFTARlarsWc4dKGlgghIVnNUWuM4VnObnn5b+6U+wTlbmBNsaZprPjSyl4BikrP99N2YIF2i1B2s7ekBZUVoufNp53bFCulrx1mGrgLlGZ14epN03TY+GbSYpM/CDLoae5SW96DK5t3dnwLij7ymbc615vEuz/iAZJSvQhyRdkwFZ4zjr/2ZFJRoNBoXizk3TpMaZ5xalUNDFrHFQc3HO5zAN1HANbuZXmXT0XVAKWlY2LIN0pf7b4bl2rtV5qNQcF+6+txT/500bLD/MvDR1g2DE7UZloyhWdBkwLaQFgaoNhAsrw1mpWPCQCIZniEMI6f0rPyST8ejjKP0yHhzur9PYIG/IWzIkKdknh+QzOSYTIsgv8odcRyS6jP72wi+5Le1F654dcge9rRt+/rHz</latexit><latexit sha1_base64="N46xc2aFCg/Ztbtuk1eTBKVbj1k=">AAACUnicbVJNbxMxFPSGr3YpNNAjF4sUKT0Q7eZSOFSqxIXeitTQSnFYeb1vE1Pbu9hvUVfW/keE1EP/CBcOrZPmQFtGsjSaee/ZHjuvlXSYJFdR79HjJ0+fbWzGz7devNzuv3r91VWNFTARlarsWc4dKGlgghIVnNUWuM4VnObnn5b+6U+wTlbmBNsaZprPjSyl4BikrP99N2YIF2i1B2s7ekBZUVoufNp53bFCulrx1mGrgLlGZ14epN03TY+GbSYpM/CDLoae5SW96DK5t3dnwLij7ymbc615vEuz/iAZJSvQhyRdkwFZ4zjr/2ZFJRoNBoXizk3TpMaZ5xalUNDFrHFQc3HO5zAN1HANbuZXmXT0XVAKWlY2LIN0pf7b4bl2rtV5qNQcF+6+txT/500bLD/MvDR1g2DE7UZloyhWdBkwLaQFgaoNhAsrw1mpWPCQCIZniEMI6f0rPyST8ejjKP0yHhzur9PYIG/IWzIkKdknh+QzOSYTIsgv8odcRyS6jP72wi+5Le1F654dcge9rRt+/rHz</latexit>
![Page 6: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/6.jpg)
Promoting Difficult Examples
6
How do we promote difficult examples?
After each iteration:
o Increase importance of training examples we got wrong on previous iteration
o Decrease importance of training examples we got right on previous iteration
Each training example will carry around a weight
Weights are nonnegative normalized so that they behave like a probability distribution
wi<latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit><latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit><latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit><latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit>
mX
i=1
wi = 1, wi � 0<latexit sha1_base64="o/WwpIg7D+Ld/IVkOPR7DtWc7HM=">AAACIHicbVA9T8MwEHX4LOGrwMhi0SIxoCrp0jIgIbEwFolCpaZEjnNtrdpJajugKOpfYeGvsDAAgg1+DW7pAC1POun5vTv57gUJZ0o7zqe1sLi0vLJaWLPXNza3tos7u9cqTiWFJo15LFsBUcBZBE3NNIdWIoGIgMNNMDgf+zd3IBWLoyudJdARpBexLqNEG8kv1su2FzKVcJIpnXHwVCr8nJ26o1uB732GT7F7jL1hSsLJ0+vBEDt2GfvFklNxJsDzxJ2SEpqi4Rc/vDCmqYBIU06UartOojs5kZpRDiPbSxUkhA5ID9qGRkSA6uSTC0f40Cgh7sbSVKTxRP09kROhVCYC0ymI7qtZbyz+57VT3a13chYlqYaI/nzUTTnWMR7HhUMmgWqeGUKoZGZXTPtEEqpNqLYJwZ09eZ40q5WTintZLZ3VpmkU0D46QEfIRTV0hi5QAzURRQ/oCb2gV+vRerberPef1gVrOrOH/sD6+gYWlaET</latexit><latexit sha1_base64="o/WwpIg7D+Ld/IVkOPR7DtWc7HM=">AAACIHicbVA9T8MwEHX4LOGrwMhi0SIxoCrp0jIgIbEwFolCpaZEjnNtrdpJajugKOpfYeGvsDAAgg1+DW7pAC1POun5vTv57gUJZ0o7zqe1sLi0vLJaWLPXNza3tos7u9cqTiWFJo15LFsBUcBZBE3NNIdWIoGIgMNNMDgf+zd3IBWLoyudJdARpBexLqNEG8kv1su2FzKVcJIpnXHwVCr8nJ26o1uB732GT7F7jL1hSsLJ0+vBEDt2GfvFklNxJsDzxJ2SEpqi4Rc/vDCmqYBIU06UartOojs5kZpRDiPbSxUkhA5ID9qGRkSA6uSTC0f40Cgh7sbSVKTxRP09kROhVCYC0ymI7qtZbyz+57VT3a13chYlqYaI/nzUTTnWMR7HhUMmgWqeGUKoZGZXTPtEEqpNqLYJwZ09eZ40q5WTintZLZ3VpmkU0D46QEfIRTV0hi5QAzURRQ/oCb2gV+vRerberPef1gVrOrOH/sD6+gYWlaET</latexit><latexit sha1_base64="o/WwpIg7D+Ld/IVkOPR7DtWc7HM=">AAACIHicbVA9T8MwEHX4LOGrwMhi0SIxoCrp0jIgIbEwFolCpaZEjnNtrdpJajugKOpfYeGvsDAAgg1+DW7pAC1POun5vTv57gUJZ0o7zqe1sLi0vLJaWLPXNza3tos7u9cqTiWFJo15LFsBUcBZBE3NNIdWIoGIgMNNMDgf+zd3IBWLoyudJdARpBexLqNEG8kv1su2FzKVcJIpnXHwVCr8nJ26o1uB732GT7F7jL1hSsLJ0+vBEDt2GfvFklNxJsDzxJ2SEpqi4Rc/vDCmqYBIU06UartOojs5kZpRDiPbSxUkhA5ID9qGRkSA6uSTC0f40Cgh7sbSVKTxRP09kROhVCYC0ymI7qtZbyz+57VT3a13chYlqYaI/nzUTTnWMR7HhUMmgWqeGUKoZGZXTPtEEqpNqLYJwZ09eZ40q5WTintZLZ3VpmkU0D46QEfIRTV0hi5QAzURRQ/oCb2gV+vRerberPef1gVrOrOH/sD6+gYWlaET</latexit><latexit sha1_base64="o/WwpIg7D+Ld/IVkOPR7DtWc7HM=">AAACIHicbVA9T8MwEHX4LOGrwMhi0SIxoCrp0jIgIbEwFolCpaZEjnNtrdpJajugKOpfYeGvsDAAgg1+DW7pAC1POun5vTv57gUJZ0o7zqe1sLi0vLJaWLPXNza3tos7u9cqTiWFJo15LFsBUcBZBE3NNIdWIoGIgMNNMDgf+zd3IBWLoyudJdARpBexLqNEG8kv1su2FzKVcJIpnXHwVCr8nJ26o1uB732GT7F7jL1hSsLJ0+vBEDt2GfvFklNxJsDzxJ2SEpqi4Rc/vDCmqYBIU06UartOojs5kZpRDiPbSxUkhA5ID9qGRkSA6uSTC0f40Cgh7sbSVKTxRP09kROhVCYC0ymI7qtZbyz+57VT3a13chYlqYaI/nzUTTnWMR7HhUMmgWqeGUKoZGZXTPtEEqpNqLYJwZ09eZ40q5WTintZLZ3VpmkU0D46QEfIRTV0hi5QAzURRQ/oCb2gV+vRerberPef1gVrOrOH/sD6+gYWlaET</latexit>
{ C Ii, yD3I,
=
I = 1 , ... ,m
=
![Page 7: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/7.jpg)
Weighted Majority Vote
7
Which models get more say in the weighted majority vote?
The models that performed better on the training data get more say in the vote
For our sequence of weak learners:
Boosted classifier defined by
The weight controls vote contribution of . Think of as measure of accuracy of
h1(x), h2(x), . . . , hK(x)<latexit sha1_base64="KmuhbOlzzpDHBoE6U+fHBMjY5eQ=">AAACHnicbVDLSsNAFJ3UV42vqEs3g61QoZSkC6u7ghvBTQVrC20Jk+mkGTrJhJmJWEL9Ejf+ihsXKoIr/RunbRBtPXDhzDn3MvceL2ZUKtv+MnJLyyura/l1c2Nza3vH2t27kTwRmDQxZ1y0PSQJoxFpKqoYaceCoNBjpOUNzyd+65YISXl0rUYx6YVoEFGfYqS05FonRTNwnVLa9Xx4Nz4uw/vArf5+dlmfK1mGgXv5IxehaxXsij0FXCRORgogQ8O1Prp9jpOQRAozJGXHsWPVS5FQFDMyNruJJDHCQzQgHU0jFBLZS6f3jeGRVvrQ50JXpOBU/T2RolDKUejpzhCpQM57E/E/r5Mo/7SX0ihOFInw7CM/YVBxOAkL9qkgWLGRJggLqneFOEACYaUjNXUIzvzJi6RZrZxVnKtqoV7L0siDA3AISsABNVAHF6ABmgCDB/AEXsCr8Wg8G2/G+6w1Z2Qz++APjM9vDQ2fdA==</latexit><latexit sha1_base64="KmuhbOlzzpDHBoE6U+fHBMjY5eQ=">AAACHnicbVDLSsNAFJ3UV42vqEs3g61QoZSkC6u7ghvBTQVrC20Jk+mkGTrJhJmJWEL9Ejf+ihsXKoIr/RunbRBtPXDhzDn3MvceL2ZUKtv+MnJLyyura/l1c2Nza3vH2t27kTwRmDQxZ1y0PSQJoxFpKqoYaceCoNBjpOUNzyd+65YISXl0rUYx6YVoEFGfYqS05FonRTNwnVLa9Xx4Nz4uw/vArf5+dlmfK1mGgXv5IxehaxXsij0FXCRORgogQ8O1Prp9jpOQRAozJGXHsWPVS5FQFDMyNruJJDHCQzQgHU0jFBLZS6f3jeGRVvrQ50JXpOBU/T2RolDKUejpzhCpQM57E/E/r5Mo/7SX0ihOFInw7CM/YVBxOAkL9qkgWLGRJggLqneFOEACYaUjNXUIzvzJi6RZrZxVnKtqoV7L0siDA3AISsABNVAHF6ABmgCDB/AEXsCr8Wg8G2/G+6w1Z2Qz++APjM9vDQ2fdA==</latexit><latexit sha1_base64="KmuhbOlzzpDHBoE6U+fHBMjY5eQ=">AAACHnicbVDLSsNAFJ3UV42vqEs3g61QoZSkC6u7ghvBTQVrC20Jk+mkGTrJhJmJWEL9Ejf+ihsXKoIr/RunbRBtPXDhzDn3MvceL2ZUKtv+MnJLyyura/l1c2Nza3vH2t27kTwRmDQxZ1y0PSQJoxFpKqoYaceCoNBjpOUNzyd+65YISXl0rUYx6YVoEFGfYqS05FonRTNwnVLa9Xx4Nz4uw/vArf5+dlmfK1mGgXv5IxehaxXsij0FXCRORgogQ8O1Prp9jpOQRAozJGXHsWPVS5FQFDMyNruJJDHCQzQgHU0jFBLZS6f3jeGRVvrQ50JXpOBU/T2RolDKUejpzhCpQM57E/E/r5Mo/7SX0ihOFInw7CM/YVBxOAkL9qkgWLGRJggLqneFOEACYaUjNXUIzvzJi6RZrZxVnKtqoV7L0siDA3AISsABNVAHF6ABmgCDB/AEXsCr8Wg8G2/G+6w1Z2Qz++APjM9vDQ2fdA==</latexit><latexit sha1_base64="KmuhbOlzzpDHBoE6U+fHBMjY5eQ=">AAACHnicbVDLSsNAFJ3UV42vqEs3g61QoZSkC6u7ghvBTQVrC20Jk+mkGTrJhJmJWEL9Ejf+ihsXKoIr/RunbRBtPXDhzDn3MvceL2ZUKtv+MnJLyyura/l1c2Nza3vH2t27kTwRmDQxZ1y0PSQJoxFpKqoYaceCoNBjpOUNzyd+65YISXl0rUYx6YVoEFGfYqS05FonRTNwnVLa9Xx4Nz4uw/vArf5+dlmfK1mGgXv5IxehaxXsij0FXCRORgogQ8O1Prp9jpOQRAozJGXHsWPVS5FQFDMyNruJJDHCQzQgHU0jFBLZS6f3jeGRVvrQ50JXpOBU/T2RolDKUejpzhCpQM57E/E/r5Mo/7SX0ihOFInw7CM/YVBxOAkL9qkgWLGRJggLqneFOEACYaUjNXUIzvzJi6RZrZxVnKtqoV7L0siDA3AISsABNVAHF6ABmgCDB/AEXsCr8Wg8G2/G+6w1Z2Qz++APjM9vDQ2fdA==</latexit>
H(x) = sign
"KX
k=1
↵khk(x)
#
<latexit sha1_base64="YI891tToZdmhPJ3sylEthqlHFwI=">AAACSnicbVBNb9QwEHWWQkv46AJHLhZbpHJZJb0UDpUqcanEpZVYWmkdIsc72VixncieVI2i7O/jwqW3/gkuHKDigne7EtDypJGe3psZe15WK+kwiq6Cwb2N+w82tx6Gjx4/ebo9fPb8k6saK2AiKlXZs4w7UNLABCUqOKstcJ0pOM3K90v/9Bysk5X5iG0NieZzI3MpOHopHWY74dFux7KcXvRv6AFlCBdodefk3PSUKchxulhQNpOuVrx12CpgrtFpVx7E/ecPlHFVFzwtaZGWfxYtFszKeYFJuEPT4SgaRyvQuyRekxFZ4zgdXrJZJRoNBoXizk3jqMak4xalUNCHrHFQc1HyOUw9NVyDS7pVFj197ZUZzSvryyBdqX9PdFw71+rMd2qOhbvtLcX/edMG87dJJ03dIBhx81DeKIoVXQZLZ9KCQNV6woWV/q9UFNxygT7+0IcQ3z75Lpnsjd+N45O90eH+Oo0t8pK8IrskJvvkkByRYzIhgnwh38gP8jP4GnwProNfN62DYD3zgvyDwcZvGNeydw==</latexit><latexit sha1_base64="YI891tToZdmhPJ3sylEthqlHFwI=">AAACSnicbVBNb9QwEHWWQkv46AJHLhZbpHJZJb0UDpUqcanEpZVYWmkdIsc72VixncieVI2i7O/jwqW3/gkuHKDigne7EtDypJGe3psZe15WK+kwiq6Cwb2N+w82tx6Gjx4/ebo9fPb8k6saK2AiKlXZs4w7UNLABCUqOKstcJ0pOM3K90v/9Bysk5X5iG0NieZzI3MpOHopHWY74dFux7KcXvRv6AFlCBdodefk3PSUKchxulhQNpOuVrx12CpgrtFpVx7E/ecPlHFVFzwtaZGWfxYtFszKeYFJuEPT4SgaRyvQuyRekxFZ4zgdXrJZJRoNBoXizk3jqMak4xalUNCHrHFQc1HyOUw9NVyDS7pVFj197ZUZzSvryyBdqX9PdFw71+rMd2qOhbvtLcX/edMG87dJJ03dIBhx81DeKIoVXQZLZ9KCQNV6woWV/q9UFNxygT7+0IcQ3z75Lpnsjd+N45O90eH+Oo0t8pK8IrskJvvkkByRYzIhgnwh38gP8jP4GnwProNfN62DYD3zgvyDwcZvGNeydw==</latexit><latexit sha1_base64="YI891tToZdmhPJ3sylEthqlHFwI=">AAACSnicbVBNb9QwEHWWQkv46AJHLhZbpHJZJb0UDpUqcanEpZVYWmkdIsc72VixncieVI2i7O/jwqW3/gkuHKDigne7EtDypJGe3psZe15WK+kwiq6Cwb2N+w82tx6Gjx4/ebo9fPb8k6saK2AiKlXZs4w7UNLABCUqOKstcJ0pOM3K90v/9Bysk5X5iG0NieZzI3MpOHopHWY74dFux7KcXvRv6AFlCBdodefk3PSUKchxulhQNpOuVrx12CpgrtFpVx7E/ecPlHFVFzwtaZGWfxYtFszKeYFJuEPT4SgaRyvQuyRekxFZ4zgdXrJZJRoNBoXizk3jqMak4xalUNCHrHFQc1HyOUw9NVyDS7pVFj197ZUZzSvryyBdqX9PdFw71+rMd2qOhbvtLcX/edMG87dJJ03dIBhx81DeKIoVXQZLZ9KCQNV6woWV/q9UFNxygT7+0IcQ3z75Lpnsjd+N45O90eH+Oo0t8pK8IrskJvvkkByRYzIhgnwh38gP8jP4GnwProNfN62DYD3zgvyDwcZvGNeydw==</latexit><latexit sha1_base64="YI891tToZdmhPJ3sylEthqlHFwI=">AAACSnicbVBNb9QwEHWWQkv46AJHLhZbpHJZJb0UDpUqcanEpZVYWmkdIsc72VixncieVI2i7O/jwqW3/gkuHKDigne7EtDypJGe3psZe15WK+kwiq6Cwb2N+w82tx6Gjx4/ebo9fPb8k6saK2AiKlXZs4w7UNLABCUqOKstcJ0pOM3K90v/9Bysk5X5iG0NieZzI3MpOHopHWY74dFux7KcXvRv6AFlCBdodefk3PSUKchxulhQNpOuVrx12CpgrtFpVx7E/ecPlHFVFzwtaZGWfxYtFszKeYFJuEPT4SgaRyvQuyRekxFZ4zgdXrJZJRoNBoXizk3jqMak4xalUNCHrHFQc1HyOUw9NVyDS7pVFj197ZUZzSvryyBdqX9PdFw71+rMd2qOhbvtLcX/edMG87dJJ03dIBhx81DeKIoVXQZLZ9KCQNV6woWV/q9UFNxygT7+0IcQ3z75Lpnsjd+N45O90eH+Oo0t8pK8IrskJvvkkByRYzIhgnwh38gP8jP4GnwProNfN62DYD3zgvyDwcZvGNeydw==</latexit>
hk(x)<latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit>
hk(x)<latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit>
↵k<latexit sha1_base64="QiyFHT5MzMU1IklJTRnj++Msrrc=">AAAB83icbVA9TwJBEJ3DLzy/UEubjWBiRe5o0I7ExhITT0zgQuaWPdiw9+HuHgm58DtsLNTY+mfs/DcucIWCL5nk5b2ZzMwLUsGVdpxvq7SxubW9U9619/YPDo8qxycPKskkZR5NRCIfA1RM8Jh5mmvBHlPJMAoE6wTjm7nfmTCpeBLf62nK/AiHMQ85RW0kv2b3UKQj7I/tGulXqk7dWYCsE7cgVSjQ7le+eoOEZhGLNRWoVNd1Uu3nKDWngs3sXqZYinSMQ9Y1NMaIKT9fHD0jF0YZkDCRpmJNFurviRwjpaZRYDoj1CO16s3F/7xupsMrP+dxmmkW0+WiMBNEJ2SeABlwyagWU0OQSm5uJXSEEqk2OdkmBHf15XXiNerXdfeuUW01izTKcAbncAkuNKEFt9AGDyg8wTO8wps1sV6sd+tj2VqyiplT+APr8wf0m5Br</latexit><latexit sha1_base64="QiyFHT5MzMU1IklJTRnj++Msrrc=">AAAB83icbVA9TwJBEJ3DLzy/UEubjWBiRe5o0I7ExhITT0zgQuaWPdiw9+HuHgm58DtsLNTY+mfs/DcucIWCL5nk5b2ZzMwLUsGVdpxvq7SxubW9U9619/YPDo8qxycPKskkZR5NRCIfA1RM8Jh5mmvBHlPJMAoE6wTjm7nfmTCpeBLf62nK/AiHMQ85RW0kv2b3UKQj7I/tGulXqk7dWYCsE7cgVSjQ7le+eoOEZhGLNRWoVNd1Uu3nKDWngs3sXqZYinSMQ9Y1NMaIKT9fHD0jF0YZkDCRpmJNFurviRwjpaZRYDoj1CO16s3F/7xupsMrP+dxmmkW0+WiMBNEJ2SeABlwyagWU0OQSm5uJXSEEqk2OdkmBHf15XXiNerXdfeuUW01izTKcAbncAkuNKEFt9AGDyg8wTO8wps1sV6sd+tj2VqyiplT+APr8wf0m5Br</latexit><latexit sha1_base64="QiyFHT5MzMU1IklJTRnj++Msrrc=">AAAB83icbVA9TwJBEJ3DLzy/UEubjWBiRe5o0I7ExhITT0zgQuaWPdiw9+HuHgm58DtsLNTY+mfs/DcucIWCL5nk5b2ZzMwLUsGVdpxvq7SxubW9U9619/YPDo8qxycPKskkZR5NRCIfA1RM8Jh5mmvBHlPJMAoE6wTjm7nfmTCpeBLf62nK/AiHMQ85RW0kv2b3UKQj7I/tGulXqk7dWYCsE7cgVSjQ7le+eoOEZhGLNRWoVNd1Uu3nKDWngs3sXqZYinSMQ9Y1NMaIKT9fHD0jF0YZkDCRpmJNFurviRwjpaZRYDoj1CO16s3F/7xupsMrP+dxmmkW0+WiMBNEJ2SeABlwyagWU0OQSm5uJXSEEqk2OdkmBHf15XXiNerXdfeuUW01izTKcAbncAkuNKEFt9AGDyg8wTO8wps1sV6sd+tj2VqyiplT+APr8wf0m5Br</latexit><latexit sha1_base64="QiyFHT5MzMU1IklJTRnj++Msrrc=">AAAB83icbVA9TwJBEJ3DLzy/UEubjWBiRe5o0I7ExhITT0zgQuaWPdiw9+HuHgm58DtsLNTY+mfs/DcucIWCL5nk5b2ZzMwLUsGVdpxvq7SxubW9U9619/YPDo8qxycPKskkZR5NRCIfA1RM8Jh5mmvBHlPJMAoE6wTjm7nfmTCpeBLf62nK/AiHMQ85RW0kv2b3UKQj7I/tGulXqk7dWYCsE7cgVSjQ7le+eoOEZhGLNRWoVNd1Uu3nKDWngs3sXqZYinSMQ9Y1NMaIKT9fHD0jF0YZkDCRpmJNFurviRwjpaZRYDoj1CO16s3F/7xupsMrP+dxmmkW0+WiMBNEJ2SeABlwyagWU0OQSm5uJXSEEqk2OdkmBHf15XXiNerXdfeuUW01izTKcAbncAkuNKEFt9AGDyg8wTO8wps1sV6sd+tj2VqyiplT+APr8wf0m5Br</latexit>
=
![Page 8: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/8.jpg)
The Plan
8
o Look at example of popular Boosting method
o Unpack it for intuition
o Come back later and show the math
Usual binary classification assumptions:
o Training set
o Feature vectors
o Labels are
{(xi, yi)}mi=1<latexit sha1_base64="sqjdef+WkBOANhOscs3Ed4OjZyM=">AAACFnicbVDLSsNAFJ34rPEVdelmsBUqSEm6qS6EghuXFawtNDVMppN26OTBzI1YQv7Cjb/ixoWKW3Hn3zh9LLT1wIXDOfdy7z1+IrgC2/42lpZXVtfWCxvm5tb2zq61t3+r4lRS1qSxiGXbJ4oJHrEmcBCsnUhGQl+wlj+8HPuteyYVj6MbGCWsG5J+xANOCWjJsyol0xUsADfD5cz1A/yQe/wUjzyOT1zJ+wNwcy/jF05+F5ol7FlFu2JPgBeJMyNFNEPDs77cXkzTkEVABVGq49gJdDMigVPBctNNFUsIHZI+62gakZCpbjb5K8fHWunhIJa6IsAT9fdERkKlRqGvO0MCAzXvjcX/vE4KwVk341GSAovodFGQCgwxHoeEe1wyCmKkCaGS61sxHRBJKOgoTR2CM//yImlWK+cV57parNdmaRTQITpCZeSgGqqjK9RATUTRI3pGr+jNeDJejHfjY9q6ZMxmDtAfGJ8/Aqedfg==</latexit><latexit sha1_base64="sqjdef+WkBOANhOscs3Ed4OjZyM=">AAACFnicbVDLSsNAFJ34rPEVdelmsBUqSEm6qS6EghuXFawtNDVMppN26OTBzI1YQv7Cjb/ixoWKW3Hn3zh9LLT1wIXDOfdy7z1+IrgC2/42lpZXVtfWCxvm5tb2zq61t3+r4lRS1qSxiGXbJ4oJHrEmcBCsnUhGQl+wlj+8HPuteyYVj6MbGCWsG5J+xANOCWjJsyol0xUsADfD5cz1A/yQe/wUjzyOT1zJ+wNwcy/jF05+F5ol7FlFu2JPgBeJMyNFNEPDs77cXkzTkEVABVGq49gJdDMigVPBctNNFUsIHZI+62gakZCpbjb5K8fHWunhIJa6IsAT9fdERkKlRqGvO0MCAzXvjcX/vE4KwVk341GSAovodFGQCgwxHoeEe1wyCmKkCaGS61sxHRBJKOgoTR2CM//yImlWK+cV57parNdmaRTQITpCZeSgGqqjK9RATUTRI3pGr+jNeDJejHfjY9q6ZMxmDtAfGJ8/Aqedfg==</latexit><latexit sha1_base64="sqjdef+WkBOANhOscs3Ed4OjZyM=">AAACFnicbVDLSsNAFJ34rPEVdelmsBUqSEm6qS6EghuXFawtNDVMppN26OTBzI1YQv7Cjb/ixoWKW3Hn3zh9LLT1wIXDOfdy7z1+IrgC2/42lpZXVtfWCxvm5tb2zq61t3+r4lRS1qSxiGXbJ4oJHrEmcBCsnUhGQl+wlj+8HPuteyYVj6MbGCWsG5J+xANOCWjJsyol0xUsADfD5cz1A/yQe/wUjzyOT1zJ+wNwcy/jF05+F5ol7FlFu2JPgBeJMyNFNEPDs77cXkzTkEVABVGq49gJdDMigVPBctNNFUsIHZI+62gakZCpbjb5K8fHWunhIJa6IsAT9fdERkKlRqGvO0MCAzXvjcX/vE4KwVk341GSAovodFGQCgwxHoeEe1wyCmKkCaGS61sxHRBJKOgoTR2CM//yImlWK+cV57parNdmaRTQITpCZeSgGqqjK9RATUTRI3pGr+jNeDJejHfjY9q6ZMxmDtAfGJ8/Aqedfg==</latexit><latexit sha1_base64="sqjdef+WkBOANhOscs3Ed4OjZyM=">AAACFnicbVDLSsNAFJ34rPEVdelmsBUqSEm6qS6EghuXFawtNDVMppN26OTBzI1YQv7Cjb/ixoWKW3Hn3zh9LLT1wIXDOfdy7z1+IrgC2/42lpZXVtfWCxvm5tb2zq61t3+r4lRS1qSxiGXbJ4oJHrEmcBCsnUhGQl+wlj+8HPuteyYVj6MbGCWsG5J+xANOCWjJsyol0xUsADfD5cz1A/yQe/wUjzyOT1zJ+wNwcy/jF05+F5ol7FlFu2JPgBeJMyNFNEPDs77cXkzTkEVABVGq49gJdDMigVPBctNNFUsIHZI+62gakZCpbjb5K8fHWunhIJa6IsAT9fdERkKlRqGvO0MCAzXvjcX/vE4KwVk341GSAovodFGQCgwxHoeEe1wyCmKkCaGS61sxHRBJKOgoTR2CM//yImlWK+cV57parNdmaRTQITpCZeSgGqqjK9RATUTRI3pGr+jNeDJejHfjY9q6ZMxmDtAfGJ8/Aqedfg==</latexit>
xi 2 Rp<latexit sha1_base64="K/IaIzQCvWEfm6x6GrBnNem7tRM=">AAACBnicbVA9T8MwEHX4LOErwIiELFokpirpUtgqsTAWRGilJkSO67RWHSeyHUQVdWPhr7AwAGLlN7Dxb3DaDNDypJOe3rvT3b0wZVQq2/42lpZXVtfWKxvm5tb2zq61t38rk0xg4uKEJaIbIkkY5cRVVDHSTQVBcchIJxxdFH7nnghJE36jxinxYzTgNKIYKS0F1lHNzL0wgg+TgEKPcujFSA3DML+e3KU1GFhVu25PAReJU5IqKNEOrC+vn+AsJlxhhqTsOXaq/BwJRTEjE9PLJEkRHqEB6WnKUUykn0//mMATrfRhlAhdXMGp+nsiR7GU4zjUncWVct4rxP+8XqaiMz+nPM0U4Xi2KMoYVAksQoF9KghWbKwJwoLqWyEeIoGw0tGZOgRn/uVF4jbq53XnqlFtNcs0KuAQHINT4IAmaIFL0AYuwOARPINX8GY8GS/Gu/Exa10yypkD8AfG5w9a25fh</latexit><latexit sha1_base64="K/IaIzQCvWEfm6x6GrBnNem7tRM=">AAACBnicbVA9T8MwEHX4LOErwIiELFokpirpUtgqsTAWRGilJkSO67RWHSeyHUQVdWPhr7AwAGLlN7Dxb3DaDNDypJOe3rvT3b0wZVQq2/42lpZXVtfWKxvm5tb2zq61t38rk0xg4uKEJaIbIkkY5cRVVDHSTQVBcchIJxxdFH7nnghJE36jxinxYzTgNKIYKS0F1lHNzL0wgg+TgEKPcujFSA3DML+e3KU1GFhVu25PAReJU5IqKNEOrC+vn+AsJlxhhqTsOXaq/BwJRTEjE9PLJEkRHqEB6WnKUUykn0//mMATrfRhlAhdXMGp+nsiR7GU4zjUncWVct4rxP+8XqaiMz+nPM0U4Xi2KMoYVAksQoF9KghWbKwJwoLqWyEeIoGw0tGZOgRn/uVF4jbq53XnqlFtNcs0KuAQHINT4IAmaIFL0AYuwOARPINX8GY8GS/Gu/Exa10yypkD8AfG5w9a25fh</latexit><latexit sha1_base64="K/IaIzQCvWEfm6x6GrBnNem7tRM=">AAACBnicbVA9T8MwEHX4LOErwIiELFokpirpUtgqsTAWRGilJkSO67RWHSeyHUQVdWPhr7AwAGLlN7Dxb3DaDNDypJOe3rvT3b0wZVQq2/42lpZXVtfWKxvm5tb2zq61t38rk0xg4uKEJaIbIkkY5cRVVDHSTQVBcchIJxxdFH7nnghJE36jxinxYzTgNKIYKS0F1lHNzL0wgg+TgEKPcujFSA3DML+e3KU1GFhVu25PAReJU5IqKNEOrC+vn+AsJlxhhqTsOXaq/BwJRTEjE9PLJEkRHqEB6WnKUUykn0//mMATrfRhlAhdXMGp+nsiR7GU4zjUncWVct4rxP+8XqaiMz+nPM0U4Xi2KMoYVAksQoF9KghWbKwJwoLqWyEeIoGw0tGZOgRn/uVF4jbq53XnqlFtNcs0KuAQHINT4IAmaIFL0AYuwOARPINX8GY8GS/Gu/Exa10yypkD8AfG5w9a25fh</latexit><latexit sha1_base64="K/IaIzQCvWEfm6x6GrBnNem7tRM=">AAACBnicbVA9T8MwEHX4LOErwIiELFokpirpUtgqsTAWRGilJkSO67RWHSeyHUQVdWPhr7AwAGLlN7Dxb3DaDNDypJOe3rvT3b0wZVQq2/42lpZXVtfWKxvm5tb2zq61t38rk0xg4uKEJaIbIkkY5cRVVDHSTQVBcchIJxxdFH7nnghJE36jxinxYzTgNKIYKS0F1lHNzL0wgg+TgEKPcujFSA3DML+e3KU1GFhVu25PAReJU5IqKNEOrC+vn+AsJlxhhqTsOXaq/BwJRTEjE9PLJEkRHqEB6WnKUUykn0//mMATrfRhlAhdXMGp+nsiR7GU4zjUncWVct4rxP+8XqaiMz+nPM0U4Xi2KMoYVAksQoF9KghWbKwJwoLqWyEeIoGw0tGZOgRn/uVF4jbq53XnqlFtNcs0KuAQHINT4IAmaIFL0AYuwOARPINX8GY8GS/Gu/Exa10yypkD8AfG5w9a25fh</latexit>
yi 2 {+1,�1}<latexit sha1_base64="wSYWNWE1VdElKwT63+784KrK8aA=">AAAB/nicbVBNS8NAEN3Urxq/ooIXL4utIKgl6aV6K3jxWMFYoQlhs920SzebsLsRQuzBv+LFg4pXf4c3/43bNgdtfTDweG+GmXlhyqhUtv1tVJaWV1bXquvmxubW9o61u3cnk0xg4uKEJeI+RJIwyomrqGLkPhUExSEj3XB0NfG7D0RImvBblafEj9GA04hipLQUWAd1Mw8o9CiHXnHqnMFzxxub9cCq2Q17CrhInJLUQIlOYH15/QRnMeEKMyRlz7FT5RdIKIoZGZteJkmK8AgNSE9TjmIi/WJ6/xgea6UPo0To4gpO1d8TBYqlzONQd8ZIDeW8NxH/83qZii78gvI0U4Tj2aIoY1AlcBIG7FNBsGK5JggLqm+FeIgEwkpHZuoQnPmXF4nbbFw2nJtmrd0q06iCQ3AEToADWqANrkEHuACDR/AMXsGb8WS8GO/Gx6y1YpQz++APjM8fFxSTFA==</latexit><latexit sha1_base64="wSYWNWE1VdElKwT63+784KrK8aA=">AAAB/nicbVBNS8NAEN3Urxq/ooIXL4utIKgl6aV6K3jxWMFYoQlhs920SzebsLsRQuzBv+LFg4pXf4c3/43bNgdtfTDweG+GmXlhyqhUtv1tVJaWV1bXquvmxubW9o61u3cnk0xg4uKEJeI+RJIwyomrqGLkPhUExSEj3XB0NfG7D0RImvBblafEj9GA04hipLQUWAd1Mw8o9CiHXnHqnMFzxxub9cCq2Q17CrhInJLUQIlOYH15/QRnMeEKMyRlz7FT5RdIKIoZGZteJkmK8AgNSE9TjmIi/WJ6/xgea6UPo0To4gpO1d8TBYqlzONQd8ZIDeW8NxH/83qZii78gvI0U4Tj2aIoY1AlcBIG7FNBsGK5JggLqm+FeIgEwkpHZuoQnPmXF4nbbFw2nJtmrd0q06iCQ3AEToADWqANrkEHuACDR/AMXsGb8WS8GO/Gx6y1YpQz++APjM8fFxSTFA==</latexit><latexit sha1_base64="wSYWNWE1VdElKwT63+784KrK8aA=">AAAB/nicbVBNS8NAEN3Urxq/ooIXL4utIKgl6aV6K3jxWMFYoQlhs920SzebsLsRQuzBv+LFg4pXf4c3/43bNgdtfTDweG+GmXlhyqhUtv1tVJaWV1bXquvmxubW9o61u3cnk0xg4uKEJeI+RJIwyomrqGLkPhUExSEj3XB0NfG7D0RImvBblafEj9GA04hipLQUWAd1Mw8o9CiHXnHqnMFzxxub9cCq2Q17CrhInJLUQIlOYH15/QRnMeEKMyRlz7FT5RdIKIoZGZteJkmK8AgNSE9TjmIi/WJ6/xgea6UPo0To4gpO1d8TBYqlzONQd8ZIDeW8NxH/83qZii78gvI0U4Tj2aIoY1AlcBIG7FNBsGK5JggLqm+FeIgEwkpHZuoQnPmXF4nbbFw2nJtmrd0q06iCQ3AEToADWqANrkEHuACDR/AMXsGb8WS8GO/Gx6y1YpQz++APjM8fFxSTFA==</latexit><latexit sha1_base64="wSYWNWE1VdElKwT63+784KrK8aA=">AAAB/nicbVBNS8NAEN3Urxq/ooIXL4utIKgl6aV6K3jxWMFYoQlhs920SzebsLsRQuzBv+LFg4pXf4c3/43bNgdtfTDweG+GmXlhyqhUtv1tVJaWV1bXquvmxubW9o61u3cnk0xg4uKEJeI+RJIwyomrqGLkPhUExSEj3XB0NfG7D0RImvBblafEj9GA04hipLQUWAd1Mw8o9CiHXnHqnMFzxxub9cCq2Q17CrhInJLUQIlOYH15/QRnMeEKMyRlz7FT5RdIKIoZGZteJkmK8AgNSE9TjmIi/WJ6/xgea6UPo0To4gpO1d8TBYqlzONQd8ZIDeW8NxH/83qZii78gvI0U4Tj2aIoY1AlcBIG7FNBsGK5JggLqm+FeIgEwkpHZuoQnPmXF4nbbFw2nJtmrd0q06iCQ3AEToADWqANrkEHuACDR/AMXsGb8WS8GO/Gx6y1YpQz++APjM8fFxSTFA==</latexit>
![Page 9: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/9.jpg)
Adaboost Algorithm
9
1. Initialize training example weights to uniform distribution
2. For
3. Output H(x) = signhPK
k=1 ↵khk(x)i
<latexit sha1_base64="kdKSZawqyJ2SMSjAvU8WHIIVoU4=">AAACOXicbVBNa9wwFJTTtE3dptm2x15ENoXksth7SXoIBHoJ5JJAN7uwdoysfV4LS7KRnksW49/VS39Fbz30kkNbeu0fqPYD8rEZEAwzb3h6k1ZSWAyCH97Gk82nz55vvfBfvtp+vdN58/bSlrXhMOClLM0oZRak0DBAgRJGlQGmUgnDtPg094dfwFhR6s84qyBWbKpFJjhDJyWdiz3/dL+J0oxetwf0mEYI12hUY8VUtzSSkOGYRrZWSVMch+3VGY2YrHKWFDRPittkZMQ0x9jfo0mnG/SCBeg6CVekS1Y4Tzrfo0nJawUauWTWjsOgwrhhBgWX0PpRbaFivGBTGDuqmQIbN4vTW/rBKROalcY9jXSh3k00TFk7U6mbVAxz+9Cbi4954xqzo7gRuqoRNF8uympJsaTzHulEGOAoZ44wboT7K+U5M4yja9t3JYQPT14ng37vYy+86HdPDldtbJH3ZJfsk5AckhNySs7JgHDylfwkv8hv75t34/3x/i5HN7xV5h25B+/ff/X7q1E=</latexit><latexit sha1_base64="kdKSZawqyJ2SMSjAvU8WHIIVoU4=">AAACOXicbVBNa9wwFJTTtE3dptm2x15ENoXksth7SXoIBHoJ5JJAN7uwdoysfV4LS7KRnksW49/VS39Fbz30kkNbeu0fqPYD8rEZEAwzb3h6k1ZSWAyCH97Gk82nz55vvfBfvtp+vdN58/bSlrXhMOClLM0oZRak0DBAgRJGlQGmUgnDtPg094dfwFhR6s84qyBWbKpFJjhDJyWdiz3/dL+J0oxetwf0mEYI12hUY8VUtzSSkOGYRrZWSVMch+3VGY2YrHKWFDRPittkZMQ0x9jfo0mnG/SCBeg6CVekS1Y4Tzrfo0nJawUauWTWjsOgwrhhBgWX0PpRbaFivGBTGDuqmQIbN4vTW/rBKROalcY9jXSh3k00TFk7U6mbVAxz+9Cbi4954xqzo7gRuqoRNF8uympJsaTzHulEGOAoZ44wboT7K+U5M4yja9t3JYQPT14ng37vYy+86HdPDldtbJH3ZJfsk5AckhNySs7JgHDylfwkv8hv75t34/3x/i5HN7xV5h25B+/ff/X7q1E=</latexit><latexit sha1_base64="kdKSZawqyJ2SMSjAvU8WHIIVoU4=">AAACOXicbVBNa9wwFJTTtE3dptm2x15ENoXksth7SXoIBHoJ5JJAN7uwdoysfV4LS7KRnksW49/VS39Fbz30kkNbeu0fqPYD8rEZEAwzb3h6k1ZSWAyCH97Gk82nz55vvfBfvtp+vdN58/bSlrXhMOClLM0oZRak0DBAgRJGlQGmUgnDtPg094dfwFhR6s84qyBWbKpFJjhDJyWdiz3/dL+J0oxetwf0mEYI12hUY8VUtzSSkOGYRrZWSVMch+3VGY2YrHKWFDRPittkZMQ0x9jfo0mnG/SCBeg6CVekS1Y4Tzrfo0nJawUauWTWjsOgwrhhBgWX0PpRbaFivGBTGDuqmQIbN4vTW/rBKROalcY9jXSh3k00TFk7U6mbVAxz+9Cbi4954xqzo7gRuqoRNF8uympJsaTzHulEGOAoZ44wboT7K+U5M4yja9t3JYQPT14ng37vYy+86HdPDldtbJH3ZJfsk5AckhNySs7JgHDylfwkv8hv75t34/3x/i5HN7xV5h25B+/ff/X7q1E=</latexit><latexit sha1_base64="kdKSZawqyJ2SMSjAvU8WHIIVoU4=">AAACOXicbVBNa9wwFJTTtE3dptm2x15ENoXksth7SXoIBHoJ5JJAN7uwdoysfV4LS7KRnksW49/VS39Fbz30kkNbeu0fqPYD8rEZEAwzb3h6k1ZSWAyCH97Gk82nz55vvfBfvtp+vdN58/bSlrXhMOClLM0oZRak0DBAgRJGlQGmUgnDtPg094dfwFhR6s84qyBWbKpFJjhDJyWdiz3/dL+J0oxetwf0mEYI12hUY8VUtzSSkOGYRrZWSVMch+3VGY2YrHKWFDRPittkZMQ0x9jfo0mnG/SCBeg6CVekS1Y4Tzrfo0nJawUauWTWjsOgwrhhBgWX0PpRbaFivGBTGDuqmQIbN4vTW/rBKROalcY9jXSh3k00TFk7U6mbVAxz+9Cbi4954xqzo7gRuqoRNF8uympJsaTzHulEGOAoZ44wboT7K+U5M4yja9t3JYQPT14ng37vYy+86HdPDldtbJH3ZJfsk5AckhNySs7JgHDylfwkv8hv75t34/3x/i5HN7xV5h25B+/ff/X7q1E=</latexit>
wi =1
mfor i = 1, 2, . . . ,m
<latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit><latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit><latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit><latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit>
k = 1, 2, . . . ,K :<latexit sha1_base64="RGAPVcQTsuavolgGhqL8ZCNpDfs=">AAACAHicbVBNS8NAEN34WeNX1IvgZbEVPJSS9FIVhIIXwUsFYwttKJvNpl262Q27G6GUevGvePGg4tWf4c1/47bNQVsfDDzem2FmXpgyqrTrfltLyyura+uFDXtza3tn19nbv1cik5j4WDAhWyFShFFOfE01I61UEpSEjDTDwdXEbz4Qqajgd3qYkiBBPU5jipE2Utc5LNkDeAm9MqyWYYdFQqsyvLkowa5TdCvuFHCReDkpghyNrvPViQTOEsI1ZkiptuemOhghqSlmZGx3MkVShAeoR9qGcpQQFYymH4zhiVEiGAtpims4VX9PjFCi1DAJTWeCdF/NexPxP6+d6fgsGFGeZppwPFsUZwxqASdxwIhKgjUbGoKwpOZWiPtIIqxNaLYJwZt/eZH41cp5xbutFuu1PI0COALH4BR4oAbq4Bo0gA8weATP4BW8WU/Wi/Vufcxal6x85gD8gfX5AzBakwo=</latexit><latexit sha1_base64="RGAPVcQTsuavolgGhqL8ZCNpDfs=">AAACAHicbVBNS8NAEN34WeNX1IvgZbEVPJSS9FIVhIIXwUsFYwttKJvNpl262Q27G6GUevGvePGg4tWf4c1/47bNQVsfDDzem2FmXpgyqrTrfltLyyura+uFDXtza3tn19nbv1cik5j4WDAhWyFShFFOfE01I61UEpSEjDTDwdXEbz4Qqajgd3qYkiBBPU5jipE2Utc5LNkDeAm9MqyWYYdFQqsyvLkowa5TdCvuFHCReDkpghyNrvPViQTOEsI1ZkiptuemOhghqSlmZGx3MkVShAeoR9qGcpQQFYymH4zhiVEiGAtpims4VX9PjFCi1DAJTWeCdF/NexPxP6+d6fgsGFGeZppwPFsUZwxqASdxwIhKgjUbGoKwpOZWiPtIIqxNaLYJwZt/eZH41cp5xbutFuu1PI0COALH4BR4oAbq4Bo0gA8weATP4BW8WU/Wi/Vufcxal6x85gD8gfX5AzBakwo=</latexit><latexit sha1_base64="RGAPVcQTsuavolgGhqL8ZCNpDfs=">AAACAHicbVBNS8NAEN34WeNX1IvgZbEVPJSS9FIVhIIXwUsFYwttKJvNpl262Q27G6GUevGvePGg4tWf4c1/47bNQVsfDDzem2FmXpgyqrTrfltLyyura+uFDXtza3tn19nbv1cik5j4WDAhWyFShFFOfE01I61UEpSEjDTDwdXEbz4Qqajgd3qYkiBBPU5jipE2Utc5LNkDeAm9MqyWYYdFQqsyvLkowa5TdCvuFHCReDkpghyNrvPViQTOEsI1ZkiptuemOhghqSlmZGx3MkVShAeoR9qGcpQQFYymH4zhiVEiGAtpims4VX9PjFCi1DAJTWeCdF/NexPxP6+d6fgsGFGeZppwPFsUZwxqASdxwIhKgjUbGoKwpOZWiPtIIqxNaLYJwZt/eZH41cp5xbutFuu1PI0COALH4BR4oAbq4Bo0gA8weATP4BW8WU/Wi/Vufcxal6x85gD8gfX5AzBakwo=</latexit><latexit sha1_base64="RGAPVcQTsuavolgGhqL8ZCNpDfs=">AAACAHicbVBNS8NAEN34WeNX1IvgZbEVPJSS9FIVhIIXwUsFYwttKJvNpl262Q27G6GUevGvePGg4tWf4c1/47bNQVsfDDzem2FmXpgyqrTrfltLyyura+uFDXtza3tn19nbv1cik5j4WDAhWyFShFFOfE01I61UEpSEjDTDwdXEbz4Qqajgd3qYkiBBPU5jipE2Utc5LNkDeAm9MqyWYYdFQqsyvLkowa5TdCvuFHCReDkpghyNrvPViQTOEsI1ZkiptuemOhghqSlmZGx3MkVShAeoR9qGcpQQFYymH4zhiVEiGAtpims4VX9PjFCi1DAJTWeCdF/NexPxP6+d6fgsGFGeZppwPFsUZwxqASdxwIhKgjUbGoKwpOZWiPtIIqxNaLYJwZt/eZH41cp5xbutFuu1PI0COALH4BR4oAbq4Bo0gA8weATP4BW8WU/Wi/Vufcxal6x85gD8gfX5AzBakwo=</latexit>
a) Fit classifier to training data with weights
b) Compute weighted error
c) Compute vote weight
d) Update training example weights
hk(x)<latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit>
errk =Pm
i=1 wi·I(yi 6=hk(xi))Pmi=1 wi
<latexit sha1_base64="DYudvI+ljGimk1yuXdxu0qV4n14=">AAACSHicbZBLbxMxFIU9KY8yvNJ2yeaKFCndRDPdFBaVKrGBXSsRWikTjMe501ixPYN9hzYazd9jw45d/wMbFgWxw0mzgIYjWTo6517Z/vJKK09JchV1Nu7cvXd/80H88NHjJ0+7W9vvfVk7iUNZ6tKd5cKjVhaHpEjjWeVQmFzjaT57vehPP6PzqrTvaF7h2IhzqwolBYWIdz/uxhnhJTnToHMtn8EhZIUTssl8bXijDtP2g4ELrjI5KQne9udcQWbxE0z5rN9keQGXLVd7e9CurbTxLvBuLxkkS8G6SVemx1Y65t1v2aSUtUFLUgvvR2lS0bgRjpTU2MZZ7bEScibOcRSsFQb9uFmSaOFFSCZQlC4cS7BM/95ohPF+bvIwaQRN/e1uEf6vG9VUvBw3ylY1oZU3FxW1BiphgRUmyqEkPQ9GSKfCW0FOReBIAX4cIKS3v7xuhvuDV4P0ZL93dLCiscmeseesz1J2wI7YG3bMhkyyL+w7u2Y/o6/Rj+hX9PtmtBOtdnbYP+p0/gDKcbEx</latexit><latexit sha1_base64="DYudvI+ljGimk1yuXdxu0qV4n14=">AAACSHicbZBLbxMxFIU9KY8yvNJ2yeaKFCndRDPdFBaVKrGBXSsRWikTjMe501ixPYN9hzYazd9jw45d/wMbFgWxw0mzgIYjWTo6517Z/vJKK09JchV1Nu7cvXd/80H88NHjJ0+7W9vvfVk7iUNZ6tKd5cKjVhaHpEjjWeVQmFzjaT57vehPP6PzqrTvaF7h2IhzqwolBYWIdz/uxhnhJTnToHMtn8EhZIUTssl8bXijDtP2g4ELrjI5KQne9udcQWbxE0z5rN9keQGXLVd7e9CurbTxLvBuLxkkS8G6SVemx1Y65t1v2aSUtUFLUgvvR2lS0bgRjpTU2MZZ7bEScibOcRSsFQb9uFmSaOFFSCZQlC4cS7BM/95ohPF+bvIwaQRN/e1uEf6vG9VUvBw3ylY1oZU3FxW1BiphgRUmyqEkPQ9GSKfCW0FOReBIAX4cIKS3v7xuhvuDV4P0ZL93dLCiscmeseesz1J2wI7YG3bMhkyyL+w7u2Y/o6/Rj+hX9PtmtBOtdnbYP+p0/gDKcbEx</latexit><latexit sha1_base64="DYudvI+ljGimk1yuXdxu0qV4n14=">AAACSHicbZBLbxMxFIU9KY8yvNJ2yeaKFCndRDPdFBaVKrGBXSsRWikTjMe501ixPYN9hzYazd9jw45d/wMbFgWxw0mzgIYjWTo6517Z/vJKK09JchV1Nu7cvXd/80H88NHjJ0+7W9vvfVk7iUNZ6tKd5cKjVhaHpEjjWeVQmFzjaT57vehPP6PzqrTvaF7h2IhzqwolBYWIdz/uxhnhJTnToHMtn8EhZIUTssl8bXijDtP2g4ELrjI5KQne9udcQWbxE0z5rN9keQGXLVd7e9CurbTxLvBuLxkkS8G6SVemx1Y65t1v2aSUtUFLUgvvR2lS0bgRjpTU2MZZ7bEScibOcRSsFQb9uFmSaOFFSCZQlC4cS7BM/95ohPF+bvIwaQRN/e1uEf6vG9VUvBw3ylY1oZU3FxW1BiphgRUmyqEkPQ9GSKfCW0FOReBIAX4cIKS3v7xuhvuDV4P0ZL93dLCiscmeseesz1J2wI7YG3bMhkyyL+w7u2Y/o6/Rj+hX9PtmtBOtdnbYP+p0/gDKcbEx</latexit><latexit sha1_base64="DYudvI+ljGimk1yuXdxu0qV4n14=">AAACSHicbZBLbxMxFIU9KY8yvNJ2yeaKFCndRDPdFBaVKrGBXSsRWikTjMe501ixPYN9hzYazd9jw45d/wMbFgWxw0mzgIYjWTo6517Z/vJKK09JchV1Nu7cvXd/80H88NHjJ0+7W9vvfVk7iUNZ6tKd5cKjVhaHpEjjWeVQmFzjaT57vehPP6PzqrTvaF7h2IhzqwolBYWIdz/uxhnhJTnToHMtn8EhZIUTssl8bXijDtP2g4ELrjI5KQne9udcQWbxE0z5rN9keQGXLVd7e9CurbTxLvBuLxkkS8G6SVemx1Y65t1v2aSUtUFLUgvvR2lS0bgRjpTU2MZZ7bEScibOcRSsFQb9uFmSaOFFSCZQlC4cS7BM/95ohPF+bvIwaQRN/e1uEf6vG9VUvBw3ylY1oZU3FxW1BiphgRUmyqEkPQ9GSKfCW0FOReBIAX4cIKS3v7xuhvuDV4P0ZL93dLCiscmeseesz1J2wI7YG3bMhkyyL+w7u2Y/o6/Rj+hX9PtmtBOtdnbYP+p0/gDKcbEx</latexit>
↵k = 12 log((1� errk)/errk)
<latexit sha1_base64="SfQna/o3NSXYOMTJbcHkpR5TUR0=">AAACKnicbVC7TsNAEDzzxrwClDQnAlIoCHaaQIGEoKEMEgGkOLLOl3VyyvmhuzUisvw/NPwKBRQQ0fIhXEIKCIy00tzMrm53glQKjY4ztGZm5+YXFpeW7ZXVtfWN0ubWjU4yxaHJE5mou4BpkCKGJgqUcJcqYFEg4TboX4z823tQWiTxNQ5SaEesG4tQcIZG8kvne7bHZNpjfp+eUi9UjOdukdcKTybdSsU99BAeUEU5KFX4/YOjqbe9R/1S2ak6Y9C/xJ2QMpmg4ZdevE7Cswhi5JJp3XKdFNs5Uyi4hML2Mg0p433WhZahMYtAt/PxrQXdN0qHhokyFSMdqz8nchZpPYgC0xkx7OlpbyT+57UyDI/buYjTDCHm3x+FmaSY0FFwtCMUcJQDQxhXwuxKeY+ZvNDEa5sQ3OmT/5JmrXpSda9q5bP6JI0lskN2SYW4pE7OyCVpkCbh5JE8kzfybj1Zr9bQ+vhunbEmM9vkF6zPL7a4plk=</latexit><latexit sha1_base64="SfQna/o3NSXYOMTJbcHkpR5TUR0=">AAACKnicbVC7TsNAEDzzxrwClDQnAlIoCHaaQIGEoKEMEgGkOLLOl3VyyvmhuzUisvw/NPwKBRQQ0fIhXEIKCIy00tzMrm53glQKjY4ztGZm5+YXFpeW7ZXVtfWN0ubWjU4yxaHJE5mou4BpkCKGJgqUcJcqYFEg4TboX4z823tQWiTxNQ5SaEesG4tQcIZG8kvne7bHZNpjfp+eUi9UjOdukdcKTybdSsU99BAeUEU5KFX4/YOjqbe9R/1S2ak6Y9C/xJ2QMpmg4ZdevE7Cswhi5JJp3XKdFNs5Uyi4hML2Mg0p433WhZahMYtAt/PxrQXdN0qHhokyFSMdqz8nchZpPYgC0xkx7OlpbyT+57UyDI/buYjTDCHm3x+FmaSY0FFwtCMUcJQDQxhXwuxKeY+ZvNDEa5sQ3OmT/5JmrXpSda9q5bP6JI0lskN2SYW4pE7OyCVpkCbh5JE8kzfybj1Zr9bQ+vhunbEmM9vkF6zPL7a4plk=</latexit><latexit sha1_base64="SfQna/o3NSXYOMTJbcHkpR5TUR0=">AAACKnicbVC7TsNAEDzzxrwClDQnAlIoCHaaQIGEoKEMEgGkOLLOl3VyyvmhuzUisvw/NPwKBRQQ0fIhXEIKCIy00tzMrm53glQKjY4ztGZm5+YXFpeW7ZXVtfWN0ubWjU4yxaHJE5mou4BpkCKGJgqUcJcqYFEg4TboX4z823tQWiTxNQ5SaEesG4tQcIZG8kvne7bHZNpjfp+eUi9UjOdukdcKTybdSsU99BAeUEU5KFX4/YOjqbe9R/1S2ak6Y9C/xJ2QMpmg4ZdevE7Cswhi5JJp3XKdFNs5Uyi4hML2Mg0p433WhZahMYtAt/PxrQXdN0qHhokyFSMdqz8nchZpPYgC0xkx7OlpbyT+57UyDI/buYjTDCHm3x+FmaSY0FFwtCMUcJQDQxhXwuxKeY+ZvNDEa5sQ3OmT/5JmrXpSda9q5bP6JI0lskN2SYW4pE7OyCVpkCbh5JE8kzfybj1Zr9bQ+vhunbEmM9vkF6zPL7a4plk=</latexit><latexit sha1_base64="SfQna/o3NSXYOMTJbcHkpR5TUR0=">AAACKnicbVC7TsNAEDzzxrwClDQnAlIoCHaaQIGEoKEMEgGkOLLOl3VyyvmhuzUisvw/NPwKBRQQ0fIhXEIKCIy00tzMrm53glQKjY4ztGZm5+YXFpeW7ZXVtfWN0ubWjU4yxaHJE5mou4BpkCKGJgqUcJcqYFEg4TboX4z823tQWiTxNQ5SaEesG4tQcIZG8kvne7bHZNpjfp+eUi9UjOdukdcKTybdSsU99BAeUEU5KFX4/YOjqbe9R/1S2ak6Y9C/xJ2QMpmg4ZdevE7Cswhi5JJp3XKdFNs5Uyi4hML2Mg0p433WhZahMYtAt/PxrQXdN0qHhokyFSMdqz8nchZpPYgC0xkx7OlpbyT+57UyDI/buYjTDCHm3x+FmaSY0FFwtCMUcJQDQxhXwuxKeY+ZvNDEa5sQ3OmT/5JmrXpSda9q5bP6JI0lskN2SYW4pE7OyCVpkCbh5JE8kzfybj1Zr9bQ+vhunbEmM9vkF6zPL7a4plk=</latexit>
wi<latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit><latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit><latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit><latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit>
wi 1Zk
· wi exp [�↵kyihk(xi)]<latexit sha1_base64="+Md+EfCAFodoBKQgI79wGNjvoIc=">AAACQXicbVA9b9QwGHbKVxu+rjCyWFyRysAp6VK6VWJhLBJHS89R9Mb35mLFiSP7De0pym9j4Rd06w9gYWgRaxd81xtoyyNZevR8yPaTNVo5iqLzYO3e/QcPH61vhI+fPH32fLD54oszrZU4lkYbe5SBQ61qHJMijUeNRagyjYdZ+WHhH35D65SpP9O8waSCWa1yJYG8lA6+boUnqeJCY05grTnhIrcgu7jvjtOy50JODXEfEXjaLFOTdwJ0U0Ba8rlvFmm53Yks56d9qt4Kq2YFJTzc4ulgGI2iJfhdEq/IkK1wkA7OxNTItsKapAbnJnHUUNKBJSU19qFoHTYgS5jhxNMaKnRJt5yg52+8MuW5sf7UxJfqv40OKufmVeaTFVDhbnsL8X/epKX8fdKpumkJa3l9Ud5qToYv9uRTZVGSnnsC0ir/Vi4L8BOSXz30I8S3v3yXjHdGe6P4085wf3e1xjp7xV6zbRazXbbPPrIDNmaSfWc/2QW7DH4Ev4LfwZ/r6Fqw6rxkNxBc/QWQoK+2</latexit><latexit sha1_base64="+Md+EfCAFodoBKQgI79wGNjvoIc=">AAACQXicbVA9b9QwGHbKVxu+rjCyWFyRysAp6VK6VWJhLBJHS89R9Mb35mLFiSP7De0pym9j4Rd06w9gYWgRaxd81xtoyyNZevR8yPaTNVo5iqLzYO3e/QcPH61vhI+fPH32fLD54oszrZU4lkYbe5SBQ61qHJMijUeNRagyjYdZ+WHhH35D65SpP9O8waSCWa1yJYG8lA6+boUnqeJCY05grTnhIrcgu7jvjtOy50JODXEfEXjaLFOTdwJ0U0Ba8rlvFmm53Yks56d9qt4Kq2YFJTzc4ulgGI2iJfhdEq/IkK1wkA7OxNTItsKapAbnJnHUUNKBJSU19qFoHTYgS5jhxNMaKnRJt5yg52+8MuW5sf7UxJfqv40OKufmVeaTFVDhbnsL8X/epKX8fdKpumkJa3l9Ud5qToYv9uRTZVGSnnsC0ir/Vi4L8BOSXz30I8S3v3yXjHdGe6P4085wf3e1xjp7xV6zbRazXbbPPrIDNmaSfWc/2QW7DH4Ev4LfwZ/r6Fqw6rxkNxBc/QWQoK+2</latexit><latexit sha1_base64="+Md+EfCAFodoBKQgI79wGNjvoIc=">AAACQXicbVA9b9QwGHbKVxu+rjCyWFyRysAp6VK6VWJhLBJHS89R9Mb35mLFiSP7De0pym9j4Rd06w9gYWgRaxd81xtoyyNZevR8yPaTNVo5iqLzYO3e/QcPH61vhI+fPH32fLD54oszrZU4lkYbe5SBQ61qHJMijUeNRagyjYdZ+WHhH35D65SpP9O8waSCWa1yJYG8lA6+boUnqeJCY05grTnhIrcgu7jvjtOy50JODXEfEXjaLFOTdwJ0U0Ba8rlvFmm53Yks56d9qt4Kq2YFJTzc4ulgGI2iJfhdEq/IkK1wkA7OxNTItsKapAbnJnHUUNKBJSU19qFoHTYgS5jhxNMaKnRJt5yg52+8MuW5sf7UxJfqv40OKufmVeaTFVDhbnsL8X/epKX8fdKpumkJa3l9Ud5qToYv9uRTZVGSnnsC0ir/Vi4L8BOSXz30I8S3v3yXjHdGe6P4085wf3e1xjp7xV6zbRazXbbPPrIDNmaSfWc/2QW7DH4Ev4LfwZ/r6Fqw6rxkNxBc/QWQoK+2</latexit><latexit sha1_base64="+Md+EfCAFodoBKQgI79wGNjvoIc=">AAACQXicbVA9b9QwGHbKVxu+rjCyWFyRysAp6VK6VWJhLBJHS89R9Mb35mLFiSP7De0pym9j4Rd06w9gYWgRaxd81xtoyyNZevR8yPaTNVo5iqLzYO3e/QcPH61vhI+fPH32fLD54oszrZU4lkYbe5SBQ61qHJMijUeNRagyjYdZ+WHhH35D65SpP9O8waSCWa1yJYG8lA6+boUnqeJCY05grTnhIrcgu7jvjtOy50JODXEfEXjaLFOTdwJ0U0Ba8rlvFmm53Yks56d9qt4Kq2YFJTzc4ulgGI2iJfhdEq/IkK1wkA7OxNTItsKapAbnJnHUUNKBJSU19qFoHTYgS5jhxNMaKnRJt5yg52+8MuW5sf7UxJfqv40OKufmVeaTFVDhbnsL8X/epKX8fdKpumkJa3l9Ud5qToYv9uRTZVGSnnsC0ir/Vi4L8BOSXz30I8S3v3yXjHdGe6P4085wf3e1xjp7xV6zbRazXbbPPrIDNmaSfWc/2QW7DH4Ev4LfwZ/r6Fqw6rxkNxBc/QWQoK+2</latexit>
7
=( ADDING
MORE WEAK LEARNERS )
![Page 10: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/10.jpg)
Adaboost Algorithm
10
1. Initialize training example weights to uniform distribution
Weights are initialized to uniform dist. At beginning, all training examples treated the same
wi =1
mfor i = 1, 2, . . . ,m
<latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit><latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit><latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit><latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit>
![Page 11: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/11.jpg)
Adaboost Algorithm
11
1. Initialize training example weights to uniform distribution
2a) Fit classifier to training data with weights
Decide Decision Stump split based on Impurity and Information gain as usual
Before: p was fraction of examples with positive label
Now: p is fraction of weights in pos class
wi =1
mfor i = 1, 2, . . . ,m
<latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit><latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit><latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit><latexit sha1_base64="oPRkYAeIMeueNJ5fkgd+uMoDQ2s=">AAACIHicbVC7TsNAEDzzxrwClDQrAhJFFNlpgAIJiYYSJAJIcRSdz2c4ceez7tZAZPlXaPgVGgpA0MHXcAkpeE01mtnVzk6cS2ExCN69sfGJyanpmVl/bn5hcam2vHJqdWEYbzMttTmPqeVSZLyNAiU/zw2nKpb8LL46GPhn19xYobMT7Oe8q+hFJlLBKDqpV9vZ8G96AvYgSlJDWRlWpaogQn6LRpWQagMViL2wAa0GRDLRaBug/A3o1epBMxgC/pJwROpkhKNe7S1KNCsUz5BJam0nDHLsltSgYJJXflRYnlN2RS94x9GMKm675fDDCjadkgzTpDpDGKrfN0qqrO2r2E0qipf2tzcQ//M6BaY73VJkeYE8Y1+H0kICahjUBYkwnKHsO0KZES4rsEvqmkJXqu9KCH+//Je0W83dZnjcqu9vj9qYIWtknWyRkGyTfXJIjkibMHJHHsgTefbuvUfvxXv9Gh3zRjur5Ae8j0/iVaBf</latexit>
hk(x)<latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit><latexit sha1_base64="3vQxizms4nIQmSBPqwBJtpLqCv8=">AAAB+nicbVBNT8JAEJ3iF9avikcvG8EEL6Tlgt5IvHjERIQEmma7bGHD9iO7WwNp+le8eFDj1V/izX/jAj0o+JJJXt6bycw8P+FMKtv+Nkpb2zu7e+V98+Dw6PjEOq08yjgVhHZJzGPR97GknEW0q5jitJ8IikOf054/vV34vScqJIujBzVPqBviccQCRrDSkmdVaubEm9azoR+gWX6FzBryrKrdsJdAm8QpSBUKdDzraziKSRrSSBGOpRw4dqLcDAvFCKe5OUwlTTCZ4jEdaBrhkEo3W96eo0utjFAQC12RQkv190SGQynnoa87Q6wmct1biP95g1QF127GoiRVNCKrRUHKkYrRIgg0YoISxeeaYCKYvhWRCRaYKB2XqUNw1l/eJN1m46bh3Der7VaRRhnO4QLq4EAL2nAHHegCgRk8wyu8GbnxYrwbH6vWklHMnMEfGJ8//1uR+Q==</latexit>
wi<latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit><latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit><latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit><latexit sha1_base64="oDiG3MzYIndo1l38OySeJflDm6w=">AAAB7nicbVBNT8JAEJ36ifUL9ehlI5h4Ii0X9EbixSMmVkigIdtlCxt2t3V3qyENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMyLUs608bxvZ219Y3Nru7Tj7u7tHxyWj47vdZIpQgOS8ER1IqwpZ5IGhhlOO6miWESctqPx9cxvP1KlWSLvzCSlocBDyWJGsLFSp+o+9ZlbRf1yxat5c6BV4hekAgVa/fJXb5CQTFBpCMdad30vNWGOlWGE06nbyzRNMRnjIe1aKrGgOszn907RuVUGKE6ULWnQXP09kWOh9UREtlNgM9LL3kz8z+tmJr4McybTzFBJFovijCOToNnzaMAUJYZPLMFEMXsrIiOsMDE2IteG4C+/vEqCeu2q5t/WK81GkUYJTuEMLsCHBjThBloQAAEOz/AKb86D8+K8Ox+L1jWnmDmBP3A+fwBB/45M</latexit>
I(D) = �p log2 p� (1� p) log2(1� p)<latexit sha1_base64="LA6upKk73lmL6XceCf7/jU3useA=">AAACEHicbZC9TsMwFIUdfkv4CzCyWLRI7dAq6VIYkCrBAFuRCK3URpXjOq1VJ7FsB6mK+gosvAoLAyBWRjbeBjfNAC1HsvT53Htl3+NzRqWy7W9jZXVtfWOzsGVu7+zu7VsHh/cyTgQmLo5ZLDo+koTRiLiKKkY6XBAU+oy0/fHlrN5+IELSOLpTE068EA0jGlCMlLb6Vrlk3pSvKvACVnmPxcN+ncMqLDtVXplfMzRLsG8V7ZqdCS6Dk0MR5Gr1ra/eIMZJSCKFGZKy69hceSkSimJGpmYvkYQjPEZD0tUYoZBIL802msJT7QxgEAt9IgUz9/dEikIpJ6GvO0OkRnKxNjP/q3UTFZx5KY14okiE5w8FCYMqhrN44IAKghWbaEBYUP1XiEdIIKx0iKYOwVlceRnceu285tzWi81GnkYBHIMTUAYOaIAmuAYt4AIMHsEzeAVvxpPxYrwbH/PWFSOfOQJ/ZHz+AH1Bl+8=</latexit><latexit sha1_base64="LA6upKk73lmL6XceCf7/jU3useA=">AAACEHicbZC9TsMwFIUdfkv4CzCyWLRI7dAq6VIYkCrBAFuRCK3URpXjOq1VJ7FsB6mK+gosvAoLAyBWRjbeBjfNAC1HsvT53Htl3+NzRqWy7W9jZXVtfWOzsGVu7+zu7VsHh/cyTgQmLo5ZLDo+koTRiLiKKkY6XBAU+oy0/fHlrN5+IELSOLpTE068EA0jGlCMlLb6Vrlk3pSvKvACVnmPxcN+ncMqLDtVXplfMzRLsG8V7ZqdCS6Dk0MR5Gr1ra/eIMZJSCKFGZKy69hceSkSimJGpmYvkYQjPEZD0tUYoZBIL802msJT7QxgEAt9IgUz9/dEikIpJ6GvO0OkRnKxNjP/q3UTFZx5KY14okiE5w8FCYMqhrN44IAKghWbaEBYUP1XiEdIIKx0iKYOwVlceRnceu285tzWi81GnkYBHIMTUAYOaIAmuAYt4AIMHsEzeAVvxpPxYrwbH/PWFSOfOQJ/ZHz+AH1Bl+8=</latexit><latexit sha1_base64="LA6upKk73lmL6XceCf7/jU3useA=">AAACEHicbZC9TsMwFIUdfkv4CzCyWLRI7dAq6VIYkCrBAFuRCK3URpXjOq1VJ7FsB6mK+gosvAoLAyBWRjbeBjfNAC1HsvT53Htl3+NzRqWy7W9jZXVtfWOzsGVu7+zu7VsHh/cyTgQmLo5ZLDo+koTRiLiKKkY6XBAU+oy0/fHlrN5+IELSOLpTE068EA0jGlCMlLb6Vrlk3pSvKvACVnmPxcN+ncMqLDtVXplfMzRLsG8V7ZqdCS6Dk0MR5Gr1ra/eIMZJSCKFGZKy69hceSkSimJGpmYvkYQjPEZD0tUYoZBIL802msJT7QxgEAt9IgUz9/dEikIpJ6GvO0OkRnKxNjP/q3UTFZx5KY14okiE5w8FCYMqhrN44IAKghWbaEBYUP1XiEdIIKx0iKYOwVlceRnceu285tzWi81GnkYBHIMTUAYOaIAmuAYt4AIMHsEzeAVvxpPxYrwbH/PWFSOfOQJ/ZHz+AH1Bl+8=</latexit><latexit sha1_base64="LA6upKk73lmL6XceCf7/jU3useA=">AAACEHicbZC9TsMwFIUdfkv4CzCyWLRI7dAq6VIYkCrBAFuRCK3URpXjOq1VJ7FsB6mK+gosvAoLAyBWRjbeBjfNAC1HsvT53Htl3+NzRqWy7W9jZXVtfWOzsGVu7+zu7VsHh/cyTgQmLo5ZLDo+koTRiLiKKkY6XBAU+oy0/fHlrN5+IELSOLpTE068EA0jGlCMlLb6Vrlk3pSvKvACVnmPxcN+ncMqLDtVXplfMzRLsG8V7ZqdCS6Dk0MR5Gr1ra/eIMZJSCKFGZKy69hceSkSimJGpmYvkYQjPEZD0tUYoZBIL802msJT7QxgEAt9IgUz9/dEikIpJ6GvO0OkRnKxNjP/q3UTFZx5KY14okiE5w8FCYMqhrN44IAKghWbaEBYUP1XiEdIIKx0iKYOwVlceRnceu285tzWi81GnkYBHIMTUAYOaIAmuAYt4AIMHsEzeAVvxpPxYrwbH/PWFSOfOQJ/ZHz+AH1Bl+8=</latexit>
p =
Pmi=1 wi · I(yi = +1)Pm
i=1 wi<latexit sha1_base64="9LEm29yj4UjovgL3uLk/JI0n8kM=">AAACLXicbVDLSsNAFJ3UV42vqks3g61QEUrSTXVRKIiguwrWFpoaJpNJO3QmCTMTpYR8kRt/RReCD9z6G04fC209MHA451zu3OPFjEplWW9Gbml5ZXUtv25ubG5t7xR2925llAhMWjhikeh4SBJGQ9JSVDHSiQVB3GOk7Q3Px377nghJo/BGjWLS46gf0oBipLTkFi5KZgzr0PEDgXDqyIS7Ka3b2R2HDy6FDvYjBa/KI83r8MQ+zhYymVmCbqFoVawJ4CKxZ6QIZmi6hRfHj3DCSagwQ1J2bStWvRQJRTEjmekkksQID1GfdDUNESeyl07OzeCRVnwYREK/UMGJ+nsiRVzKEfd0kiM1kPPeWPzP6yYqOO2lNIwTRUI8XRQkDKoIjruDPhUEKzbSBGFB9V8hHiBdnNINm7oEe/7kRdKqVs4q9nW12KjN2siDA3AIysAGNdAAl6AJWgCDR/AM3sGH8WS8Gp/G1zSaM2Yz++APjO8faK+l5Q==</latexit><latexit sha1_base64="9LEm29yj4UjovgL3uLk/JI0n8kM=">AAACLXicbVDLSsNAFJ3UV42vqks3g61QEUrSTXVRKIiguwrWFpoaJpNJO3QmCTMTpYR8kRt/RReCD9z6G04fC209MHA451zu3OPFjEplWW9Gbml5ZXUtv25ubG5t7xR2925llAhMWjhikeh4SBJGQ9JSVDHSiQVB3GOk7Q3Px377nghJo/BGjWLS46gf0oBipLTkFi5KZgzr0PEDgXDqyIS7Ka3b2R2HDy6FDvYjBa/KI83r8MQ+zhYymVmCbqFoVawJ4CKxZ6QIZmi6hRfHj3DCSagwQ1J2bStWvRQJRTEjmekkksQID1GfdDUNESeyl07OzeCRVnwYREK/UMGJ+nsiRVzKEfd0kiM1kPPeWPzP6yYqOO2lNIwTRUI8XRQkDKoIjruDPhUEKzbSBGFB9V8hHiBdnNINm7oEe/7kRdKqVs4q9nW12KjN2siDA3AIysAGNdAAl6AJWgCDR/AM3sGH8WS8Gp/G1zSaM2Yz++APjO8faK+l5Q==</latexit><latexit sha1_base64="9LEm29yj4UjovgL3uLk/JI0n8kM=">AAACLXicbVDLSsNAFJ3UV42vqks3g61QEUrSTXVRKIiguwrWFpoaJpNJO3QmCTMTpYR8kRt/RReCD9z6G04fC209MHA451zu3OPFjEplWW9Gbml5ZXUtv25ubG5t7xR2925llAhMWjhikeh4SBJGQ9JSVDHSiQVB3GOk7Q3Px377nghJo/BGjWLS46gf0oBipLTkFi5KZgzr0PEDgXDqyIS7Ka3b2R2HDy6FDvYjBa/KI83r8MQ+zhYymVmCbqFoVawJ4CKxZ6QIZmi6hRfHj3DCSagwQ1J2bStWvRQJRTEjmekkksQID1GfdDUNESeyl07OzeCRVnwYREK/UMGJ+nsiRVzKEfd0kiM1kPPeWPzP6yYqOO2lNIwTRUI8XRQkDKoIjruDPhUEKzbSBGFB9V8hHiBdnNINm7oEe/7kRdKqVs4q9nW12KjN2siDA3AIysAGNdAAl6AJWgCDR/AM3sGH8WS8Gp/G1zSaM2Yz++APjO8faK+l5Q==</latexit><latexit sha1_base64="9LEm29yj4UjovgL3uLk/JI0n8kM=">AAACLXicbVDLSsNAFJ3UV42vqks3g61QEUrSTXVRKIiguwrWFpoaJpNJO3QmCTMTpYR8kRt/RReCD9z6G04fC209MHA451zu3OPFjEplWW9Gbml5ZXUtv25ubG5t7xR2925llAhMWjhikeh4SBJGQ9JSVDHSiQVB3GOk7Q3Px377nghJo/BGjWLS46gf0oBipLTkFi5KZgzr0PEDgXDqyIS7Ka3b2R2HDy6FDvYjBa/KI83r8MQ+zhYymVmCbqFoVawJ4CKxZ6QIZmi6hRfHj3DCSagwQ1J2bStWvRQJRTEjmekkksQID1GfdDUNESeyl07OzeCRVnwYREK/UMGJ+nsiRVzKEfd0kiM1kPPeWPzP6yYqOO2lNIwTRUI8XRQkDKoIjruDPhUEKzbSBGFB9V8hHiBdnNINm7oEe/7kRdKqVs4q9nW12KjN2siDA3AIysAGNdAAl6AJWgCDR/AM3sGH8WS8Gp/G1zSaM2Yz++APjO8faK+l5Q==</latexit>
p =
Pmi=1 I(yi = +1)
m<latexit sha1_base64="2uC9p4I2Fqb43ljzO9gLpsbw4qI=">AAACE3icbVBNS8NAEN34WeNX1aOXxVaoFErSS/VQKHjRWwVrC20Nm82mXbqbhN2NEEJ+hBf/ihcPKl69ePPfuG1z0NYHA4/3ZpiZ50aMSmVZ38bK6tr6xmZhy9ze2d3bLx4c3skwFph0cMhC0XORJIwGpKOoYqQXCYK4y0jXnVxO/e4DEZKGwa1KIjLkaBRQn2KktOQUq2Uzgk048HyBcDqQMXdS2rSzew6vK4lDtVe1z7KUZ2XoFEtWzZoBLhM7JyWQo+0UvwZeiGNOAoUZkrJvW5EapkgoihnJzEEsSYTwBI1IX9MAcSKH6eypDJ5qxYN+KHQFCs7U3xMp4lIm3NWdHKmxXPSm4n9eP1b++TClQRQrEuD5Ij9mUIVwmhD0qCBYsUQThAXVt0I8RjoepXM0dQj24svLpFOvXdTsm3qp1cjTKIBjcAIqwAYN0AJXoA06AINH8AxewZvxZLwY78bHvHXFyGeOwB8Ynz/Vgpu2</latexit><latexit sha1_base64="2uC9p4I2Fqb43ljzO9gLpsbw4qI=">AAACE3icbVBNS8NAEN34WeNX1aOXxVaoFErSS/VQKHjRWwVrC20Nm82mXbqbhN2NEEJ+hBf/ihcPKl69ePPfuG1z0NYHA4/3ZpiZ50aMSmVZ38bK6tr6xmZhy9ze2d3bLx4c3skwFph0cMhC0XORJIwGpKOoYqQXCYK4y0jXnVxO/e4DEZKGwa1KIjLkaBRQn2KktOQUq2Uzgk048HyBcDqQMXdS2rSzew6vK4lDtVe1z7KUZ2XoFEtWzZoBLhM7JyWQo+0UvwZeiGNOAoUZkrJvW5EapkgoihnJzEEsSYTwBI1IX9MAcSKH6eypDJ5qxYN+KHQFCs7U3xMp4lIm3NWdHKmxXPSm4n9eP1b++TClQRQrEuD5Ij9mUIVwmhD0qCBYsUQThAXVt0I8RjoepXM0dQj24svLpFOvXdTsm3qp1cjTKIBjcAIqwAYN0AJXoA06AINH8AxewZvxZLwY78bHvHXFyGeOwB8Ynz/Vgpu2</latexit><latexit sha1_base64="2uC9p4I2Fqb43ljzO9gLpsbw4qI=">AAACE3icbVBNS8NAEN34WeNX1aOXxVaoFErSS/VQKHjRWwVrC20Nm82mXbqbhN2NEEJ+hBf/ihcPKl69ePPfuG1z0NYHA4/3ZpiZ50aMSmVZ38bK6tr6xmZhy9ze2d3bLx4c3skwFph0cMhC0XORJIwGpKOoYqQXCYK4y0jXnVxO/e4DEZKGwa1KIjLkaBRQn2KktOQUq2Uzgk048HyBcDqQMXdS2rSzew6vK4lDtVe1z7KUZ2XoFEtWzZoBLhM7JyWQo+0UvwZeiGNOAoUZkrJvW5EapkgoihnJzEEsSYTwBI1IX9MAcSKH6eypDJ5qxYN+KHQFCs7U3xMp4lIm3NWdHKmxXPSm4n9eP1b++TClQRQrEuD5Ij9mUIVwmhD0qCBYsUQThAXVt0I8RjoepXM0dQj24svLpFOvXdTsm3qp1cjTKIBjcAIqwAYN0AJXoA06AINH8AxewZvxZLwY78bHvHXFyGeOwB8Ynz/Vgpu2</latexit><latexit sha1_base64="2uC9p4I2Fqb43ljzO9gLpsbw4qI=">AAACE3icbVBNS8NAEN34WeNX1aOXxVaoFErSS/VQKHjRWwVrC20Nm82mXbqbhN2NEEJ+hBf/ihcPKl69ePPfuG1z0NYHA4/3ZpiZ50aMSmVZ38bK6tr6xmZhy9ze2d3bLx4c3skwFph0cMhC0XORJIwGpKOoYqQXCYK4y0jXnVxO/e4DEZKGwa1KIjLkaBRQn2KktOQUq2Uzgk048HyBcDqQMXdS2rSzew6vK4lDtVe1z7KUZ2XoFEtWzZoBLhM7JyWQo+0UvwZeiGNOAoUZkrJvW5EapkgoihnJzEEsSYTwBI1IX9MAcSKH6eypDJ5qxYN+KHQFCs7U3xMp4lIm3NWdHKmxXPSm4n9eP1b++TClQRQrEuD5Ij9mUIVwmhD0qCBYsUQThAXVt0I8RjoepXM0dQj24svLpFOvXdTsm3qp1cjTKIBjcAIqwAYN0AJXoA06AINH8AxewZvxZLwY78bHvHXFyGeOwB8Ynz/Vgpu2</latexit>
_
=
← pos pts=
0=←
![Page 12: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/12.jpg)
Adaboost Algorithm
12
2b) Compute weighted errors:
o Still gives an error rate in [0, 1]
o Mistakes on highly weighted examples hurt more
o Mistakes on lowly weighted examples don’t hurt as much
errk =
Pmi=1 wi · I(yi 6= hk(xi))Pm
i=1 wi<latexit sha1_base64="//GA8Is51nY3KAR0Aj27RMeVcYM=">AAACSXicbZBLbxMxFIU9KY8yvAIs2VyRIqWbaKabtotKlbqBXZEIrZQJlsdzp7Fiewb7DjQaze/rhhU7fgQbFoBY4aRZQMORLB2dc69sf3mtlack+Rr1tm7dvnN3+158/8HDR4/7T56+81XjJI5lpSt3nguPWlkckyKN57VDYXKNZ/n8ZNmffUTnVWXf0qLGqREXVpVKCgoR74udOCO8JGdadK7jcziCrCidkG3mG8NbdZR27w184iqTRUXwerjgCjKLH2DG58M2y0u47Lja3YVuY6WLd4D3B8koWQk2Tbo2A7bWKe9/yYpKNgYtSS28n6RJTdNWOFJSYxdnjcdayLm4wEmwVhj003aFooOXISmgrFw4lmCV/r3RCuP9wuRh0gia+ZvdMvxfN2moPJi2ytYNoZXXF5WNBqpgyRUK5VCSXgQjpFPhrSBnInCkQD8OENKbX940473R4Sh9szc43l/T2GbP2Qs2ZCnbZ8fsFTtlYybZFfvGfrCf0efoe/Qr+n092ovWO8/YP+pt/QGsLrGf</latexit><latexit sha1_base64="//GA8Is51nY3KAR0Aj27RMeVcYM=">AAACSXicbZBLbxMxFIU9KY8yvAIs2VyRIqWbaKabtotKlbqBXZEIrZQJlsdzp7Fiewb7DjQaze/rhhU7fgQbFoBY4aRZQMORLB2dc69sf3mtlack+Rr1tm7dvnN3+158/8HDR4/7T56+81XjJI5lpSt3nguPWlkckyKN57VDYXKNZ/n8ZNmffUTnVWXf0qLGqREXVpVKCgoR74udOCO8JGdadK7jcziCrCidkG3mG8NbdZR27w184iqTRUXwerjgCjKLH2DG58M2y0u47Lja3YVuY6WLd4D3B8koWQk2Tbo2A7bWKe9/yYpKNgYtSS28n6RJTdNWOFJSYxdnjcdayLm4wEmwVhj003aFooOXISmgrFw4lmCV/r3RCuP9wuRh0gia+ZvdMvxfN2moPJi2ytYNoZXXF5WNBqpgyRUK5VCSXgQjpFPhrSBnInCkQD8OENKbX940473R4Sh9szc43l/T2GbP2Qs2ZCnbZ8fsFTtlYybZFfvGfrCf0efoe/Qr+n092ovWO8/YP+pt/QGsLrGf</latexit><latexit sha1_base64="//GA8Is51nY3KAR0Aj27RMeVcYM=">AAACSXicbZBLbxMxFIU9KY8yvAIs2VyRIqWbaKabtotKlbqBXZEIrZQJlsdzp7Fiewb7DjQaze/rhhU7fgQbFoBY4aRZQMORLB2dc69sf3mtlack+Rr1tm7dvnN3+158/8HDR4/7T56+81XjJI5lpSt3nguPWlkckyKN57VDYXKNZ/n8ZNmffUTnVWXf0qLGqREXVpVKCgoR74udOCO8JGdadK7jcziCrCidkG3mG8NbdZR27w184iqTRUXwerjgCjKLH2DG58M2y0u47Lja3YVuY6WLd4D3B8koWQk2Tbo2A7bWKe9/yYpKNgYtSS28n6RJTdNWOFJSYxdnjcdayLm4wEmwVhj003aFooOXISmgrFw4lmCV/r3RCuP9wuRh0gia+ZvdMvxfN2moPJi2ytYNoZXXF5WNBqpgyRUK5VCSXgQjpFPhrSBnInCkQD8OENKbX940473R4Sh9szc43l/T2GbP2Qs2ZCnbZ8fsFTtlYybZFfvGfrCf0efoe/Qr+n092ovWO8/YP+pt/QGsLrGf</latexit><latexit sha1_base64="//GA8Is51nY3KAR0Aj27RMeVcYM=">AAACSXicbZBLbxMxFIU9KY8yvAIs2VyRIqWbaKabtotKlbqBXZEIrZQJlsdzp7Fiewb7DjQaze/rhhU7fgQbFoBY4aRZQMORLB2dc69sf3mtlack+Rr1tm7dvnN3+158/8HDR4/7T56+81XjJI5lpSt3nguPWlkckyKN57VDYXKNZ/n8ZNmffUTnVWXf0qLGqREXVpVKCgoR74udOCO8JGdadK7jcziCrCidkG3mG8NbdZR27w184iqTRUXwerjgCjKLH2DG58M2y0u47Lja3YVuY6WLd4D3B8koWQk2Tbo2A7bWKe9/yYpKNgYtSS28n6RJTdNWOFJSYxdnjcdayLm4wEmwVhj003aFooOXISmgrFw4lmCV/r3RCuP9wuRh0gia+ZvdMvxfN2moPJi2ytYNoZXXF5WNBqpgyRUK5VCSXgQjpFPhrSBnInCkQD8OENKbX940473R4Sh9szc43l/T2GbP2Qs2ZCnbZ8fsFTtlYybZFfvGfrCf0efoe/Qr+n092ovWO8/YP+pt/QGsLrGf</latexit>
# gotis # wrong
=
![Page 13: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/13.jpg)
Adaboost Algorithm
13
2c) Compute vote weight for the current weak learner
o Models with small weighted error get a larger vote
o Models with large weighted error get less vote (or, sometimes a negative vote!)
↵k = 12 log((1� errk)/errk)
<latexit sha1_base64="SfQna/o3NSXYOMTJbcHkpR5TUR0=">AAACKnicbVC7TsNAEDzzxrwClDQnAlIoCHaaQIGEoKEMEgGkOLLOl3VyyvmhuzUisvw/NPwKBRQQ0fIhXEIKCIy00tzMrm53glQKjY4ztGZm5+YXFpeW7ZXVtfWN0ubWjU4yxaHJE5mou4BpkCKGJgqUcJcqYFEg4TboX4z823tQWiTxNQ5SaEesG4tQcIZG8kvne7bHZNpjfp+eUi9UjOdukdcKTybdSsU99BAeUEU5KFX4/YOjqbe9R/1S2ak6Y9C/xJ2QMpmg4ZdevE7Cswhi5JJp3XKdFNs5Uyi4hML2Mg0p433WhZahMYtAt/PxrQXdN0qHhokyFSMdqz8nchZpPYgC0xkx7OlpbyT+57UyDI/buYjTDCHm3x+FmaSY0FFwtCMUcJQDQxhXwuxKeY+ZvNDEa5sQ3OmT/5JmrXpSda9q5bP6JI0lskN2SYW4pE7OyCVpkCbh5JE8kzfybj1Zr9bQ+vhunbEmM9vkF6zPL7a4plk=</latexit><latexit sha1_base64="SfQna/o3NSXYOMTJbcHkpR5TUR0=">AAACKnicbVC7TsNAEDzzxrwClDQnAlIoCHaaQIGEoKEMEgGkOLLOl3VyyvmhuzUisvw/NPwKBRQQ0fIhXEIKCIy00tzMrm53glQKjY4ztGZm5+YXFpeW7ZXVtfWN0ubWjU4yxaHJE5mou4BpkCKGJgqUcJcqYFEg4TboX4z823tQWiTxNQ5SaEesG4tQcIZG8kvne7bHZNpjfp+eUi9UjOdukdcKTybdSsU99BAeUEU5KFX4/YOjqbe9R/1S2ak6Y9C/xJ2QMpmg4ZdevE7Cswhi5JJp3XKdFNs5Uyi4hML2Mg0p433WhZahMYtAt/PxrQXdN0qHhokyFSMdqz8nchZpPYgC0xkx7OlpbyT+57UyDI/buYjTDCHm3x+FmaSY0FFwtCMUcJQDQxhXwuxKeY+ZvNDEa5sQ3OmT/5JmrXpSda9q5bP6JI0lskN2SYW4pE7OyCVpkCbh5JE8kzfybj1Zr9bQ+vhunbEmM9vkF6zPL7a4plk=</latexit><latexit sha1_base64="SfQna/o3NSXYOMTJbcHkpR5TUR0=">AAACKnicbVC7TsNAEDzzxrwClDQnAlIoCHaaQIGEoKEMEgGkOLLOl3VyyvmhuzUisvw/NPwKBRQQ0fIhXEIKCIy00tzMrm53glQKjY4ztGZm5+YXFpeW7ZXVtfWN0ubWjU4yxaHJE5mou4BpkCKGJgqUcJcqYFEg4TboX4z823tQWiTxNQ5SaEesG4tQcIZG8kvne7bHZNpjfp+eUi9UjOdukdcKTybdSsU99BAeUEU5KFX4/YOjqbe9R/1S2ak6Y9C/xJ2QMpmg4ZdevE7Cswhi5JJp3XKdFNs5Uyi4hML2Mg0p433WhZahMYtAt/PxrQXdN0qHhokyFSMdqz8nchZpPYgC0xkx7OlpbyT+57UyDI/buYjTDCHm3x+FmaSY0FFwtCMUcJQDQxhXwuxKeY+ZvNDEa5sQ3OmT/5JmrXpSda9q5bP6JI0lskN2SYW4pE7OyCVpkCbh5JE8kzfybj1Zr9bQ+vhunbEmM9vkF6zPL7a4plk=</latexit><latexit sha1_base64="SfQna/o3NSXYOMTJbcHkpR5TUR0=">AAACKnicbVC7TsNAEDzzxrwClDQnAlIoCHaaQIGEoKEMEgGkOLLOl3VyyvmhuzUisvw/NPwKBRQQ0fIhXEIKCIy00tzMrm53glQKjY4ztGZm5+YXFpeW7ZXVtfWN0ubWjU4yxaHJE5mou4BpkCKGJgqUcJcqYFEg4TboX4z823tQWiTxNQ5SaEesG4tQcIZG8kvne7bHZNpjfp+eUi9UjOdukdcKTybdSsU99BAeUEU5KFX4/YOjqbe9R/1S2ak6Y9C/xJ2QMpmg4ZdevE7Cswhi5JJp3XKdFNs5Uyi4hML2Mg0p433WhZahMYtAt/PxrQXdN0qHhokyFSMdqz8nchZpPYgC0xkx7OlpbyT+57UyDI/buYjTDCHm3x+FmaSY0FFwtCMUcJQDQxhXwuxKeY+ZvNDEa5sQ3OmT/5JmrXpSda9q5bP6JI0lskN2SYW4pE7OyCVpkCbh5JE8kzfybj1Zr9bQ+vhunbEmM9vkF6zPL7a4plk=</latexit>
= -
×
×
× ja= - 0,5X
×
X 1
![Page 14: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/14.jpg)
Adaboost Algorithm
14
2d) Update training example weights
o If example is misclassified then weight goes up
o If example is classified correctly then weight goes down
o How big of a jump up/down depends on how much vote the current learner has
o is just a normalizing constant to make it a probability distribution
o Don’t really have to compute , just scale weights so they sum to 1
wi 1Zk
· wi exp [�↵kyihk(xi)]<latexit sha1_base64="+Md+EfCAFodoBKQgI79wGNjvoIc=">AAACQXicbVA9b9QwGHbKVxu+rjCyWFyRysAp6VK6VWJhLBJHS89R9Mb35mLFiSP7De0pym9j4Rd06w9gYWgRaxd81xtoyyNZevR8yPaTNVo5iqLzYO3e/QcPH61vhI+fPH32fLD54oszrZU4lkYbe5SBQ61qHJMijUeNRagyjYdZ+WHhH35D65SpP9O8waSCWa1yJYG8lA6+boUnqeJCY05grTnhIrcgu7jvjtOy50JODXEfEXjaLFOTdwJ0U0Ba8rlvFmm53Yks56d9qt4Kq2YFJTzc4ulgGI2iJfhdEq/IkK1wkA7OxNTItsKapAbnJnHUUNKBJSU19qFoHTYgS5jhxNMaKnRJt5yg52+8MuW5sf7UxJfqv40OKufmVeaTFVDhbnsL8X/epKX8fdKpumkJa3l9Ud5qToYv9uRTZVGSnnsC0ir/Vi4L8BOSXz30I8S3v3yXjHdGe6P4085wf3e1xjp7xV6zbRazXbbPPrIDNmaSfWc/2QW7DH4Ev4LfwZ/r6Fqw6rxkNxBc/QWQoK+2</latexit><latexit sha1_base64="+Md+EfCAFodoBKQgI79wGNjvoIc=">AAACQXicbVA9b9QwGHbKVxu+rjCyWFyRysAp6VK6VWJhLBJHS89R9Mb35mLFiSP7De0pym9j4Rd06w9gYWgRaxd81xtoyyNZevR8yPaTNVo5iqLzYO3e/QcPH61vhI+fPH32fLD54oszrZU4lkYbe5SBQ61qHJMijUeNRagyjYdZ+WHhH35D65SpP9O8waSCWa1yJYG8lA6+boUnqeJCY05grTnhIrcgu7jvjtOy50JODXEfEXjaLFOTdwJ0U0Ba8rlvFmm53Yks56d9qt4Kq2YFJTzc4ulgGI2iJfhdEq/IkK1wkA7OxNTItsKapAbnJnHUUNKBJSU19qFoHTYgS5jhxNMaKnRJt5yg52+8MuW5sf7UxJfqv40OKufmVeaTFVDhbnsL8X/epKX8fdKpumkJa3l9Ud5qToYv9uRTZVGSnnsC0ir/Vi4L8BOSXz30I8S3v3yXjHdGe6P4085wf3e1xjp7xV6zbRazXbbPPrIDNmaSfWc/2QW7DH4Ev4LfwZ/r6Fqw6rxkNxBc/QWQoK+2</latexit><latexit sha1_base64="+Md+EfCAFodoBKQgI79wGNjvoIc=">AAACQXicbVA9b9QwGHbKVxu+rjCyWFyRysAp6VK6VWJhLBJHS89R9Mb35mLFiSP7De0pym9j4Rd06w9gYWgRaxd81xtoyyNZevR8yPaTNVo5iqLzYO3e/QcPH61vhI+fPH32fLD54oszrZU4lkYbe5SBQ61qHJMijUeNRagyjYdZ+WHhH35D65SpP9O8waSCWa1yJYG8lA6+boUnqeJCY05grTnhIrcgu7jvjtOy50JODXEfEXjaLFOTdwJ0U0Ba8rlvFmm53Yks56d9qt4Kq2YFJTzc4ulgGI2iJfhdEq/IkK1wkA7OxNTItsKapAbnJnHUUNKBJSU19qFoHTYgS5jhxNMaKnRJt5yg52+8MuW5sf7UxJfqv40OKufmVeaTFVDhbnsL8X/epKX8fdKpumkJa3l9Ud5qToYv9uRTZVGSnnsC0ir/Vi4L8BOSXz30I8S3v3yXjHdGe6P4085wf3e1xjp7xV6zbRazXbbPPrIDNmaSfWc/2QW7DH4Ev4LfwZ/r6Fqw6rxkNxBc/QWQoK+2</latexit><latexit sha1_base64="+Md+EfCAFodoBKQgI79wGNjvoIc=">AAACQXicbVA9b9QwGHbKVxu+rjCyWFyRysAp6VK6VWJhLBJHS89R9Mb35mLFiSP7De0pym9j4Rd06w9gYWgRaxd81xtoyyNZevR8yPaTNVo5iqLzYO3e/QcPH61vhI+fPH32fLD54oszrZU4lkYbe5SBQ61qHJMijUeNRagyjYdZ+WHhH35D65SpP9O8waSCWa1yJYG8lA6+boUnqeJCY05grTnhIrcgu7jvjtOy50JODXEfEXjaLFOTdwJ0U0Ba8rlvFmm53Yks56d9qt4Kq2YFJTzc4ulgGI2iJfhdEq/IkK1wkA7OxNTItsKapAbnJnHUUNKBJSU19qFoHTYgS5jhxNMaKnRJt5yg52+8MuW5sf7UxJfqv40OKufmVeaTFVDhbnsL8X/epKX8fdKpumkJa3l9Ud5qToYv9uRTZVGSnnsC0ir/Vi4L8BOSXz30I8S3v3yXjHdGe6P4085wf3e1xjp7xV6zbRazXbbPPrIDNmaSfWc/2QW7DH4Ev4LfwZ/r6Fqw6rxkNxBc/QWQoK+2</latexit>
Zk<latexit sha1_base64="Bj3g7RvCGumNu7fhJbsHkjatqAI=">AAAB7nicbVBNT8JAEJ31E+sX6tHLRjDxRFou6I3Ei0dMrBChIdtlCxu227q7NSENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMwLU8G1cd1vtLa+sbm1Xdpxdvf2Dw7LR8f3OskUZT5NRKI6IdFMcMl8w41gnVQxEoeCtcPx9cxvPzGleSLvzCRlQUyGkkecEmOlTtV56I+dKu6XK27NnQOvEq8gFSjQ6pe/eoOEZjGThgqidddzUxPkRBlOBZs6vUyzlNAxGbKupZLETAf5/N4pPrfKAEeJsiUNnqu/J3ISaz2JQ9sZEzPSy95M/M/rZia6DHIu08wwSReLokxgk+DZ83jAFaNGTCwhVHF7K6Yjogg1NiLHhuAtv7xK/Hrtqubd1ivNRpFGCU7hDC7AgwY04QZa4AMFAc/wCm/oEb2gd/SxaF1DxcwJ/AH6/AEYo44x</latexit><latexit sha1_base64="Bj3g7RvCGumNu7fhJbsHkjatqAI=">AAAB7nicbVBNT8JAEJ31E+sX6tHLRjDxRFou6I3Ei0dMrBChIdtlCxu227q7NSENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMwLU8G1cd1vtLa+sbm1Xdpxdvf2Dw7LR8f3OskUZT5NRKI6IdFMcMl8w41gnVQxEoeCtcPx9cxvPzGleSLvzCRlQUyGkkecEmOlTtV56I+dKu6XK27NnQOvEq8gFSjQ6pe/eoOEZjGThgqidddzUxPkRBlOBZs6vUyzlNAxGbKupZLETAf5/N4pPrfKAEeJsiUNnqu/J3ISaz2JQ9sZEzPSy95M/M/rZia6DHIu08wwSReLokxgk+DZ83jAFaNGTCwhVHF7K6Yjogg1NiLHhuAtv7xK/Hrtqubd1ivNRpFGCU7hDC7AgwY04QZa4AMFAc/wCm/oEb2gd/SxaF1DxcwJ/AH6/AEYo44x</latexit><latexit sha1_base64="Bj3g7RvCGumNu7fhJbsHkjatqAI=">AAAB7nicbVBNT8JAEJ31E+sX6tHLRjDxRFou6I3Ei0dMrBChIdtlCxu227q7NSENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMwLU8G1cd1vtLa+sbm1Xdpxdvf2Dw7LR8f3OskUZT5NRKI6IdFMcMl8w41gnVQxEoeCtcPx9cxvPzGleSLvzCRlQUyGkkecEmOlTtV56I+dKu6XK27NnQOvEq8gFSjQ6pe/eoOEZjGThgqidddzUxPkRBlOBZs6vUyzlNAxGbKupZLETAf5/N4pPrfKAEeJsiUNnqu/J3ISaz2JQ9sZEzPSy95M/M/rZia6DHIu08wwSReLokxgk+DZ83jAFaNGTCwhVHF7K6Yjogg1NiLHhuAtv7xK/Hrtqubd1ivNRpFGCU7hDC7AgwY04QZa4AMFAc/wCm/oEb2gd/SxaF1DxcwJ/AH6/AEYo44x</latexit><latexit sha1_base64="Bj3g7RvCGumNu7fhJbsHkjatqAI=">AAAB7nicbVBNT8JAEJ31E+sX6tHLRjDxRFou6I3Ei0dMrBChIdtlCxu227q7NSENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMwLU8G1cd1vtLa+sbm1Xdpxdvf2Dw7LR8f3OskUZT5NRKI6IdFMcMl8w41gnVQxEoeCtcPx9cxvPzGleSLvzCRlQUyGkkecEmOlTtV56I+dKu6XK27NnQOvEq8gFSjQ6pe/eoOEZjGThgqidddzUxPkRBlOBZs6vUyzlNAxGbKupZLETAf5/N4pPrfKAEeJsiUNnqu/J3ISaz2JQ9sZEzPSy95M/M/rZia6DHIu08wwSReLokxgk+DZ83jAFaNGTCwhVHF7K6Yjogg1NiLHhuAtv7xK/Hrtqubd1ivNRpFGCU7hDC7AgwY04QZa4AMFAc/wCm/oEb2gd/SxaF1DxcwJ/AH6/AEYo44x</latexit>
Zk<latexit sha1_base64="Bj3g7RvCGumNu7fhJbsHkjatqAI=">AAAB7nicbVBNT8JAEJ31E+sX6tHLRjDxRFou6I3Ei0dMrBChIdtlCxu227q7NSENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMwLU8G1cd1vtLa+sbm1Xdpxdvf2Dw7LR8f3OskUZT5NRKI6IdFMcMl8w41gnVQxEoeCtcPx9cxvPzGleSLvzCRlQUyGkkecEmOlTtV56I+dKu6XK27NnQOvEq8gFSjQ6pe/eoOEZjGThgqidddzUxPkRBlOBZs6vUyzlNAxGbKupZLETAf5/N4pPrfKAEeJsiUNnqu/J3ISaz2JQ9sZEzPSy95M/M/rZia6DHIu08wwSReLokxgk+DZ83jAFaNGTCwhVHF7K6Yjogg1NiLHhuAtv7xK/Hrtqubd1ivNRpFGCU7hDC7AgwY04QZa4AMFAc/wCm/oEb2gd/SxaF1DxcwJ/AH6/AEYo44x</latexit><latexit sha1_base64="Bj3g7RvCGumNu7fhJbsHkjatqAI=">AAAB7nicbVBNT8JAEJ31E+sX6tHLRjDxRFou6I3Ei0dMrBChIdtlCxu227q7NSENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMwLU8G1cd1vtLa+sbm1Xdpxdvf2Dw7LR8f3OskUZT5NRKI6IdFMcMl8w41gnVQxEoeCtcPx9cxvPzGleSLvzCRlQUyGkkecEmOlTtV56I+dKu6XK27NnQOvEq8gFSjQ6pe/eoOEZjGThgqidddzUxPkRBlOBZs6vUyzlNAxGbKupZLETAf5/N4pPrfKAEeJsiUNnqu/J3ISaz2JQ9sZEzPSy95M/M/rZia6DHIu08wwSReLokxgk+DZ83jAFaNGTCwhVHF7K6Yjogg1NiLHhuAtv7xK/Hrtqubd1ivNRpFGCU7hDC7AgwY04QZa4AMFAc/wCm/oEb2gd/SxaF1DxcwJ/AH6/AEYo44x</latexit><latexit sha1_base64="Bj3g7RvCGumNu7fhJbsHkjatqAI=">AAAB7nicbVBNT8JAEJ31E+sX6tHLRjDxRFou6I3Ei0dMrBChIdtlCxu227q7NSENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMwLU8G1cd1vtLa+sbm1Xdpxdvf2Dw7LR8f3OskUZT5NRKI6IdFMcMl8w41gnVQxEoeCtcPx9cxvPzGleSLvzCRlQUyGkkecEmOlTtV56I+dKu6XK27NnQOvEq8gFSjQ6pe/eoOEZjGThgqidddzUxPkRBlOBZs6vUyzlNAxGbKupZLETAf5/N4pPrfKAEeJsiUNnqu/J3ISaz2JQ9sZEzPSy95M/M/rZia6DHIu08wwSReLokxgk+DZ83jAFaNGTCwhVHF7K6Yjogg1NiLHhuAtv7xK/Hrtqubd1ivNRpFGCU7hDC7AgwY04QZa4AMFAc/wCm/oEb2gd/SxaF1DxcwJ/AH6/AEYo44x</latexit><latexit sha1_base64="Bj3g7RvCGumNu7fhJbsHkjatqAI=">AAAB7nicbVBNT8JAEJ31E+sX6tHLRjDxRFou6I3Ei0dMrBChIdtlCxu227q7NSENf8KLBzVe/T3e/Dcu0IOCL5nk5b2ZzMwLU8G1cd1vtLa+sbm1Xdpxdvf2Dw7LR8f3OskUZT5NRKI6IdFMcMl8w41gnVQxEoeCtcPx9cxvPzGleSLvzCRlQUyGkkecEmOlTtV56I+dKu6XK27NnQOvEq8gFSjQ6pe/eoOEZjGThgqidddzUxPkRBlOBZs6vUyzlNAxGbKupZLETAf5/N4pPrfKAEeJsiUNnqu/J3ISaz2JQ9sZEzPSy95M/M/rZia6DHIu08wwSReLokxgk+DZ83jAFaNGTCwhVHF7K6Yjogg1NiLHhuAtv7xK/Hrtqubd1ivNRpFGCU7hDC7AgwY04QZa4AMFAc/wCm/oEb2gd/SxaF1DxcwJ/AH6/AEYo44x</latexit>
PREPlot .
L
- =a
- =
uol.net#Yane1strung #
![Page 15: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/15.jpg)
Adaboost Algorithm
15
3) Algorithm produces the final boosted classifier
o Sum up weighted votes from each model
o Apply sign function so that it produces predictions in {+1,�1}<latexit sha1_base64="821zbZXswSZ6cqdHmYVyZcSBD4Q=">AAAB9HicbVBNT8JAEJ3iF9Yv1KOXjWBiopKWC3oj8eIREysktJLtssCG7bbZ3WpIw//w4kGNV3+MN/+NC/Sg4EsmeXlvJjPzwoQzpR3n2yqsrK6tbxQ37a3tnd290v7BvYpTSahHYh7LdogV5UxQTzPNaTuRFEchp61wdD31W49UKhaLOz1OaBDhgWB9RrA20kPF9rMz9/zC9Sd2BXVLZafqzICWiZuTMuRodktffi8maUSFJhwr1XGdRAcZlpoRTie2nyqaYDLCA9oxVOCIqiCbXT1BJ0bpoX4sTQmNZurviQxHSo2j0HRGWA/VojcV//M6qe5fBhkTSaqpIPNF/ZQjHaNpBKjHJCWajw3BRDJzKyJDLDHRJijbhOAuvrxMvFr1qure1sqNep5GEY7gGE7BhTo04Aaa4AEBCc/wCm/Wk/VivVsf89aClc8cwh9Ynz80ao/f</latexit><latexit sha1_base64="821zbZXswSZ6cqdHmYVyZcSBD4Q=">AAAB9HicbVBNT8JAEJ3iF9Yv1KOXjWBiopKWC3oj8eIREysktJLtssCG7bbZ3WpIw//w4kGNV3+MN/+NC/Sg4EsmeXlvJjPzwoQzpR3n2yqsrK6tbxQ37a3tnd290v7BvYpTSahHYh7LdogV5UxQTzPNaTuRFEchp61wdD31W49UKhaLOz1OaBDhgWB9RrA20kPF9rMz9/zC9Sd2BXVLZafqzICWiZuTMuRodktffi8maUSFJhwr1XGdRAcZlpoRTie2nyqaYDLCA9oxVOCIqiCbXT1BJ0bpoX4sTQmNZurviQxHSo2j0HRGWA/VojcV//M6qe5fBhkTSaqpIPNF/ZQjHaNpBKjHJCWajw3BRDJzKyJDLDHRJijbhOAuvrxMvFr1qure1sqNep5GEY7gGE7BhTo04Aaa4AEBCc/wCm/Wk/VivVsf89aClc8cwh9Ynz80ao/f</latexit><latexit sha1_base64="821zbZXswSZ6cqdHmYVyZcSBD4Q=">AAAB9HicbVBNT8JAEJ3iF9Yv1KOXjWBiopKWC3oj8eIREysktJLtssCG7bbZ3WpIw//w4kGNV3+MN/+NC/Sg4EsmeXlvJjPzwoQzpR3n2yqsrK6tbxQ37a3tnd290v7BvYpTSahHYh7LdogV5UxQTzPNaTuRFEchp61wdD31W49UKhaLOz1OaBDhgWB9RrA20kPF9rMz9/zC9Sd2BXVLZafqzICWiZuTMuRodktffi8maUSFJhwr1XGdRAcZlpoRTie2nyqaYDLCA9oxVOCIqiCbXT1BJ0bpoX4sTQmNZurviQxHSo2j0HRGWA/VojcV//M6qe5fBhkTSaqpIPNF/ZQjHaNpBKjHJCWajw3BRDJzKyJDLDHRJijbhOAuvrxMvFr1qure1sqNep5GEY7gGE7BhTo04Aaa4AEBCc/wCm/Wk/VivVsf89aClc8cwh9Ynz80ao/f</latexit><latexit sha1_base64="821zbZXswSZ6cqdHmYVyZcSBD4Q=">AAAB9HicbVBNT8JAEJ3iF9Yv1KOXjWBiopKWC3oj8eIREysktJLtssCG7bbZ3WpIw//w4kGNV3+MN/+NC/Sg4EsmeXlvJjPzwoQzpR3n2yqsrK6tbxQ37a3tnd290v7BvYpTSahHYh7LdogV5UxQTzPNaTuRFEchp61wdD31W49UKhaLOz1OaBDhgWB9RrA20kPF9rMz9/zC9Sd2BXVLZafqzICWiZuTMuRodktffi8maUSFJhwr1XGdRAcZlpoRTie2nyqaYDLCA9oxVOCIqiCbXT1BJ0bpoX4sTQmNZurviQxHSo2j0HRGWA/VojcV//M6qe5fBhkTSaqpIPNF/ZQjHaNpBKjHJCWajw3BRDJzKyJDLDHRJijbhOAuvrxMvFr1qure1sqNep5GEY7gGE7BhTo04Aaa4AEBCc/wCm/Wk/VivVsf89aClc8cwh9Ynz80ao/f</latexit>
H(x) = signhPK
k=1 ↵khk(x)i
<latexit sha1_base64="kdKSZawqyJ2SMSjAvU8WHIIVoU4=">AAACOXicbVBNa9wwFJTTtE3dptm2x15ENoXksth7SXoIBHoJ5JJAN7uwdoysfV4LS7KRnksW49/VS39Fbz30kkNbeu0fqPYD8rEZEAwzb3h6k1ZSWAyCH97Gk82nz55vvfBfvtp+vdN58/bSlrXhMOClLM0oZRak0DBAgRJGlQGmUgnDtPg094dfwFhR6s84qyBWbKpFJjhDJyWdiz3/dL+J0oxetwf0mEYI12hUY8VUtzSSkOGYRrZWSVMch+3VGY2YrHKWFDRPittkZMQ0x9jfo0mnG/SCBeg6CVekS1Y4Tzrfo0nJawUauWTWjsOgwrhhBgWX0PpRbaFivGBTGDuqmQIbN4vTW/rBKROalcY9jXSh3k00TFk7U6mbVAxz+9Cbi4954xqzo7gRuqoRNF8uympJsaTzHulEGOAoZ44wboT7K+U5M4yja9t3JYQPT14ng37vYy+86HdPDldtbJH3ZJfsk5AckhNySs7JgHDylfwkv8hv75t34/3x/i5HN7xV5h25B+/ff/X7q1E=</latexit><latexit sha1_base64="kdKSZawqyJ2SMSjAvU8WHIIVoU4=">AAACOXicbVBNa9wwFJTTtE3dptm2x15ENoXksth7SXoIBHoJ5JJAN7uwdoysfV4LS7KRnksW49/VS39Fbz30kkNbeu0fqPYD8rEZEAwzb3h6k1ZSWAyCH97Gk82nz55vvfBfvtp+vdN58/bSlrXhMOClLM0oZRak0DBAgRJGlQGmUgnDtPg094dfwFhR6s84qyBWbKpFJjhDJyWdiz3/dL+J0oxetwf0mEYI12hUY8VUtzSSkOGYRrZWSVMch+3VGY2YrHKWFDRPittkZMQ0x9jfo0mnG/SCBeg6CVekS1Y4Tzrfo0nJawUauWTWjsOgwrhhBgWX0PpRbaFivGBTGDuqmQIbN4vTW/rBKROalcY9jXSh3k00TFk7U6mbVAxz+9Cbi4954xqzo7gRuqoRNF8uympJsaTzHulEGOAoZ44wboT7K+U5M4yja9t3JYQPT14ng37vYy+86HdPDldtbJH3ZJfsk5AckhNySs7JgHDylfwkv8hv75t34/3x/i5HN7xV5h25B+/ff/X7q1E=</latexit><latexit sha1_base64="kdKSZawqyJ2SMSjAvU8WHIIVoU4=">AAACOXicbVBNa9wwFJTTtE3dptm2x15ENoXksth7SXoIBHoJ5JJAN7uwdoysfV4LS7KRnksW49/VS39Fbz30kkNbeu0fqPYD8rEZEAwzb3h6k1ZSWAyCH97Gk82nz55vvfBfvtp+vdN58/bSlrXhMOClLM0oZRak0DBAgRJGlQGmUgnDtPg094dfwFhR6s84qyBWbKpFJjhDJyWdiz3/dL+J0oxetwf0mEYI12hUY8VUtzSSkOGYRrZWSVMch+3VGY2YrHKWFDRPittkZMQ0x9jfo0mnG/SCBeg6CVekS1Y4Tzrfo0nJawUauWTWjsOgwrhhBgWX0PpRbaFivGBTGDuqmQIbN4vTW/rBKROalcY9jXSh3k00TFk7U6mbVAxz+9Cbi4954xqzo7gRuqoRNF8uympJsaTzHulEGOAoZ44wboT7K+U5M4yja9t3JYQPT14ng37vYy+86HdPDldtbJH3ZJfsk5AckhNySs7JgHDylfwkv8hv75t34/3x/i5HN7xV5h25B+/ff/X7q1E=</latexit><latexit sha1_base64="kdKSZawqyJ2SMSjAvU8WHIIVoU4=">AAACOXicbVBNa9wwFJTTtE3dptm2x15ENoXksth7SXoIBHoJ5JJAN7uwdoysfV4LS7KRnksW49/VS39Fbz30kkNbeu0fqPYD8rEZEAwzb3h6k1ZSWAyCH97Gk82nz55vvfBfvtp+vdN58/bSlrXhMOClLM0oZRak0DBAgRJGlQGmUgnDtPg094dfwFhR6s84qyBWbKpFJjhDJyWdiz3/dL+J0oxetwf0mEYI12hUY8VUtzSSkOGYRrZWSVMch+3VGY2YrHKWFDRPittkZMQ0x9jfo0mnG/SCBeg6CVekS1Y4Tzrfo0nJawUauWTWjsOgwrhhBgWX0PpRbaFivGBTGDuqmQIbN4vTW/rBKROalcY9jXSh3k00TFk7U6mbVAxz+9Cbi4954xqzo7gRuqoRNF8uympJsaTzHulEGOAoZ44wboT7K+U5M4yja9t3JYQPT14ng37vYy+86HdPDldtbJH3ZJfsk5AckhNySs7JgHDylfwkv8hv75t34/3x/i5HN7xV5h25B+/ff/X7q1E=</latexit>
-
![Page 16: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/16.jpg)
Adaboost Example
16
Example: Suppose you have the following training data
![Page 17: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/17.jpg)
Adaboost Example
17
o First Decision Stump:
0°0
![Page 18: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/18.jpg)
Adaboost Example
18
o First Decision Stump:
err1 = 0.30↵1 = 0.42
<latexit sha1_base64="VfKIA+emCuEy8V1BOGlSXIJ9ZDg=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTFxYVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9EK+d7zHg/pBwKPIj0BlI/gnfm34Eep+6fZ3eK9aC+rBDPwtCUtSYyVavept1E9FnqAmocDaThhk1HUDSQqFbpfcYgbiAobYcVRDgrZbzC6f8F2n9PkgNe5p4jP1f0cBibXjJHadCdDIvq5NxfdqnZwGh91C6iwn1GL+0SBXnFI+jZH3pUFBauwICCPdrlyMwIAgF7bvQghfn/yWtBv17/XwR6PWPCrTqLDP7AvbZyH7xprslLVYmwl2ze7ZI3vybrwH79n7PW9d8ErPNnsB789fmV6pPA==</latexit><latexit sha1_base64="VfKIA+emCuEy8V1BOGlSXIJ9ZDg=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTFxYVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9EK+d7zHg/pBwKPIj0BlI/gnfm34Eep+6fZ3eK9aC+rBDPwtCUtSYyVavept1E9FnqAmocDaThhk1HUDSQqFbpfcYgbiAobYcVRDgrZbzC6f8F2n9PkgNe5p4jP1f0cBibXjJHadCdDIvq5NxfdqnZwGh91C6iwn1GL+0SBXnFI+jZH3pUFBauwICCPdrlyMwIAgF7bvQghfn/yWtBv17/XwR6PWPCrTqLDP7AvbZyH7xprslLVYmwl2ze7ZI3vybrwH79n7PW9d8ErPNnsB789fmV6pPA==</latexit><latexit sha1_base64="VfKIA+emCuEy8V1BOGlSXIJ9ZDg=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTFxYVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9EK+d7zHg/pBwKPIj0BlI/gnfm34Eep+6fZ3eK9aC+rBDPwtCUtSYyVavept1E9FnqAmocDaThhk1HUDSQqFbpfcYgbiAobYcVRDgrZbzC6f8F2n9PkgNe5p4jP1f0cBibXjJHadCdDIvq5NxfdqnZwGh91C6iwn1GL+0SBXnFI+jZH3pUFBauwICCPdrlyMwIAgF7bvQghfn/yWtBv17/XwR6PWPCrTqLDP7AvbZyH7xprslLVYmwl2ze7ZI3vybrwH79n7PW9d8ErPNnsB789fmV6pPA==</latexit><latexit sha1_base64="VfKIA+emCuEy8V1BOGlSXIJ9ZDg=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTFxYVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9EK+d7zHg/pBwKPIj0BlI/gnfm34Eep+6fZ3eK9aC+rBDPwtCUtSYyVavept1E9FnqAmocDaThhk1HUDSQqFbpfcYgbiAobYcVRDgrZbzC6f8F2n9PkgNe5p4jP1f0cBibXjJHadCdDIvq5NxfdqnZwGh91C6iwn1GL+0SBXnFI+jZH3pUFBauwICCPdrlyMwIAgF7bvQghfn/yWtBv17/XwR6PWPCrTqLDP7AvbZyH7xprslLVYmwl2ze7ZI3vybrwH79n7PW9d8ErPNnsB789fmV6pPA==</latexit>
=
=
=
![Page 19: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/19.jpg)
Adaboost Example
19
o First Decision Stump:
err1 = 0.30↵1 = 0.42
<latexit sha1_base64="VfKIA+emCuEy8V1BOGlSXIJ9ZDg=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTFxYVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9EK+d7zHg/pBwKPIj0BlI/gnfm34Eep+6fZ3eK9aC+rBDPwtCUtSYyVavept1E9FnqAmocDaThhk1HUDSQqFbpfcYgbiAobYcVRDgrZbzC6f8F2n9PkgNe5p4jP1f0cBibXjJHadCdDIvq5NxfdqnZwGh91C6iwn1GL+0SBXnFI+jZH3pUFBauwICCPdrlyMwIAgF7bvQghfn/yWtBv17/XwR6PWPCrTqLDP7AvbZyH7xprslLVYmwl2ze7ZI3vybrwH79n7PW9d8ErPNnsB789fmV6pPA==</latexit><latexit sha1_base64="VfKIA+emCuEy8V1BOGlSXIJ9ZDg=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTFxYVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9EK+d7zHg/pBwKPIj0BlI/gnfm34Eep+6fZ3eK9aC+rBDPwtCUtSYyVavept1E9FnqAmocDaThhk1HUDSQqFbpfcYgbiAobYcVRDgrZbzC6f8F2n9PkgNe5p4jP1f0cBibXjJHadCdDIvq5NxfdqnZwGh91C6iwn1GL+0SBXnFI+jZH3pUFBauwICCPdrlyMwIAgF7bvQghfn/yWtBv17/XwR6PWPCrTqLDP7AvbZyH7xprslLVYmwl2ze7ZI3vybrwH79n7PW9d8ErPNnsB789fmV6pPA==</latexit><latexit sha1_base64="VfKIA+emCuEy8V1BOGlSXIJ9ZDg=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTFxYVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9EK+d7zHg/pBwKPIj0BlI/gnfm34Eep+6fZ3eK9aC+rBDPwtCUtSYyVavept1E9FnqAmocDaThhk1HUDSQqFbpfcYgbiAobYcVRDgrZbzC6f8F2n9PkgNe5p4jP1f0cBibXjJHadCdDIvq5NxfdqnZwGh91C6iwn1GL+0SBXnFI+jZH3pUFBauwICCPdrlyMwIAgF7bvQghfn/yWtBv17/XwR6PWPCrTqLDP7AvbZyH7xprslLVYmwl2ze7ZI3vybrwH79n7PW9d8ErPNnsB789fmV6pPA==</latexit><latexit sha1_base64="VfKIA+emCuEy8V1BOGlSXIJ9ZDg=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTFxYVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9EK+d7zHg/pBwKPIj0BlI/gnfm34Eep+6fZ3eK9aC+rBDPwtCUtSYyVavept1E9FnqAmocDaThhk1HUDSQqFbpfcYgbiAobYcVRDgrZbzC6f8F2n9PkgNe5p4jP1f0cBibXjJHadCdDIvq5NxfdqnZwGh91C6iwn1GL+0SBXnFI+jZH3pUFBauwICCPdrlyMwIAgF7bvQghfn/yWtBv17/XwR6PWPCrTqLDP7AvbZyH7xprslLVYmwl2ze7ZI3vybrwH79n7PW9d8ErPNnsB789fmV6pPA==</latexit>
![Page 20: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/20.jpg)
Adaboost Example
20
o Second Decision Stump:
=
=
=
-
:-
![Page 21: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/21.jpg)
Adaboost Example
21
o Second Decision Stump:
err2 = 0.21↵2 = 0.65
<latexit sha1_base64="pxGzwmjh9bwZluY3uPjjFj+47E4=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTMBdVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9Bp873iPB/VGyKPIj0BlI/gnfj3wI9T90u3v8F61FtSDGfhbEpakxkq0etXbqJ+KPEFNQoG1nTDIqOsGkhQK3S65xQzEBQyx46iGBG23mF0+4btO6fNBatzTxGfq/44CEmvHSew6E6CRfV2biu/VOjkNDruF1FlOqMX8o0GuOKV8GiPvS4OC1NgREEa6XbkYgQFBLmzfhRC+PvktaTfq3+vhj0ateVSmUWGf2Re2z0L2jTXZKWuxNhPsmt2zR/bk3XgP3rP3e9664JWebfYC3p+/pISpQw==</latexit><latexit sha1_base64="pxGzwmjh9bwZluY3uPjjFj+47E4=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTMBdVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9Bp873iPB/VGyKPIj0BlI/gnfj3wI9T90u3v8F61FtSDGfhbEpakxkq0etXbqJ+KPEFNQoG1nTDIqOsGkhQK3S65xQzEBQyx46iGBG23mF0+4btO6fNBatzTxGfq/44CEmvHSew6E6CRfV2biu/VOjkNDruF1FlOqMX8o0GuOKV8GiPvS4OC1NgREEa6XbkYgQFBLmzfhRC+PvktaTfq3+vhj0ateVSmUWGf2Re2z0L2jTXZKWuxNhPsmt2zR/bk3XgP3rP3e9664JWebfYC3p+/pISpQw==</latexit><latexit sha1_base64="pxGzwmjh9bwZluY3uPjjFj+47E4=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTMBdVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9Bp873iPB/VGyKPIj0BlI/gnfj3wI9T90u3v8F61FtSDGfhbEpakxkq0etXbqJ+KPEFNQoG1nTDIqOsGkhQK3S65xQzEBQyx46iGBG23mF0+4btO6fNBatzTxGfq/44CEmvHSew6E6CRfV2biu/VOjkNDruF1FlOqMX8o0GuOKV8GiPvS4OC1NgREEa6XbkYgQFBLmzfhRC+PvktaTfq3+vhj0ateVSmUWGf2Re2z0L2jTXZKWuxNhPsmt2zR/bk3XgP3rP3e9664JWebfYC3p+/pISpQw==</latexit><latexit sha1_base64="pxGzwmjh9bwZluY3uPjjFj+47E4=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTMBdVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9Bp873iPB/VGyKPIj0BlI/gnfj3wI9T90u3v8F61FtSDGfhbEpakxkq0etXbqJ+KPEFNQoG1nTDIqOsGkhQK3S65xQzEBQyx46iGBG23mF0+4btO6fNBatzTxGfq/44CEmvHSew6E6CRfV2biu/VOjkNDruF1FlOqMX8o0GuOKV8GiPvS4OC1NgREEa6XbkYgQFBLmzfhRC+PvktaTfq3+vhj0ateVSmUWGf2Re2z0L2jTXZKWuxNhPsmt2zR/bk3XgP3rP3e9664JWebfYC3p+/pISpQw==</latexit>
.
-
÷
![Page 22: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/22.jpg)
Adaboost Example
22
o Second Decision Stump:
err2 = 0.21↵2 = 0.65
<latexit sha1_base64="pxGzwmjh9bwZluY3uPjjFj+47E4=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTMBdVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9Bp873iPB/VGyKPIj0BlI/gnfj3wI9T90u3v8F61FtSDGfhbEpakxkq0etXbqJ+KPEFNQoG1nTDIqOsGkhQK3S65xQzEBQyx46iGBG23mF0+4btO6fNBatzTxGfq/44CEmvHSew6E6CRfV2biu/VOjkNDruF1FlOqMX8o0GuOKV8GiPvS4OC1NgREEa6XbkYgQFBLmzfhRC+PvktaTfq3+vhj0ateVSmUWGf2Re2z0L2jTXZKWuxNhPsmt2zR/bk3XgP3rP3e9664JWebfYC3p+/pISpQw==</latexit><latexit sha1_base64="pxGzwmjh9bwZluY3uPjjFj+47E4=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTMBdVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9Bp873iPB/VGyKPIj0BlI/gnfj3wI9T90u3v8F61FtSDGfhbEpakxkq0etXbqJ+KPEFNQoG1nTDIqOsGkhQK3S65xQzEBQyx46iGBG23mF0+4btO6fNBatzTxGfq/44CEmvHSew6E6CRfV2biu/VOjkNDruF1FlOqMX8o0GuOKV8GiPvS4OC1NgREEa6XbkYgQFBLmzfhRC+PvktaTfq3+vhj0ateVSmUWGf2Re2z0L2jTXZKWuxNhPsmt2zR/bk3XgP3rP3e9664JWebfYC3p+/pISpQw==</latexit><latexit sha1_base64="pxGzwmjh9bwZluY3uPjjFj+47E4=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTMBdVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9Bp873iPB/VGyKPIj0BlI/gnfj3wI9T90u3v8F61FtSDGfhbEpakxkq0etXbqJ+KPEFNQoG1nTDIqOsGkhQK3S65xQzEBQyx46iGBG23mF0+4btO6fNBatzTxGfq/44CEmvHSew6E6CRfV2biu/VOjkNDruF1FlOqMX8o0GuOKV8GiPvS4OC1NgREEa6XbkYgQFBLmzfhRC+PvktaTfq3+vhj0ateVSmUWGf2Re2z0L2jTXZKWuxNhPsmt2zR/bk3XgP3rP3e9664JWebfYC3p+/pISpQw==</latexit><latexit sha1_base64="pxGzwmjh9bwZluY3uPjjFj+47E4=">AAACOHicbVBNSyNBEO3RVbPjV1aPe2k2Kp7CTMBdVxACXjxmYbMKmRBqOpWksadn6K4Rw5C/5cV/4U3w4kFlr/sLtpMMy/rxoOHxql511YszJS0FwZ23sPhhaXml8tFfXVvf2Kx+2vpl09wIbItUpeY8BotKamyTJIXnmUFIYoVn8cXJtH52icbKVP+kcYbdBIZaDqQAclKv2trxoxiHUhdgDIwnhRFq4keEV2SSAo2Z9Bp873iPB/VGyKPIj0BlI/gnfj3wI9T90u3v8F61FtSDGfhbEpakxkq0etXbqJ+KPEFNQoG1nTDIqOsGkhQK3S65xQzEBQyx46iGBG23mF0+4btO6fNBatzTxGfq/44CEmvHSew6E6CRfV2biu/VOjkNDruF1FlOqMX8o0GuOKV8GiPvS4OC1NgREEa6XbkYgQFBLmzfhRC+PvktaTfq3+vhj0ateVSmUWGf2Re2z0L2jTXZKWuxNhPsmt2zR/bk3XgP3rP3e9664JWebfYC3p+/pISpQw==</latexit>
![Page 23: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/23.jpg)
Adaboost Example
23
o Third Decision Stump:
![Page 24: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/24.jpg)
Adaboost Example
24
o Third Decision Stump:
err3 = 0.14↵3 = 0.92
<latexit sha1_base64="aCuZFloAeoN85IH1WHegez00ejU=">AAACOHicbVBNa9tAEF2laeuqH3GbYy5L7JqejOQE2gQChlxydCFODJYxo/XYXrxaid1RqRH+W7nkX/RWyCWHNuTaX9C1I0Jq98HC48282ZkXZ0paCoKf3taz7ecvXlZe+a/fvH23U33/4cKmuRHYFalKTS8Gi0pq7JIkhb3MICSxwst4drqsX35DY2Wqz2me4SCBiZZjKYCcNKx26n4U40TqAoyB+aIwQi38iPA7maRAYxbDA944afCgGR7yKPIjUNkUHsWjlh+hHpVuv86H1VrQDFbgmyQsSY2V6AyrP6JRKvIENQkF1vbDIKOBG0hSKHS75BYzEDOYYN9RDQnaQbG6fME/OmXEx6lxTxNfqU8dBSTWzpPYdSZAU7teW4r/q/VzGn8ZFFJnOaEWDx+Nc8Up5csY+UgaFKTmjoAw0u3KxRQMCHJh+y6EcP3kTdJtNY+a4ddWrX1cplFhe2yffWIh+8za7Ix1WJcJdsVu2C/227v2br077/6hdcsrPbvsH3h//gKrH6lH</latexit><latexit sha1_base64="aCuZFloAeoN85IH1WHegez00ejU=">AAACOHicbVBNa9tAEF2laeuqH3GbYy5L7JqejOQE2gQChlxydCFODJYxo/XYXrxaid1RqRH+W7nkX/RWyCWHNuTaX9C1I0Jq98HC48282ZkXZ0paCoKf3taz7ecvXlZe+a/fvH23U33/4cKmuRHYFalKTS8Gi0pq7JIkhb3MICSxwst4drqsX35DY2Wqz2me4SCBiZZjKYCcNKx26n4U40TqAoyB+aIwQi38iPA7maRAYxbDA944afCgGR7yKPIjUNkUHsWjlh+hHpVuv86H1VrQDFbgmyQsSY2V6AyrP6JRKvIENQkF1vbDIKOBG0hSKHS75BYzEDOYYN9RDQnaQbG6fME/OmXEx6lxTxNfqU8dBSTWzpPYdSZAU7teW4r/q/VzGn8ZFFJnOaEWDx+Nc8Up5csY+UgaFKTmjoAw0u3KxRQMCHJh+y6EcP3kTdJtNY+a4ddWrX1cplFhe2yffWIh+8za7Ix1WJcJdsVu2C/227v2br077/6hdcsrPbvsH3h//gKrH6lH</latexit><latexit sha1_base64="aCuZFloAeoN85IH1WHegez00ejU=">AAACOHicbVBNa9tAEF2laeuqH3GbYy5L7JqejOQE2gQChlxydCFODJYxo/XYXrxaid1RqRH+W7nkX/RWyCWHNuTaX9C1I0Jq98HC48282ZkXZ0paCoKf3taz7ecvXlZe+a/fvH23U33/4cKmuRHYFalKTS8Gi0pq7JIkhb3MICSxwst4drqsX35DY2Wqz2me4SCBiZZjKYCcNKx26n4U40TqAoyB+aIwQi38iPA7maRAYxbDA944afCgGR7yKPIjUNkUHsWjlh+hHpVuv86H1VrQDFbgmyQsSY2V6AyrP6JRKvIENQkF1vbDIKOBG0hSKHS75BYzEDOYYN9RDQnaQbG6fME/OmXEx6lxTxNfqU8dBSTWzpPYdSZAU7teW4r/q/VzGn8ZFFJnOaEWDx+Nc8Up5csY+UgaFKTmjoAw0u3KxRQMCHJh+y6EcP3kTdJtNY+a4ddWrX1cplFhe2yffWIh+8za7Ix1WJcJdsVu2C/227v2br077/6hdcsrPbvsH3h//gKrH6lH</latexit><latexit sha1_base64="aCuZFloAeoN85IH1WHegez00ejU=">AAACOHicbVBNa9tAEF2laeuqH3GbYy5L7JqejOQE2gQChlxydCFODJYxo/XYXrxaid1RqRH+W7nkX/RWyCWHNuTaX9C1I0Jq98HC48282ZkXZ0paCoKf3taz7ecvXlZe+a/fvH23U33/4cKmuRHYFalKTS8Gi0pq7JIkhb3MICSxwst4drqsX35DY2Wqz2me4SCBiZZjKYCcNKx26n4U40TqAoyB+aIwQi38iPA7maRAYxbDA944afCgGR7yKPIjUNkUHsWjlh+hHpVuv86H1VrQDFbgmyQsSY2V6AyrP6JRKvIENQkF1vbDIKOBG0hSKHS75BYzEDOYYN9RDQnaQbG6fME/OmXEx6lxTxNfqU8dBSTWzpPYdSZAU7teW4r/q/VzGn8ZFFJnOaEWDx+Nc8Up5csY+UgaFKTmjoAw0u3KxRQMCHJh+y6EcP3kTdJtNY+a4ddWrX1cplFhe2yffWIh+8za7Ix1WJcJdsVu2C/227v2br077/6hdcsrPbvsH3h//gKrH6lH</latexit>
![Page 25: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/25.jpg)
Example: Boosted Classifier
25
0.42 + 0.65 + 0.92
=
" "a 1 1+1
-
'- -1
--1 L
•
42C1) t .65( 1) f. 921 D
×x
.yzf , ) + .
651 - 1) + r{2( D
= 7 0= -1.07+0-92=
- g.15<0⇒ +
![Page 26: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/26.jpg)
Performance
26
Recall the standard experiment of measuring train and validation error vs model complexity
Once overfitting begins, validation error goes up
x
![Page 27: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/27.jpg)
Performance
27
Boosting has a remarkable affect. Think of number of weak learners as model complexity
Even after training error becomes zero, validation error continues to improve!
tie
![Page 28: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/28.jpg)
Performance
28
HOW CAN THIS BE?!
![Page 29: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/29.jpg)
Boosting as Margin Maximizing Algo
29
Adaboost actually induces a margin, and tries to maximize it
Despite the fact that we make the model more complex with each weak learner, the model actually becomes more confident with each boosting iteration
This confidence can be represented as a margin
Our learn classifier with K weak learners is
Define an intermediate classifier by where
H(x) = signh PK
k=1 ↵khk(x)i
<latexit sha1_base64="FEylyjCW/EITZelDkNS1QMOHjQU=">AAACPXicbVA9a9xAFFw5sWPLX5e4dLPkbLCbQ7rGSWEwpDGkcSCKDSdZrPaeTot2V2L3KfgQuj+WJr8hXco0LpKQNm32zgdJbA8sDDNvePsmq6WwGARfvZUnT1fXnq1v+Jtb2zu7vecvPtiqMRwiXsnKXGXMghQaIhQo4ao2wFQm4TIr38z9y49grKj0e5zWkCg20SIXnKGT0l504J8ftXGW05vumJ7SGOEGjWqtmOiOxhJyHM1mNLaNStvyNOyu39KYybpgaUmLtPybnc1iIyYFJv4BTXv9YBAsQB+ScEn6ZImLtPclHle8UaCRS2btKAxqTFpmUHAJnR83FmrGSzaBkaOaKbBJuzi/o4dOGdO8Mu5ppAv130TLlLVTlblJxbCw9725+Jg3ajB/lbRC1w2C5neL8kZSrOi8SzoWBjjKqSOMG+H+SnnBDOPoGvddCeH9kx+SaDh4PQjfDftnJ8s21sk+eUmOSEhOyBk5JxckIpx8It/Id/LD++zdej+9X3ejK94ys0f+g/f7Dwo8rXE=</latexit><latexit sha1_base64="FEylyjCW/EITZelDkNS1QMOHjQU=">AAACPXicbVA9a9xAFFw5sWPLX5e4dLPkbLCbQ7rGSWEwpDGkcSCKDSdZrPaeTot2V2L3KfgQuj+WJr8hXco0LpKQNm32zgdJbA8sDDNvePsmq6WwGARfvZUnT1fXnq1v+Jtb2zu7vecvPtiqMRwiXsnKXGXMghQaIhQo4ao2wFQm4TIr38z9y49grKj0e5zWkCg20SIXnKGT0l504J8ftXGW05vumJ7SGOEGjWqtmOiOxhJyHM1mNLaNStvyNOyu39KYybpgaUmLtPybnc1iIyYFJv4BTXv9YBAsQB+ScEn6ZImLtPclHle8UaCRS2btKAxqTFpmUHAJnR83FmrGSzaBkaOaKbBJuzi/o4dOGdO8Mu5ppAv130TLlLVTlblJxbCw9725+Jg3ajB/lbRC1w2C5neL8kZSrOi8SzoWBjjKqSOMG+H+SnnBDOPoGvddCeH9kx+SaDh4PQjfDftnJ8s21sk+eUmOSEhOyBk5JxckIpx8It/Id/LD++zdej+9X3ejK94ys0f+g/f7Dwo8rXE=</latexit><latexit sha1_base64="FEylyjCW/EITZelDkNS1QMOHjQU=">AAACPXicbVA9a9xAFFw5sWPLX5e4dLPkbLCbQ7rGSWEwpDGkcSCKDSdZrPaeTot2V2L3KfgQuj+WJr8hXco0LpKQNm32zgdJbA8sDDNvePsmq6WwGARfvZUnT1fXnq1v+Jtb2zu7vecvPtiqMRwiXsnKXGXMghQaIhQo4ao2wFQm4TIr38z9y49grKj0e5zWkCg20SIXnKGT0l504J8ftXGW05vumJ7SGOEGjWqtmOiOxhJyHM1mNLaNStvyNOyu39KYybpgaUmLtPybnc1iIyYFJv4BTXv9YBAsQB+ScEn6ZImLtPclHle8UaCRS2btKAxqTFpmUHAJnR83FmrGSzaBkaOaKbBJuzi/o4dOGdO8Mu5ppAv130TLlLVTlblJxbCw9725+Jg3ajB/lbRC1w2C5neL8kZSrOi8SzoWBjjKqSOMG+H+SnnBDOPoGvddCeH9kx+SaDh4PQjfDftnJ8s21sk+eUmOSEhOyBk5JxckIpx8It/Id/LD++zdej+9X3ejK94ys0f+g/f7Dwo8rXE=</latexit><latexit sha1_base64="FEylyjCW/EITZelDkNS1QMOHjQU=">AAACPXicbVA9a9xAFFw5sWPLX5e4dLPkbLCbQ7rGSWEwpDGkcSCKDSdZrPaeTot2V2L3KfgQuj+WJr8hXco0LpKQNm32zgdJbA8sDDNvePsmq6WwGARfvZUnT1fXnq1v+Jtb2zu7vecvPtiqMRwiXsnKXGXMghQaIhQo4ao2wFQm4TIr38z9y49grKj0e5zWkCg20SIXnKGT0l504J8ftXGW05vumJ7SGOEGjWqtmOiOxhJyHM1mNLaNStvyNOyu39KYybpgaUmLtPybnc1iIyYFJv4BTXv9YBAsQB+ScEn6ZImLtPclHle8UaCRS2btKAxqTFpmUHAJnR83FmrGSzaBkaOaKbBJuzi/o4dOGdO8Mu5ppAv130TLlLVTlblJxbCw9725+Jg3ajB/lbRC1w2C5neL8kZSrOi8SzoWBjjKqSOMG+H+SnnBDOPoGvddCeH9kx+SaDh4PQjfDftnJ8s21sk+eUmOSEhOyBk5JxckIpx8It/Id/LD++zdej+9X3ejK94ys0f+g/f7Dwo8rXE=</latexit>
H`(x) = signh P`
k=1 ↵khk(x)i
<latexit sha1_base64="umnZDpFWX+FMRSuW0+Ga8B7yjrU=">AAACRXicbVBNb9QwFHRKCyUUWOiRi9UtUrmskl5KD5UqcemxlVhaaR0ix/uysWI7kf1SdRVl/xwXztz4CVx6aCuu4N2uRD8Y6UmjmTey32S1kg6j6Gew8mR17emz9efhi42Xr1733rz94qrGChiKSlX2LOMOlDQwRIkKzmoLXGcKTrPy09w/PQfrZGU+47SGRPOJkbkUHL2U9pLt8ChloNROy7KcXnQf6AFlCBdodevkxHSUKchxNJtR5hqdtuVB3H2dJyjjqi54WtIiLf/FZzNm5aTAJNymaa8fDaIF6GMSL0mfLHGc9n6wcSUaDQaF4s6N4qjGpOUWpVDQhaxxUHNR8gmMPDVcg0vaRQ0dfe+VMc0r68cgXah3Ey3Xzk115jc1x8I99Obi/7xRg/nHpJWmbhCMuH0obxTFis47pWNpQaCaesKFlf6vVBTccoG++dCXED88+TEZ7g72B/HJbv9wb9nGOnlHtsgOickeOSRH5JgMiSDfyC9yRa6D78FlcBP8vl1dCZaZTXIPwZ+/PNmxBw==</latexit><latexit sha1_base64="umnZDpFWX+FMRSuW0+Ga8B7yjrU=">AAACRXicbVBNb9QwFHRKCyUUWOiRi9UtUrmskl5KD5UqcemxlVhaaR0ix/uysWI7kf1SdRVl/xwXztz4CVx6aCuu4N2uRD8Y6UmjmTey32S1kg6j6Gew8mR17emz9efhi42Xr1733rz94qrGChiKSlX2LOMOlDQwRIkKzmoLXGcKTrPy09w/PQfrZGU+47SGRPOJkbkUHL2U9pLt8ChloNROy7KcXnQf6AFlCBdodevkxHSUKchxNJtR5hqdtuVB3H2dJyjjqi54WtIiLf/FZzNm5aTAJNymaa8fDaIF6GMSL0mfLHGc9n6wcSUaDQaF4s6N4qjGpOUWpVDQhaxxUHNR8gmMPDVcg0vaRQ0dfe+VMc0r68cgXah3Ey3Xzk115jc1x8I99Obi/7xRg/nHpJWmbhCMuH0obxTFis47pWNpQaCaesKFlf6vVBTccoG++dCXED88+TEZ7g72B/HJbv9wb9nGOnlHtsgOickeOSRH5JgMiSDfyC9yRa6D78FlcBP8vl1dCZaZTXIPwZ+/PNmxBw==</latexit><latexit sha1_base64="umnZDpFWX+FMRSuW0+Ga8B7yjrU=">AAACRXicbVBNb9QwFHRKCyUUWOiRi9UtUrmskl5KD5UqcemxlVhaaR0ix/uysWI7kf1SdRVl/xwXztz4CVx6aCuu4N2uRD8Y6UmjmTey32S1kg6j6Gew8mR17emz9efhi42Xr1733rz94qrGChiKSlX2LOMOlDQwRIkKzmoLXGcKTrPy09w/PQfrZGU+47SGRPOJkbkUHL2U9pLt8ChloNROy7KcXnQf6AFlCBdodevkxHSUKchxNJtR5hqdtuVB3H2dJyjjqi54WtIiLf/FZzNm5aTAJNymaa8fDaIF6GMSL0mfLHGc9n6wcSUaDQaF4s6N4qjGpOUWpVDQhaxxUHNR8gmMPDVcg0vaRQ0dfe+VMc0r68cgXah3Ey3Xzk115jc1x8I99Obi/7xRg/nHpJWmbhCMuH0obxTFis47pWNpQaCaesKFlf6vVBTccoG++dCXED88+TEZ7g72B/HJbv9wb9nGOnlHtsgOickeOSRH5JgMiSDfyC9yRa6D78FlcBP8vl1dCZaZTXIPwZ+/PNmxBw==</latexit><latexit sha1_base64="umnZDpFWX+FMRSuW0+Ga8B7yjrU=">AAACRXicbVBNb9QwFHRKCyUUWOiRi9UtUrmskl5KD5UqcemxlVhaaR0ix/uysWI7kf1SdRVl/xwXztz4CVx6aCuu4N2uRD8Y6UmjmTey32S1kg6j6Gew8mR17emz9efhi42Xr1733rz94qrGChiKSlX2LOMOlDQwRIkKzmoLXGcKTrPy09w/PQfrZGU+47SGRPOJkbkUHL2U9pLt8ChloNROy7KcXnQf6AFlCBdodevkxHSUKchxNJtR5hqdtuVB3H2dJyjjqi54WtIiLf/FZzNm5aTAJNymaa8fDaIF6GMSL0mfLHGc9n6wcSUaDQaF4s6N4qjGpOUWpVDQhaxxUHNR8gmMPDVcg0vaRQ0dfe+VMc0r68cgXah3Ey3Xzk115jc1x8I99Obi/7xRg/nHpJWmbhCMuH0obxTFis47pWNpQaCaesKFlf6vVBTccoG++dCXED88+TEZ7g72B/HJbv9wb9nGOnlHtsgOickeOSRH5JgMiSDfyC9yRa6D78FlcBP8vl1dCZaZTXIPwZ+/PNmxBw==</latexit>
` K<latexit sha1_base64="XRmmFao92YqeU8l4WDesJQTMhL4=">AAAB+HicbVBNT8JAEN3iF9avqkcvG8HEE2m5oDcSLyZeMBEhoQ3ZLlPYsN3W3S0JafgnXjyo8epP8ea/cYEeFHzJJC/vzWRmXphyprTrfluljc2t7Z3yrr23f3B45ByfPKokkxTaNOGJ7IZEAWcC2pppDt1UAolDDp1wfDP3OxOQiiXiQU9TCGIyFCxilGgj9R2navvAOfY5POE7u4r7TsWtuQvgdeIVpIIKtPrOlz9IaBaD0JQTpXqem+ogJ1IzymFm+5mClNAxGULPUEFiUEG+uHyGL4wywFEiTQmNF+rviZzESk3j0HTGRI/UqjcX//N6mY6ugpyJNNMg6HJRlHGsEzyPAQ+YBKr51BBCJTO3YjoiklBtwrJNCN7qy+ukXa9d17z7eqXZKNIoozN0ji6RhxqoiW5RC7URRRP0jF7Rm5VbL9a79bFsLVnFzCn6A+vzB8lakVA=</latexit><latexit sha1_base64="XRmmFao92YqeU8l4WDesJQTMhL4=">AAAB+HicbVBNT8JAEN3iF9avqkcvG8HEE2m5oDcSLyZeMBEhoQ3ZLlPYsN3W3S0JafgnXjyo8epP8ea/cYEeFHzJJC/vzWRmXphyprTrfluljc2t7Z3yrr23f3B45ByfPKokkxTaNOGJ7IZEAWcC2pppDt1UAolDDp1wfDP3OxOQiiXiQU9TCGIyFCxilGgj9R2navvAOfY5POE7u4r7TsWtuQvgdeIVpIIKtPrOlz9IaBaD0JQTpXqem+ogJ1IzymFm+5mClNAxGULPUEFiUEG+uHyGL4wywFEiTQmNF+rviZzESk3j0HTGRI/UqjcX//N6mY6ugpyJNNMg6HJRlHGsEzyPAQ+YBKr51BBCJTO3YjoiklBtwrJNCN7qy+ukXa9d17z7eqXZKNIoozN0ji6RhxqoiW5RC7URRRP0jF7Rm5VbL9a79bFsLVnFzCn6A+vzB8lakVA=</latexit><latexit sha1_base64="XRmmFao92YqeU8l4WDesJQTMhL4=">AAAB+HicbVBNT8JAEN3iF9avqkcvG8HEE2m5oDcSLyZeMBEhoQ3ZLlPYsN3W3S0JafgnXjyo8epP8ea/cYEeFHzJJC/vzWRmXphyprTrfluljc2t7Z3yrr23f3B45ByfPKokkxTaNOGJ7IZEAWcC2pppDt1UAolDDp1wfDP3OxOQiiXiQU9TCGIyFCxilGgj9R2navvAOfY5POE7u4r7TsWtuQvgdeIVpIIKtPrOlz9IaBaD0JQTpXqem+ogJ1IzymFm+5mClNAxGULPUEFiUEG+uHyGL4wywFEiTQmNF+rviZzESk3j0HTGRI/UqjcX//N6mY6ugpyJNNMg6HJRlHGsEzyPAQ+YBKr51BBCJTO3YjoiklBtwrJNCN7qy+ukXa9d17z7eqXZKNIoozN0ji6RhxqoiW5RC7URRRP0jF7Rm5VbL9a79bFsLVnFzCn6A+vzB8lakVA=</latexit><latexit sha1_base64="XRmmFao92YqeU8l4WDesJQTMhL4=">AAAB+HicbVBNT8JAEN3iF9avqkcvG8HEE2m5oDcSLyZeMBEhoQ3ZLlPYsN3W3S0JafgnXjyo8epP8ea/cYEeFHzJJC/vzWRmXphyprTrfluljc2t7Z3yrr23f3B45ByfPKokkxTaNOGJ7IZEAWcC2pppDt1UAolDDp1wfDP3OxOQiiXiQU9TCGIyFCxilGgj9R2navvAOfY5POE7u4r7TsWtuQvgdeIVpIIKtPrOlz9IaBaD0JQTpXqem+ogJ1IzymFm+5mClNAxGULPUEFiUEG+uHyGL4wywFEiTQmNF+rviZzESk3j0HTGRI/UqjcX//N6mY6ugpyJNNMg6HJRlHGsEzyPAQ+YBKr51BBCJTO3YjoiklBtwrJNCN7qy+ukXa9d17z7eqXZKNIoozN0ji6RhxqoiW5RC7URRRP0jF7Rm5VbL9a79bFsLVnFzCn6A+vzB8lakVA=</latexit>
=
-
![Page 30: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/30.jpg)
Boosting as Margin Maximizing Algo
30
Note that the classifier only returns +1 or -1, which doesn’t give us any measure of confidence
Define the normalized voting coefficients as
Define the margin of a training example after boosting iterations as
H`(x) = signh P`
k=1 ↵khk(x)i
<latexit sha1_base64="umnZDpFWX+FMRSuW0+Ga8B7yjrU=">AAACRXicbVBNb9QwFHRKCyUUWOiRi9UtUrmskl5KD5UqcemxlVhaaR0ix/uysWI7kf1SdRVl/xwXztz4CVx6aCuu4N2uRD8Y6UmjmTey32S1kg6j6Gew8mR17emz9efhi42Xr1733rz94qrGChiKSlX2LOMOlDQwRIkKzmoLXGcKTrPy09w/PQfrZGU+47SGRPOJkbkUHL2U9pLt8ChloNROy7KcXnQf6AFlCBdodevkxHSUKchxNJtR5hqdtuVB3H2dJyjjqi54WtIiLf/FZzNm5aTAJNymaa8fDaIF6GMSL0mfLHGc9n6wcSUaDQaF4s6N4qjGpOUWpVDQhaxxUHNR8gmMPDVcg0vaRQ0dfe+VMc0r68cgXah3Ey3Xzk115jc1x8I99Obi/7xRg/nHpJWmbhCMuH0obxTFis47pWNpQaCaesKFlf6vVBTccoG++dCXED88+TEZ7g72B/HJbv9wb9nGOnlHtsgOickeOSRH5JgMiSDfyC9yRa6D78FlcBP8vl1dCZaZTXIPwZ+/PNmxBw==</latexit><latexit sha1_base64="umnZDpFWX+FMRSuW0+Ga8B7yjrU=">AAACRXicbVBNb9QwFHRKCyUUWOiRi9UtUrmskl5KD5UqcemxlVhaaR0ix/uysWI7kf1SdRVl/xwXztz4CVx6aCuu4N2uRD8Y6UmjmTey32S1kg6j6Gew8mR17emz9efhi42Xr1733rz94qrGChiKSlX2LOMOlDQwRIkKzmoLXGcKTrPy09w/PQfrZGU+47SGRPOJkbkUHL2U9pLt8ChloNROy7KcXnQf6AFlCBdodevkxHSUKchxNJtR5hqdtuVB3H2dJyjjqi54WtIiLf/FZzNm5aTAJNymaa8fDaIF6GMSL0mfLHGc9n6wcSUaDQaF4s6N4qjGpOUWpVDQhaxxUHNR8gmMPDVcg0vaRQ0dfe+VMc0r68cgXah3Ey3Xzk115jc1x8I99Obi/7xRg/nHpJWmbhCMuH0obxTFis47pWNpQaCaesKFlf6vVBTccoG++dCXED88+TEZ7g72B/HJbv9wb9nGOnlHtsgOickeOSRH5JgMiSDfyC9yRa6D78FlcBP8vl1dCZaZTXIPwZ+/PNmxBw==</latexit><latexit sha1_base64="umnZDpFWX+FMRSuW0+Ga8B7yjrU=">AAACRXicbVBNb9QwFHRKCyUUWOiRi9UtUrmskl5KD5UqcemxlVhaaR0ix/uysWI7kf1SdRVl/xwXztz4CVx6aCuu4N2uRD8Y6UmjmTey32S1kg6j6Gew8mR17emz9efhi42Xr1733rz94qrGChiKSlX2LOMOlDQwRIkKzmoLXGcKTrPy09w/PQfrZGU+47SGRPOJkbkUHL2U9pLt8ChloNROy7KcXnQf6AFlCBdodevkxHSUKchxNJtR5hqdtuVB3H2dJyjjqi54WtIiLf/FZzNm5aTAJNymaa8fDaIF6GMSL0mfLHGc9n6wcSUaDQaF4s6N4qjGpOUWpVDQhaxxUHNR8gmMPDVcg0vaRQ0dfe+VMc0r68cgXah3Ey3Xzk115jc1x8I99Obi/7xRg/nHpJWmbhCMuH0obxTFis47pWNpQaCaesKFlf6vVBTccoG++dCXED88+TEZ7g72B/HJbv9wb9nGOnlHtsgOickeOSRH5JgMiSDfyC9yRa6D78FlcBP8vl1dCZaZTXIPwZ+/PNmxBw==</latexit><latexit sha1_base64="umnZDpFWX+FMRSuW0+Ga8B7yjrU=">AAACRXicbVBNb9QwFHRKCyUUWOiRi9UtUrmskl5KD5UqcemxlVhaaR0ix/uysWI7kf1SdRVl/xwXztz4CVx6aCuu4N2uRD8Y6UmjmTey32S1kg6j6Gew8mR17emz9efhi42Xr1733rz94qrGChiKSlX2LOMOlDQwRIkKzmoLXGcKTrPy09w/PQfrZGU+47SGRPOJkbkUHL2U9pLt8ChloNROy7KcXnQf6AFlCBdodevkxHSUKchxNJtR5hqdtuVB3H2dJyjjqi54WtIiLf/FZzNm5aTAJNymaa8fDaIF6GMSL0mfLHGc9n6wcSUaDQaF4s6N4qjGpOUWpVDQhaxxUHNR8gmMPDVcg0vaRQ0dfe+VMc0r68cgXah3Ey3Xzk115jc1x8I99Obi/7xRg/nHpJWmbhCMuH0obxTFis47pWNpQaCaesKFlf6vVBTccoG++dCXED88+TEZ7g72B/HJbv9wb9nGOnlHtsgOickeOSRH5JgMiSDfyC9yRa6D78FlcBP8vl1dCZaZTXIPwZ+/PNmxBw==</latexit>
↵̂k =↵kPKj=1 ↵j
<latexit sha1_base64="SZTz11lgK5a4CnIhcLXNBB/iTKk=">AAACJXicbZDLSsNAFIYnXmu8VV26GWwFVyXpprooFNwIbipYW2hqOJlO2mknF2YmQgl5Gje+ihsXVQRXvorTNgtt/WHg5zvncOb8XsyZVJb1Zaytb2xubRd2zN29/YPD4tHxg4wSQWiLRDwSHQ8k5SykLcUUp51YUAg8Ttve+HpWbz9RIVkU3qtJTHsBDELmMwJKI7dYL5vOEFTqAI+HkLljXMdO3xdAcuSOs9SRSeCmo7qdPd7iHI8ys4zdYsmqWHPhVWPnpoRyNd3i1OlHJAloqAgHKbu2FateCkIxwmlmOomkMZAxDGhX2xACKnvp/MwMn2vSx34k9AsVntPfEykEUk4CT3cGoIZyuTaD/9W6ifIveykL40TRkCwW+QnHKsKzzHCfCUoUn2gDRDD9V0yGoDNSOllTh2Avn7xqWtXKVcW+q5YatTyNAjpFZ+gC2aiGGugGNVELEfSMXtEUvRsvxpvxYXwuWteMfOYE/ZHx/QOvE6TW</latexit><latexit sha1_base64="SZTz11lgK5a4CnIhcLXNBB/iTKk=">AAACJXicbZDLSsNAFIYnXmu8VV26GWwFVyXpprooFNwIbipYW2hqOJlO2mknF2YmQgl5Gje+ihsXVQRXvorTNgtt/WHg5zvncOb8XsyZVJb1Zaytb2xubRd2zN29/YPD4tHxg4wSQWiLRDwSHQ8k5SykLcUUp51YUAg8Ttve+HpWbz9RIVkU3qtJTHsBDELmMwJKI7dYL5vOEFTqAI+HkLljXMdO3xdAcuSOs9SRSeCmo7qdPd7iHI8ys4zdYsmqWHPhVWPnpoRyNd3i1OlHJAloqAgHKbu2FateCkIxwmlmOomkMZAxDGhX2xACKnvp/MwMn2vSx34k9AsVntPfEykEUk4CT3cGoIZyuTaD/9W6ifIveykL40TRkCwW+QnHKsKzzHCfCUoUn2gDRDD9V0yGoDNSOllTh2Avn7xqWtXKVcW+q5YatTyNAjpFZ+gC2aiGGugGNVELEfSMXtEUvRsvxpvxYXwuWteMfOYE/ZHx/QOvE6TW</latexit><latexit sha1_base64="SZTz11lgK5a4CnIhcLXNBB/iTKk=">AAACJXicbZDLSsNAFIYnXmu8VV26GWwFVyXpprooFNwIbipYW2hqOJlO2mknF2YmQgl5Gje+ihsXVQRXvorTNgtt/WHg5zvncOb8XsyZVJb1Zaytb2xubRd2zN29/YPD4tHxg4wSQWiLRDwSHQ8k5SykLcUUp51YUAg8Ttve+HpWbz9RIVkU3qtJTHsBDELmMwJKI7dYL5vOEFTqAI+HkLljXMdO3xdAcuSOs9SRSeCmo7qdPd7iHI8ys4zdYsmqWHPhVWPnpoRyNd3i1OlHJAloqAgHKbu2FateCkIxwmlmOomkMZAxDGhX2xACKnvp/MwMn2vSx34k9AsVntPfEykEUk4CT3cGoIZyuTaD/9W6ifIveykL40TRkCwW+QnHKsKzzHCfCUoUn2gDRDD9V0yGoDNSOllTh2Avn7xqWtXKVcW+q5YatTyNAjpFZ+gC2aiGGugGNVELEfSMXtEUvRsvxpvxYXwuWteMfOYE/ZHx/QOvE6TW</latexit><latexit sha1_base64="SZTz11lgK5a4CnIhcLXNBB/iTKk=">AAACJXicbZDLSsNAFIYnXmu8VV26GWwFVyXpprooFNwIbipYW2hqOJlO2mknF2YmQgl5Gje+ihsXVQRXvorTNgtt/WHg5zvncOb8XsyZVJb1Zaytb2xubRd2zN29/YPD4tHxg4wSQWiLRDwSHQ8k5SykLcUUp51YUAg8Ttve+HpWbz9RIVkU3qtJTHsBDELmMwJKI7dYL5vOEFTqAI+HkLljXMdO3xdAcuSOs9SRSeCmo7qdPd7iHI8ys4zdYsmqWHPhVWPnpoRyNd3i1OlHJAloqAgHKbu2FateCkIxwmlmOomkMZAxDGhX2xACKnvp/MwMn2vSx34k9AsVntPfEykEUk4CT3cGoIZyuTaD/9W6ifIveykL40TRkCwW+QnHKsKzzHCfCUoUn2gDRDD9V0yGoDNSOllTh2Avn7xqWtXKVcW+q5YatTyNAjpFZ+gC2aiGGugGNVELEfSMXtEUvRsvxpvxYXwuWteMfOYE/ZHx/QOvE6TW</latexit>
`<latexit sha1_base64="7U7CjYA21V1TBNVh7eD2hhgBsyA=">AAAB7nicbVA9TwJBEJ3DLzy/UEubjWBiRe5o0I7ExhITT0jgQvaWOdiw9+Hungm58CdsLNTY+nvs/DcucIWCL5nk5b2ZzMwLUsGVdpxvq7SxubW9U9619/YPDo8qxycPKskkQ48lIpHdgCoUPEZPcy2wm0qkUSCwE0xu5n7nCaXiSXyvpyn6ER3FPOSMaiN1a3YfhaiRQaXq1J0FyDpxC1KFAu1B5as/TFgWYayZoEr1XCfVfk6l5kzgzO5nClPKJnSEPUNjGqHy88W9M3JhlCEJE2kq1mSh/p7IaaTUNApMZ0T1WK16c/E/r5fp8MrPeZxmGmO2XBRmguiEzJ8nQy6RaTE1hDLJza2EjamkTJuIbBOCu/ryOvEa9eu6e9eotppFGmU4g3O4BBea0IJbaIMHDAQ8wyu8WY/Wi/VufSxbS1Yxcwp/YH3+ALtNjpw=</latexit><latexit sha1_base64="7U7CjYA21V1TBNVh7eD2hhgBsyA=">AAAB7nicbVA9TwJBEJ3DLzy/UEubjWBiRe5o0I7ExhITT0jgQvaWOdiw9+Hungm58CdsLNTY+nvs/DcucIWCL5nk5b2ZzMwLUsGVdpxvq7SxubW9U9619/YPDo8qxycPKskkQ48lIpHdgCoUPEZPcy2wm0qkUSCwE0xu5n7nCaXiSXyvpyn6ER3FPOSMaiN1a3YfhaiRQaXq1J0FyDpxC1KFAu1B5as/TFgWYayZoEr1XCfVfk6l5kzgzO5nClPKJnSEPUNjGqHy88W9M3JhlCEJE2kq1mSh/p7IaaTUNApMZ0T1WK16c/E/r5fp8MrPeZxmGmO2XBRmguiEzJ8nQy6RaTE1hDLJza2EjamkTJuIbBOCu/ryOvEa9eu6e9eotppFGmU4g3O4BBea0IJbaIMHDAQ8wyu8WY/Wi/VufSxbS1Yxcwp/YH3+ALtNjpw=</latexit><latexit sha1_base64="7U7CjYA21V1TBNVh7eD2hhgBsyA=">AAAB7nicbVA9TwJBEJ3DLzy/UEubjWBiRe5o0I7ExhITT0jgQvaWOdiw9+Hungm58CdsLNTY+nvs/DcucIWCL5nk5b2ZzMwLUsGVdpxvq7SxubW9U9619/YPDo8qxycPKskkQ48lIpHdgCoUPEZPcy2wm0qkUSCwE0xu5n7nCaXiSXyvpyn6ER3FPOSMaiN1a3YfhaiRQaXq1J0FyDpxC1KFAu1B5as/TFgWYayZoEr1XCfVfk6l5kzgzO5nClPKJnSEPUNjGqHy88W9M3JhlCEJE2kq1mSh/p7IaaTUNApMZ0T1WK16c/E/r5fp8MrPeZxmGmO2XBRmguiEzJ8nQy6RaTE1hDLJza2EjamkTJuIbBOCu/ryOvEa9eu6e9eotppFGmU4g3O4BBea0IJbaIMHDAQ8wyu8WY/Wi/VufSxbS1Yxcwp/YH3+ALtNjpw=</latexit><latexit sha1_base64="7U7CjYA21V1TBNVh7eD2hhgBsyA=">AAAB7nicbVA9TwJBEJ3DLzy/UEubjWBiRe5o0I7ExhITT0jgQvaWOdiw9+Hungm58CdsLNTY+nvs/DcucIWCL5nk5b2ZzMwLUsGVdpxvq7SxubW9U9619/YPDo8qxycPKskkQ48lIpHdgCoUPEZPcy2wm0qkUSCwE0xu5n7nCaXiSXyvpyn6ER3FPOSMaiN1a3YfhaiRQaXq1J0FyDpxC1KFAu1B5as/TFgWYayZoEr1XCfVfk6l5kzgzO5nClPKJnSEPUNjGqHy88W9M3JhlCEJE2kq1mSh/p7IaaTUNApMZ0T1WK16c/E/r5fp8MrPeZxmGmO2XBRmguiEzJ8nQy6RaTE1hDLJza2EjamkTJuIbBOCu/ryOvEa9eu6e9eotppFGmU4g3O4BBea0IJbaIMHDAQ8wyu8WY/Wi/VufSxbS1Yxcwp/YH3+ALtNjpw=</latexit>
margin`(x) =P`
k=1: hk(x)=y ↵̂k �P`
k=1: hk(x) 6=y ↵̂k<latexit sha1_base64="ypH/YlW7x4zWU0a1ByjHvi5W4bk=">AAACc3icdVFNb9QwFHRSPsrytS0HDnAw3UUqh66SXigVlSpx4VgkllbaLNGL92VjxXaC/YK6itI/wL/jxs/gwhnvdg+lpSNZGs28ke15Wa2koyj6FYQbd+7eu7/5oPfw0eMnT/tb219c1ViBY1Gpyp5l4FBJg2OSpPCstgg6U3ialR+W/ul3tE5W5jMtapxqmBuZSwHkpbT/Y9hLCM/J6laDnUvTpQkqxXfbJMv5efeGH/HENTpty6P48KJIy6vOovu6mk4KoDYBVRfQpSXfuzWSGPx2S2rI0/4gGkUr8JskXpMBW+Mk7f9MZpVoNBoSCpybxFFN0xYsSaGw6yWNwxpECXOceGpAo5u2q9I6/torM55X1h9DfKVeTbSgnVvozE9qoMJd95bi/7xJQ/nBtJWmbgiNuLwobxSnii83wGfSoiC18ASElf6tXBRgQZDfU8+XEF//8k0y3h+9G8Wf9gfH79dtbLIXbIftspi9ZcfsIzthYybY7+B5wINXwZ/wZbgTDi9Hw2Cdecb+Qbj3F5zyvHI=</latexit><latexit sha1_base64="ypH/YlW7x4zWU0a1ByjHvi5W4bk=">AAACc3icdVFNb9QwFHRSPsrytS0HDnAw3UUqh66SXigVlSpx4VgkllbaLNGL92VjxXaC/YK6itI/wL/jxs/gwhnvdg+lpSNZGs28ke15Wa2koyj6FYQbd+7eu7/5oPfw0eMnT/tb219c1ViBY1Gpyp5l4FBJg2OSpPCstgg6U3ialR+W/ul3tE5W5jMtapxqmBuZSwHkpbT/Y9hLCM/J6laDnUvTpQkqxXfbJMv5efeGH/HENTpty6P48KJIy6vOovu6mk4KoDYBVRfQpSXfuzWSGPx2S2rI0/4gGkUr8JskXpMBW+Mk7f9MZpVoNBoSCpybxFFN0xYsSaGw6yWNwxpECXOceGpAo5u2q9I6/torM55X1h9DfKVeTbSgnVvozE9qoMJd95bi/7xJQ/nBtJWmbgiNuLwobxSnii83wGfSoiC18ASElf6tXBRgQZDfU8+XEF//8k0y3h+9G8Wf9gfH79dtbLIXbIftspi9ZcfsIzthYybY7+B5wINXwZ/wZbgTDi9Hw2Cdecb+Qbj3F5zyvHI=</latexit><latexit sha1_base64="ypH/YlW7x4zWU0a1ByjHvi5W4bk=">AAACc3icdVFNb9QwFHRSPsrytS0HDnAw3UUqh66SXigVlSpx4VgkllbaLNGL92VjxXaC/YK6itI/wL/jxs/gwhnvdg+lpSNZGs28ke15Wa2koyj6FYQbd+7eu7/5oPfw0eMnT/tb219c1ViBY1Gpyp5l4FBJg2OSpPCstgg6U3ialR+W/ul3tE5W5jMtapxqmBuZSwHkpbT/Y9hLCM/J6laDnUvTpQkqxXfbJMv5efeGH/HENTpty6P48KJIy6vOovu6mk4KoDYBVRfQpSXfuzWSGPx2S2rI0/4gGkUr8JskXpMBW+Mk7f9MZpVoNBoSCpybxFFN0xYsSaGw6yWNwxpECXOceGpAo5u2q9I6/torM55X1h9DfKVeTbSgnVvozE9qoMJd95bi/7xJQ/nBtJWmbgiNuLwobxSnii83wGfSoiC18ASElf6tXBRgQZDfU8+XEF//8k0y3h+9G8Wf9gfH79dtbLIXbIftspi9ZcfsIzthYybY7+B5wINXwZ/wZbgTDi9Hw2Cdecb+Qbj3F5zyvHI=</latexit><latexit sha1_base64="ypH/YlW7x4zWU0a1ByjHvi5W4bk=">AAACc3icdVFNb9QwFHRSPsrytS0HDnAw3UUqh66SXigVlSpx4VgkllbaLNGL92VjxXaC/YK6itI/wL/jxs/gwhnvdg+lpSNZGs28ke15Wa2koyj6FYQbd+7eu7/5oPfw0eMnT/tb219c1ViBY1Gpyp5l4FBJg2OSpPCstgg6U3ialR+W/ul3tE5W5jMtapxqmBuZSwHkpbT/Y9hLCM/J6laDnUvTpQkqxXfbJMv5efeGH/HENTpty6P48KJIy6vOovu6mk4KoDYBVRfQpSXfuzWSGPx2S2rI0/4gGkUr8JskXpMBW+Mk7f9MZpVoNBoSCpybxFFN0xYsSaGw6yWNwxpECXOceGpAo5u2q9I6/torM55X1h9DfKVeTbSgnVvozE9qoMJd95bi/7xJQ/nBtJWmbgiNuLwobxSnii83wGfSoiC18ASElf6tXBRgQZDfU8+XEF//8k0y3h+9G8Wf9gfH79dtbLIXbIftspi9ZcfsIzthYybY7+B5wINXwZ/wZbgTDi9Hw2Cdecb+Qbj3F5zyvHI=</latexit>
=
=
g- ==
=
![Page 31: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/31.jpg)
Boosting as Margin Maximizing Algo
31
Handwritten Digit Recognition with Adaboost
Note that Adaboost achieves zero training error after roughly 250 iterations
![Page 32: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/32.jpg)
Boosting as Margin Maximizing Algo
32
Handwritten Digit Recognition with Adaboost. Two easy and two hard training examples.
margin`(x) =P`
k=1: hk(x)=y ↵̂k �P`
k=1: hk(x) 6=y ↵̂k<latexit sha1_base64="ypH/YlW7x4zWU0a1ByjHvi5W4bk=">AAACc3icdVFNb9QwFHRSPsrytS0HDnAw3UUqh66SXigVlSpx4VgkllbaLNGL92VjxXaC/YK6itI/wL/jxs/gwhnvdg+lpSNZGs28ke15Wa2koyj6FYQbd+7eu7/5oPfw0eMnT/tb219c1ViBY1Gpyp5l4FBJg2OSpPCstgg6U3ialR+W/ul3tE5W5jMtapxqmBuZSwHkpbT/Y9hLCM/J6laDnUvTpQkqxXfbJMv5efeGH/HENTpty6P48KJIy6vOovu6mk4KoDYBVRfQpSXfuzWSGPx2S2rI0/4gGkUr8JskXpMBW+Mk7f9MZpVoNBoSCpybxFFN0xYsSaGw6yWNwxpECXOceGpAo5u2q9I6/torM55X1h9DfKVeTbSgnVvozE9qoMJd95bi/7xJQ/nBtJWmbgiNuLwobxSnii83wGfSoiC18ASElf6tXBRgQZDfU8+XEF//8k0y3h+9G8Wf9gfH79dtbLIXbIftspi9ZcfsIzthYybY7+B5wINXwZ/wZbgTDi9Hw2Cdecb+Qbj3F5zyvHI=</latexit><latexit sha1_base64="ypH/YlW7x4zWU0a1ByjHvi5W4bk=">AAACc3icdVFNb9QwFHRSPsrytS0HDnAw3UUqh66SXigVlSpx4VgkllbaLNGL92VjxXaC/YK6itI/wL/jxs/gwhnvdg+lpSNZGs28ke15Wa2koyj6FYQbd+7eu7/5oPfw0eMnT/tb219c1ViBY1Gpyp5l4FBJg2OSpPCstgg6U3ialR+W/ul3tE5W5jMtapxqmBuZSwHkpbT/Y9hLCM/J6laDnUvTpQkqxXfbJMv5efeGH/HENTpty6P48KJIy6vOovu6mk4KoDYBVRfQpSXfuzWSGPx2S2rI0/4gGkUr8JskXpMBW+Mk7f9MZpVoNBoSCpybxFFN0xYsSaGw6yWNwxpECXOceGpAo5u2q9I6/torM55X1h9DfKVeTbSgnVvozE9qoMJd95bi/7xJQ/nBtJWmbgiNuLwobxSnii83wGfSoiC18ASElf6tXBRgQZDfU8+XEF//8k0y3h+9G8Wf9gfH79dtbLIXbIftspi9ZcfsIzthYybY7+B5wINXwZ/wZbgTDi9Hw2Cdecb+Qbj3F5zyvHI=</latexit><latexit sha1_base64="ypH/YlW7x4zWU0a1ByjHvi5W4bk=">AAACc3icdVFNb9QwFHRSPsrytS0HDnAw3UUqh66SXigVlSpx4VgkllbaLNGL92VjxXaC/YK6itI/wL/jxs/gwhnvdg+lpSNZGs28ke15Wa2koyj6FYQbd+7eu7/5oPfw0eMnT/tb219c1ViBY1Gpyp5l4FBJg2OSpPCstgg6U3ialR+W/ul3tE5W5jMtapxqmBuZSwHkpbT/Y9hLCM/J6laDnUvTpQkqxXfbJMv5efeGH/HENTpty6P48KJIy6vOovu6mk4KoDYBVRfQpSXfuzWSGPx2S2rI0/4gGkUr8JskXpMBW+Mk7f9MZpVoNBoSCpybxFFN0xYsSaGw6yWNwxpECXOceGpAo5u2q9I6/torM55X1h9DfKVeTbSgnVvozE9qoMJd95bi/7xJQ/nBtJWmbgiNuLwobxSnii83wGfSoiC18ASElf6tXBRgQZDfU8+XEF//8k0y3h+9G8Wf9gfH79dtbLIXbIftspi9ZcfsIzthYybY7+B5wINXwZ/wZbgTDi9Hw2Cdecb+Qbj3F5zyvHI=</latexit><latexit sha1_base64="ypH/YlW7x4zWU0a1ByjHvi5W4bk=">AAACc3icdVFNb9QwFHRSPsrytS0HDnAw3UUqh66SXigVlSpx4VgkllbaLNGL92VjxXaC/YK6itI/wL/jxs/gwhnvdg+lpSNZGs28ke15Wa2koyj6FYQbd+7eu7/5oPfw0eMnT/tb219c1ViBY1Gpyp5l4FBJg2OSpPCstgg6U3ialR+W/ul3tE5W5jMtapxqmBuZSwHkpbT/Y9hLCM/J6laDnUvTpQkqxXfbJMv5efeGH/HENTpty6P48KJIy6vOovu6mk4KoDYBVRfQpSXfuzWSGPx2S2rI0/4gGkUr8JskXpMBW+Mk7f9MZpVoNBoSCpybxFFN0xYsSaGw6yWNwxpECXOceGpAo5u2q9I6/torM55X1h9DfKVeTbSgnVvozE9qoMJd95bi/7xJQ/nBtJWmbgiNuLwobxSnii83wGfSoiC18ASElf6tXBRgQZDfU8+XEF//8k0y3h+9G8Wf9gfH79dtbLIXbIftspi9ZcfsIzthYybY7+B5wINXwZ/wZbgTDi9Hw2Cdecb+Qbj3F5zyvHI=</latexit>
ownnunnmrnmmn
×
![Page 33: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/33.jpg)
Practical Advantages of Boosting
33
o It’s fast!
o Simple and easy to program (you’ll find out on Monday cuz we’re gonna do it!)
o No parameters to really tune, except K
o Flexible, can choose any weak learner
o Shift in mindset: Now can look for weak classifiers instead of strong classifiers
o Can be used in lots of settings: text learning, images, discrete features, continuous features)
![Page 34: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/34.jpg)
34
![Page 35: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/35.jpg)
35
![Page 36: Boosted Decision Treesketelsen/files/courses/csci4622/slides/lesson37.pdfPreviously on CSCI 4622 2 Ensemble Learning: Build a bunch of slightly different models.Predict by majority](https://reader034.fdocuments.net/reader034/viewer/2022052000/6011b3072fb21970510cbf86/html5/thumbnails/36.jpg)
36