Feature Selection with Branch and Bound

Norman Poh

Steps1. Construct an ordered tree satisfying:

2. Traverse the tree from right to left in depth-first search pattern

3. Evaluate the criterion at each level and sort them4. Prune the tree

where Jk means k variables to be eliminated and the order of Zp is determined by a discrimination criterion

The objective is to fine the optimal value:

while keeping a constantly updated lower bound:

Whenever the criterion evaluated for any node is less than the bound B, all nodes that are successors of that node also have criterion values less than B. So, we prune them.

“6 choose 2”

How to construct the order tree?

• Determine the number of levels of the tree– In the “6 choose 2” examples, we have to eliminate 4

variables and so the number of levels is 4• Each child node should have a number greater

than it’s parent’s• The right-most leave node (the note at the last

level) should reach the largest variable index (that is 6 in the “6 choose 2” example)

• A number of examples are given next

“5 choose 2”

“5 choose 3”

“6 choose 3”

Your tutorial question:1. Construct an ordered tree according to “4 choose 2”

2. Eliminate features x1, x2, x3 and x4 – calculate the criterion and sort them

3. Put the tree

Reference

• PATRENAHALLI M. NARENDRA and KEINOSUKE, FUKUNAGA , “A Branch and Bound Algorithm for Feature Subset Selection”, IEEE TRANSACTIONS ON COMPUTERS, VOL. C-26, NO. 9, SEPTEMBER 1977