On L1q Regularized Regression Authors: Han Liu and Jian Zhang Presented by Jun Liu.
-
date post
22-Dec-2015 -
Category
Documents
-
view
220 -
download
1
Transcript of On L1q Regularized Regression Authors: Han Liu and Jian Zhang Presented by Jun Liu.
Outline
• Proposition 2.1, 2.2 (Subgradient, linearly dependent)
• Definition 2.4-2.7 (Properties to be established)
• Theorem 3.1 (Variable Selection Consistency)
• Lemma 4.1 (Technical lemma)
• Assumption 1, Theorem 4.3 (Consistency, linear model)
• Assumption 2. Theorem 4.5 (Inequality, misspecified model)
• Assumption 4, Theorem 5.1 (Risk consistency)
We want to find the such that the q’-norm of is either equal to a constant value (for nonzero groups) or bounded (for zero groups)
Hint
This result is similar to the Lasso. The key is that , so that any m>n columns of X are linearly dependent.
Outline
• Proposition 2.1, 2.2 (Subgradient, linear dependent)
• Definition 2.4-2.7 (Properties to be established)
• Theorem 3.1 (Variable Selection Consistency)
• Lemma 4.1 (Technical lemma)
• Assumption 1, Theorem 4.3 (Consistency, linear model)
• Assumption 2. Theorem 4.5 (Inequality, misspecified model)
• Assumption 4, Theorem 5.1 (Risk consistency)
Outline
• Proposition 2.1, 2.2 (Subgradient, linear dependent)
• Definition 2.4-2.7 (Properties to be established)
• Theorem 3.1 (Variable Selection Consistency)
• Lemma 4.1 (Technical lemma)
• Assumption 1, Theorem 4.3 (Consistency, linear model)
• Assumption 2. Theorem 4.5 (Inequality, misspecified model)
• Assumption 4, Theorem 5.1 (Risk consistency)
Key Points in the Proof
• Objective
• Two parts
• Tools
Proof by construction Solution is not unique
Outline
• Proposition 2.1, 2.2 (Subgradient, linear dependent)
• Definition 2.4-2.7 (Properties to be established)
• Theorem 3.1 (Variable Selection Consistency)
• Lemma 4.1 (Technical lemma)
• Assumption 1, Theorem 4.3 (Consistency, linear model)
• Assumption 2. Theorem 4.5 (Inequality, misspecified model)
• Assumption 4, Theorem 5.1 (Risk consistency)
Outline
• Proposition 2.1, 2.2 (Subgradient, linear dependent)
• Definition 2.4-2.7 (Properties to be established)
• Theorem 3.1 (Variable Selection Consistency)
• Lemma 4.1 (Technical lemma)
• Assumption 1, Theorem 4.3 (Consistency, linear model)
• Assumption 2. Theorem 4.5 (Inequality, misspecified model)
• Assumption 4, Theorem 5.1 (Risk consistency)