Efficient and Accurate Estimation of Lipschitz …mahyarfa/files/Slides...E cient and Accurate...

Efficient and Accurate Estimation ofLipschitz Constants for Deep Neural Networks

Mahyar Fazlyab, Alexander RobeyHamed Hassani, Manfred Morari, George J. Pappas

NeurIPS 2019

Lipschitz Constant of Neural Networks

I A function f : Rnx → Rny is globally Lipschitz continuous on X ⊆ Rnx if

‖f(x)− f(y)‖ ≤ L‖x− y‖ ∀x, y ∈ X

I The smallest such L is called the Lipschitz constant.

L = supx,y∈X

‖f(x)− f(y)‖‖x− y‖ .

I A tight upper bound on L can be useful in several applications

– Robustness certification of classifiers– Closed-loop stability analysis of neural network controllers– Robust training– Generalization bounds

Lipschitz Constant as a Measure of Robustness

Linear Linear Linear f(x)<latexit sha1_base64="k7ufXGKAt77G/FaPA5tscWX9PSc=">AAAB7XicbVA9TwJBEJ3DL8Qv1NJmI5hgQ+6w0MKCxMYSE/lI4EL2lj1Y2du97O4ZyYX/YGOhMbb+Hzv/jQtcoeBLJnl5byYz84KYM21c99vJra1vbG7ltws7u3v7B8XDo5aWiSK0SSSXqhNgTTkTtGmY4bQTK4qjgNN2ML6Z+e1HqjST4t5MYupHeChYyAg2VmqVw8rTeblfLLlVdw60SryMlCBDo1/86g0kSSIqDOFY667nxsZPsTKMcDot9BJNY0zGeEi7lgocUe2n82un6MwqAxRKZUsYNFd/T6Q40noSBbYzwmakl72Z+J/XTUx45adMxImhgiwWhQlHRqLZ62jAFCWGTyzBRDF7KyIjrDAxNqCCDcFbfnmVtGpV76Lq3dVK9essjjycwClUwINLqMMtNKAJBB7gGV7hzZHOi/PufCxac042cwx/4Hz+ACPVjig=</latexit>

f(x?)<latexit sha1_base64="j3AFkUraq/AuU1MJs5rROJBN+MY=">AAAB83icbZBNT8JAEIan+IX4hXr00ggmeCEtFz2SePGIiXwktJLtsoUN222zOzWShr/hxYPGePXPePPfuEAPCr7JJk/emcnMvkEiuEbH+bYKG5tb2zvF3dLe/sHhUfn4pKPjVFHWprGIVS8gmgkuWRs5CtZLFCNRIFg3mNzM691HpjSP5T1OE+ZHZCR5yClBY3nVsPb04Gkk6rI6KFecurOQvQ5uDhXI1RqUv7xhTNOISaSCaN13nQT9jCjkVLBZyUs1SwidkBHrG5QkYtrPFjfP7AvjDO0wVuZJtBfu74mMRFpPo8B0RgTHerU2N/+r9VMMr/2MyyRFJulyUZgKG2N7HoA95IpRFFMDhCpubrXpmChC0cRUMiG4q19eh06j7hq+a1SazTyOIpzBOdTAhStowi20oA0UEniGV3izUuvFerc+lq0FK585hT+yPn8Ay5mQ2Q==</latexit><latexit sha1_base64="j3AFkUraq/AuU1MJs5rROJBN+MY=">AAAB83icbZBNT8JAEIan+IX4hXr00ggmeCEtFz2SePGIiXwktJLtsoUN222zOzWShr/hxYPGePXPePPfuEAPCr7JJk/emcnMvkEiuEbH+bYKG5tb2zvF3dLe/sHhUfn4pKPjVFHWprGIVS8gmgkuWRs5CtZLFCNRIFg3mNzM691HpjSP5T1OE+ZHZCR5yClBY3nVsPb04Gkk6rI6KFecurOQvQ5uDhXI1RqUv7xhTNOISaSCaN13nQT9jCjkVLBZyUs1SwidkBHrG5QkYtrPFjfP7AvjDO0wVuZJtBfu74mMRFpPo8B0RgTHerU2N/+r9VMMr/2MyyRFJulyUZgKG2N7HoA95IpRFFMDhCpubrXpmChC0cRUMiG4q19eh06j7hq+a1SazTyOIpzBOdTAhStowi20oA0UEniGV3izUuvFerc+lq0FK585hT+yPn8Ay5mQ2Q==</latexit><latexit sha1_base64="j3AFkUraq/AuU1MJs5rROJBN+MY=">AAAB83icbZBNT8JAEIan+IX4hXr00ggmeCEtFz2SePGIiXwktJLtsoUN222zOzWShr/hxYPGePXPePPfuEAPCr7JJk/emcnMvkEiuEbH+bYKG5tb2zvF3dLe/sHhUfn4pKPjVFHWprGIVS8gmgkuWRs5CtZLFCNRIFg3mNzM691HpjSP5T1OE+ZHZCR5yClBY3nVsPb04Gkk6rI6KFecurOQvQ5uDhXI1RqUv7xhTNOISaSCaN13nQT9jCjkVLBZyUs1SwidkBHrG5QkYtrPFjfP7AvjDO0wVuZJtBfu74mMRFpPo8B0RgTHerU2N/+r9VMMr/2MyyRFJulyUZgKG2N7HoA95IpRFFMDhCpubrXpmChC0cRUMiG4q19eh06j7hq+a1SazTyOIpzBOdTAhStowi20oA0UEniGV3izUuvFerc+lq0FK585hT+yPn8Ay5mQ2Q==</latexit><latexit sha1_base64="j3AFkUraq/AuU1MJs5rROJBN+MY=">AAAB83icbZBNT8JAEIan+IX4hXr00ggmeCEtFz2SePGIiXwktJLtsoUN222zOzWShr/hxYPGePXPePPfuEAPCr7JJk/emcnMvkEiuEbH+bYKG5tb2zvF3dLe/sHhUfn4pKPjVFHWprGIVS8gmgkuWRs5CtZLFCNRIFg3mNzM691HpjSP5T1OE+ZHZCR5yClBY3nVsPb04Gkk6rI6KFecurOQvQ5uDhXI1RqUv7xhTNOISaSCaN13nQT9jCjkVLBZyUs1SwidkBHrG5QkYtrPFjfP7AvjDO0wVuZJtBfu74mMRFpPo8B0RgTHerU2N/+r9VMMr/2MyyRFJulyUZgKG2N7HoA95IpRFFMDhCpubrXpmChC0cRUMiG4q19eh06j7hq+a1SazTyOIpzBOdTAhStowi20oA0UEniGV3izUuvFerc+lq0FK585hT+yPn8Ay5mQ2Q==</latexit>x?

<latexit sha1_base64="1HSaP89i6ZNuxL6ngZHnK0kxsVM=">AAAB8HicbZA9T8MwEIYvfJbyVWBkiWiRmKqkC4yVWBiLRD9QGyrHdVqrthPZF0QV9VewMIAQKz+HjX+D22aAlley9Oi9O/nuDRPBDXret7O2vrG5tV3YKe7u7R8clo6OWyZONWVNGotYd0JimOCKNZGjYJ1EMyJDwdrh+HpWbz8ybXis7nCSsECSoeIRpwStdV95eugZJLrSL5W9qjeXuwp+DmXI1eiXvnqDmKaSKaSCGNP1vQSDjGjkVLBpsZcalhA6JkPWtaiIZCbI5gtP3XPrDNwo1vYpdOfu74mMSGMmMrSdkuDILNdm5n+1borRVZBxlaTIFF18FKXCxdidXe8OuGYUxcQCoZrbXV06IppQtBkVbQj+8smr0KpVfcu3tXK9nsdRgFM4gwvw4RLqcAMNaAIFCc/wCm+Odl6cd+dj0brm5DMn8EfO5w8+Z5AE</latexit><latexit sha1_base64="1HSaP89i6ZNuxL6ngZHnK0kxsVM=">AAAB8HicbZA9T8MwEIYvfJbyVWBkiWiRmKqkC4yVWBiLRD9QGyrHdVqrthPZF0QV9VewMIAQKz+HjX+D22aAlley9Oi9O/nuDRPBDXret7O2vrG5tV3YKe7u7R8clo6OWyZONWVNGotYd0JimOCKNZGjYJ1EMyJDwdrh+HpWbz8ybXis7nCSsECSoeIRpwStdV95eugZJLrSL5W9qjeXuwp+DmXI1eiXvnqDmKaSKaSCGNP1vQSDjGjkVLBpsZcalhA6JkPWtaiIZCbI5gtP3XPrDNwo1vYpdOfu74mMSGMmMrSdkuDILNdm5n+1borRVZBxlaTIFF18FKXCxdidXe8OuGYUxcQCoZrbXV06IppQtBkVbQj+8smr0KpVfcu3tXK9nsdRgFM4gwvw4RLqcAMNaAIFCc/wCm+Odl6cd+dj0brm5DMn8EfO5w8+Z5AE</latexit><latexit sha1_base64="1HSaP89i6ZNuxL6ngZHnK0kxsVM=">AAAB8HicbZA9T8MwEIYvfJbyVWBkiWiRmKqkC4yVWBiLRD9QGyrHdVqrthPZF0QV9VewMIAQKz+HjX+D22aAlley9Oi9O/nuDRPBDXret7O2vrG5tV3YKe7u7R8clo6OWyZONWVNGotYd0JimOCKNZGjYJ1EMyJDwdrh+HpWbz8ybXis7nCSsECSoeIRpwStdV95eugZJLrSL5W9qjeXuwp+DmXI1eiXvnqDmKaSKaSCGNP1vQSDjGjkVLBpsZcalhA6JkPWtaiIZCbI5gtP3XPrDNwo1vYpdOfu74mMSGMmMrSdkuDILNdm5n+1borRVZBxlaTIFF18FKXCxdidXe8OuGYUxcQCoZrbXV06IppQtBkVbQj+8smr0KpVfcu3tXK9nsdRgFM4gwvw4RLqcAMNaAIFCc/wCm+Odl6cd+dj0brm5DMn8EfO5w8+Z5AE</latexit><latexit sha1_base64="1HSaP89i6ZNuxL6ngZHnK0kxsVM=">AAAB8HicbZA9T8MwEIYvfJbyVWBkiWiRmKqkC4yVWBiLRD9QGyrHdVqrthPZF0QV9VewMIAQKz+HjX+D22aAlley9Oi9O/nuDRPBDXret7O2vrG5tV3YKe7u7R8clo6OWyZONWVNGotYd0JimOCKNZGjYJ1EMyJDwdrh+HpWbz8ybXis7nCSsECSoeIRpwStdV95eugZJLrSL5W9qjeXuwp+DmXI1eiXvnqDmKaSKaSCGNP1vQSDjGjkVLBpsZcalhA6JkPWtaiIZCbI5gtP3XPrDNwo1vYpdOfu74mMSGMmMrSdkuDILNdm5n+1borRVZBxlaTIFF18FKXCxdidXe8OuGYUxcQCoZrbXV06IppQtBkVbQj+8smr0KpVfcu3tXK9nsdRgFM4gwvw4RLqcAMNaAIFCc/wCm+Odl6cd+dj0brm5DMn8EfO5w8+Z5AE</latexit>

kx � x?k2 ✏<latexit sha1_base64="DoHRRz7mM2yqCp0yK1NCedRVspQ=">AAACCHicbZC7TsMwFIadcivlFmBkwKJFYqFKusBYiYWxSPQiNSVy3NPWquME20Gt0o4svAoLAwix8ghsvA3uZYCWX7L06T/n6Pj8QcyZ0o7zbWVWVtfWN7Kbua3tnd09e/+gpqJEUqjSiEeyERAFnAmoaqY5NGIJJAw41IP+1aRefwCpWCRu9TCGVki6gnUYJdpYvn1c8EaD88GdpzSR3sgvYY/DPfYgVoxHouDbeafoTIWXwZ1DHs1V8e0vrx3RJAShKSdKNV0n1q2USM0oh3HOSxTEhPZJF5oGBQlBtdLpIWN8apw27kTSPKHx1P09kZJQqWEYmM6Q6J5arE3M/2rNRHcuWykTcaJB0NmiTsKxjvAkFdxmEqjmQwOESmb+immPSEK1yS5nQnAXT16GWqnoGr4p5cvleRxZdIRO0Bly0QUqo2tUQVVE0SN6Rq/ozXqyXqx362PWmrHmM4foj6zPH8u1mSs=</latexit><latexit sha1_base64="DoHRRz7mM2yqCp0yK1NCedRVspQ=">AAACCHicbZC7TsMwFIadcivlFmBkwKJFYqFKusBYiYWxSPQiNSVy3NPWquME20Gt0o4svAoLAwix8ghsvA3uZYCWX7L06T/n6Pj8QcyZ0o7zbWVWVtfWN7Kbua3tnd09e/+gpqJEUqjSiEeyERAFnAmoaqY5NGIJJAw41IP+1aRefwCpWCRu9TCGVki6gnUYJdpYvn1c8EaD88GdpzSR3sgvYY/DPfYgVoxHouDbeafoTIWXwZ1DHs1V8e0vrx3RJAShKSdKNV0n1q2USM0oh3HOSxTEhPZJF5oGBQlBtdLpIWN8apw27kTSPKHx1P09kZJQqWEYmM6Q6J5arE3M/2rNRHcuWykTcaJB0NmiTsKxjvAkFdxmEqjmQwOESmb+immPSEK1yS5nQnAXT16GWqnoGr4p5cvleRxZdIRO0Bly0QUqo2tUQVVE0SN6Rq/ozXqyXqx362PWmrHmM4foj6zPH8u1mSs=</latexit><latexit sha1_base64="DoHRRz7mM2yqCp0yK1NCedRVspQ=">AAACCHicbZC7TsMwFIadcivlFmBkwKJFYqFKusBYiYWxSPQiNSVy3NPWquME20Gt0o4svAoLAwix8ghsvA3uZYCWX7L06T/n6Pj8QcyZ0o7zbWVWVtfWN7Kbua3tnd09e/+gpqJEUqjSiEeyERAFnAmoaqY5NGIJJAw41IP+1aRefwCpWCRu9TCGVki6gnUYJdpYvn1c8EaD88GdpzSR3sgvYY/DPfYgVoxHouDbeafoTIWXwZ1DHs1V8e0vrx3RJAShKSdKNV0n1q2USM0oh3HOSxTEhPZJF5oGBQlBtdLpIWN8apw27kTSPKHx1P09kZJQqWEYmM6Q6J5arE3M/2rNRHcuWykTcaJB0NmiTsKxjvAkFdxmEqjmQwOESmb+immPSEK1yS5nQnAXT16GWqnoGr4p5cvleRxZdIRO0Bly0QUqo2tUQVVE0SN6Rq/ozXqyXqx362PWmrHmM4foj6zPH8u1mSs=</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="p3GAkxepYK0/hd8vlF9v81bI25M=">AAAB/XicbZBLTwIxFIXv4AsRFd26sBFM3Ehm2OjSxI1LTOSRMEg65QINnc7YdgwEWLrxr7hxoTH+DXf+G8tjoeBJmnw5p83tPUEsuDau++2k1tY3NrfS25md7O7efu4gW9VRohhWWCQiVQ+oRsElVgw3AuuxQhoGAmtB/3qa1x5RaR7JOzOMsRnSruQdzqixVit3XPDHg/PBva8NVf64VSK+wAfiY6y5iGShlcu7RXcmsgreAvKwULmV+/LbEUtClIYJqnXDc2PTHFFlOBM4yfiJxpiyPu1iw6KkIermaLbIhJxap006kbJHGjJzf78Y0VDrYRjYmyE1Pb2cTc3/skZiOpfNEZdxYlCy+aBOIoiJyLQV0uYKmRFDC5Qpbv9KWI8qyoztLmNL8JZXXoVqqehZvnUhDUdwAmfgwQVcwQ2UoQIMnuAF3uDdeXZenY95XSln0dsh/JHz+QM32Je8</latexit><latexit sha1_base64="p3GAkxepYK0/hd8vlF9v81bI25M=">AAAB/XicbZBLTwIxFIXv4AsRFd26sBFM3Ehm2OjSxI1LTOSRMEg65QINnc7YdgwEWLrxr7hxoTH+DXf+G8tjoeBJmnw5p83tPUEsuDau++2k1tY3NrfS25md7O7efu4gW9VRohhWWCQiVQ+oRsElVgw3AuuxQhoGAmtB/3qa1x5RaR7JOzOMsRnSruQdzqixVit3XPDHg/PBva8NVf64VSK+wAfiY6y5iGShlcu7RXcmsgreAvKwULmV+/LbEUtClIYJqnXDc2PTHFFlOBM4yfiJxpiyPu1iw6KkIermaLbIhJxap006kbJHGjJzf78Y0VDrYRjYmyE1Pb2cTc3/skZiOpfNEZdxYlCy+aBOIoiJyLQV0uYKmRFDC5Qpbv9KWI8qyoztLmNL8JZXXoVqqehZvnUhDUdwAmfgwQVcwQ2UoQIMnuAF3uDdeXZenY95XSln0dsh/JHz+QM32Je8</latexit><latexit sha1_base64="PsaHGvPWV2KMb4TyqF9oOMdYXqg=">AAACCHicbZC7TsMwFIadcivlFmBkwKJFYqFKusBYiYWxSPQiNSVy3NPWquME20Gt0o4svAoLAwix8ghsvA3uZYCWX7L06T/n6Pj8QcyZ0o7zbWVWVtfWN7Kbua3tnd09e/+gpqJEUqjSiEeyERAFnAmoaqY5NGIJJAw41IP+1aRefwCpWCRu9TCGVki6gnUYJdpYvn1c8EaD88GdpzSR3sgvYY/DPfYgVoxHouDbeafoTIWXwZ1DHs1V8e0vrx3RJAShKSdKNV0n1q2USM0oh3HOSxTEhPZJF5oGBQlBtdLpIWN8apw27kTSPKHx1P09kZJQqWEYmM6Q6J5arE3M/2rNRHcuWykTcaJB0NmiTsKxjvAkFdxmEqjmQwOESmb+immPSEK1yS5nQnAXT16GWqnoGr5x8uXyPI4sOkIn6Ay56AKV0TWqoCqi6BE9o1f0Zj1ZL9a79TFrzVjzmUP0R9bnD8sVmSk=</latexit><latexit sha1_base64="DoHRRz7mM2yqCp0yK1NCedRVspQ=">AAACCHicbZC7TsMwFIadcivlFmBkwKJFYqFKusBYiYWxSPQiNSVy3NPWquME20Gt0o4svAoLAwix8ghsvA3uZYCWX7L06T/n6Pj8QcyZ0o7zbWVWVtfWN7Kbua3tnd09e/+gpqJEUqjSiEeyERAFnAmoaqY5NGIJJAw41IP+1aRefwCpWCRu9TCGVki6gnUYJdpYvn1c8EaD88GdpzSR3sgvYY/DPfYgVoxHouDbeafoTIWXwZ1DHs1V8e0vrx3RJAShKSdKNV0n1q2USM0oh3HOSxTEhPZJF5oGBQlBtdLpIWN8apw27kTSPKHx1P09kZJQqWEYmM6Q6J5arE3M/2rNRHcuWykTcaJB0NmiTsKxjvAkFdxmEqjmQwOESmb+immPSEK1yS5nQnAXT16GWqnoGr4p5cvleRxZdIRO0Bly0QUqo2tUQVVE0SN6Rq/ozXqyXqx362PWmrHmM4foj6zPH8u1mSs=</latexit><latexit sha1_base64="DoHRRz7mM2yqCp0yK1NCedRVspQ=">AAACCHicbZC7TsMwFIadcivlFmBkwKJFYqFKusBYiYWxSPQiNSVy3NPWquME20Gt0o4svAoLAwix8ghsvA3uZYCWX7L06T/n6Pj8QcyZ0o7zbWVWVtfWN7Kbua3tnd09e/+gpqJEUqjSiEeyERAFnAmoaqY5NGIJJAw41IP+1aRefwCpWCRu9TCGVki6gnUYJdpYvn1c8EaD88GdpzSR3sgvYY/DPfYgVoxHouDbeafoTIWXwZ1DHs1V8e0vrx3RJAShKSdKNV0n1q2USM0oh3HOSxTEhPZJF5oGBQlBtdLpIWN8apw27kTSPKHx1P09kZJQqWEYmM6Q6J5arE3M/2rNRHcuWykTcaJB0NmiTsKxjvAkFdxmEqjmQwOESmb+immPSEK1yS5nQnAXT16GWqnoGr4p5cvleRxZdIRO0Bly0QUqo2tUQVVE0SN6Rq/ozXqyXqx362PWmrHmM4foj6zPH8u1mSs=</latexit><latexit sha1_base64="DoHRRz7mM2yqCp0yK1NCedRVspQ=">AAACCHicbZC7TsMwFIadcivlFmBkwKJFYqFKusBYiYWxSPQiNSVy3NPWquME20Gt0o4svAoLAwix8ghsvA3uZYCWX7L06T/n6Pj8QcyZ0o7zbWVWVtfWN7Kbua3tnd09e/+gpqJEUqjSiEeyERAFnAmoaqY5NGIJJAw41IP+1aRefwCpWCRu9TCGVki6gnUYJdpYvn1c8EaD88GdpzSR3sgvYY/DPfYgVoxHouDbeafoTIWXwZ1DHs1V8e0vrx3RJAShKSdKNV0n1q2USM0oh3HOSxTEhPZJF5oGBQlBtdLpIWN8apw27kTSPKHx1P09kZJQqWEYmM6Q6J5arE3M/2rNRHcuWykTcaJB0NmiTsKxjvAkFdxmEqjmQwOESmb+immPSEK1yS5nQnAXT16GWqnoGr4p5cvleRxZdIRO0Bly0QUqo2tUQVVE0SN6Rq/ozXqyXqx362PWmrHmM4foj6zPH8u1mSs=</latexit><latexit sha1_base64="DoHRRz7mM2yqCp0yK1NCedRVspQ=">AAACCHicbZC7TsMwFIadcivlFmBkwKJFYqFKusBYiYWxSPQiNSVy3NPWquME20Gt0o4svAoLAwix8ghsvA3uZYCWX7L06T/n6Pj8QcyZ0o7zbWVWVtfWN7Kbua3tnd09e/+gpqJEUqjSiEeyERAFnAmoaqY5NGIJJAw41IP+1aRefwCpWCRu9TCGVki6gnUYJdpYvn1c8EaD88GdpzSR3sgvYY/DPfYgVoxHouDbeafoTIWXwZ1DHs1V8e0vrx3RJAShKSdKNV0n1q2USM0oh3HOSxTEhPZJF5oGBQlBtdLpIWN8apw27kTSPKHx1P09kZJQqWEYmM6Q6J5arE3M/2rNRHcuWykTcaJB0NmiTsKxjvAkFdxmEqjmQwOESmb+immPSEK1yS5nQnAXT16GWqnoGr4p5cvleRxZdIRO0Bly0QUqo2tUQVVE0SN6Rq/ozXqyXqx362PWmrHmM4foj6zPH8u1mSs=</latexit><latexit sha1_base64="DoHRRz7mM2yqCp0yK1NCedRVspQ=">AAACCHicbZC7TsMwFIadcivlFmBkwKJFYqFKusBYiYWxSPQiNSVy3NPWquME20Gt0o4svAoLAwix8ghsvA3uZYCWX7L06T/n6Pj8QcyZ0o7zbWVWVtfWN7Kbua3tnd09e/+gpqJEUqjSiEeyERAFnAmoaqY5NGIJJAw41IP+1aRefwCpWCRu9TCGVki6gnUYJdpYvn1c8EaD88GdpzSR3sgvYY/DPfYgVoxHouDbeafoTIWXwZ1DHs1V8e0vrx3RJAShKSdKNV0n1q2USM0oh3HOSxTEhPZJF5oGBQlBtdLpIWN8apw27kTSPKHx1P09kZJQqWEYmM6Q6J5arE3M/2rNRHcuWykTcaJB0NmiTsKxjvAkFdxmEqjmQwOESmb+immPSEK1yS5nQnAXT16GWqnoGr4p5cvleRxZdIRO0Bly0QUqo2tUQVVE0SN6Rq/ozXqyXqx362PWmrHmM4foj6zPH8u1mSs=</latexit><latexit sha1_base64="DoHRRz7mM2yqCp0yK1NCedRVspQ=">AAACCHicbZC7TsMwFIadcivlFmBkwKJFYqFKusBYiYWxSPQiNSVy3NPWquME20Gt0o4svAoLAwix8ghsvA3uZYCWX7L06T/n6Pj8QcyZ0o7zbWVWVtfWN7Kbua3tnd09e/+gpqJEUqjSiEeyERAFnAmoaqY5NGIJJAw41IP+1aRefwCpWCRu9TCGVki6gnUYJdpYvn1c8EaD88GdpzSR3sgvYY/DPfYgVoxHouDbeafoTIWXwZ1DHs1V8e0vrx3RJAShKSdKNV0n1q2USM0oh3HOSxTEhPZJF5oGBQlBtdLpIWN8apw27kTSPKHx1P09kZJQqWEYmM6Q6J5arE3M/2rNRHcuWykTcaJB0NmiTsKxjvAkFdxmEqjmQwOESmb+immPSEK1yS5nQnAXT16GWqnoGr4p5cvleRxZdIRO0Bly0QUqo2tUQVVE0SN6Rq/ozXqyXqx362PWmrHmM4foj6zPH8u1mSs=</latexit> kf(x) � f(x?)k2 �

<latexit sha1_base64="kcW+Rnzd5H2LQ0zdmiQW/3R8W1g=">AAACDHicbVDLTgIxFO34RHyhLt00gokuJDO40CWJG5eYyCNhkHTKHWjsPGzvGAnwAW78FTcuNMatH+DOv7HALBQ8SZuTc85Ne48XS6HRtr+thcWl5ZXVzFp2fWNzazu3s1vTUaI4VHkkI9XwmAYpQqiiQAmNWAELPAl17/Zi7NfvQWkRhdfYj6EVsG4ofMEZGqmdyxfcoX/0cHxirhtXI1PH7rBdoq6EO+p2QCIrmJRdtCeg88RJSZ6kqLRzX24n4kkAIXLJtG46doytAVMouIRR1k00xIzfsi40DQ1ZALo1mCwzoodG6VA/UuaESCfq74kBC7TuB55JBgx7etYbi/95zQT989ZAhHGCEPLpQ34iKUZ03AztCAUcZd8QxpUwf6W8xxTjaPrLmhKc2ZXnSa1UdE6LzlUpXy6ndWTIPjkgR8QhZ6RMLkmFVAknj+SZvJI368l6sd6tj2l0wUpn9sgfWJ8/PGqZ1A==</latexit>

x<latexit sha1_base64="BmVm2kBoL99/m27NxkUbvUDufsI=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORksSG0uM8pHAhewte7Bhb++yO2ckF36CjYXG2PqL7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41Yy3WCxj3Q2o4VIo3kKBkncTzWkUSN4JJjfzeueRayNi9YDThPsRHSkRCkbRWvfVp+qgXHFr7kJkHbwcKpCrOSh/9YcxSyOukElqTM9zE/QzqlEwyWelfmp4QtmEjnjPoqIRN362WHVGLqwzJGGs7VNIFu7viYxGxkyjwHZGFMdmtTY3/6v1Ugyv/UyoJEWu2PKjMJUEYzK/mwyF5gzl1AJlWthdCRtTTRnadEo2BG/15HVo12ue5bt6pdHI4yjCGZzDJXhwBQ24hSa0gMEInuEV3hzpvDjvzseyteDkM6fwR87nD5y6jVQ=</latexit><latexit sha1_base64="BmVm2kBoL99/m27NxkUbvUDufsI=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORksSG0uM8pHAhewte7Bhb++yO2ckF36CjYXG2PqL7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41Yy3WCxj3Q2o4VIo3kKBkncTzWkUSN4JJjfzeueRayNi9YDThPsRHSkRCkbRWvfVp+qgXHFr7kJkHbwcKpCrOSh/9YcxSyOukElqTM9zE/QzqlEwyWelfmp4QtmEjnjPoqIRN362WHVGLqwzJGGs7VNIFu7viYxGxkyjwHZGFMdmtTY3/6v1Ugyv/UyoJEWu2PKjMJUEYzK/mwyF5gzl1AJlWthdCRtTTRnadEo2BG/15HVo12ue5bt6pdHI4yjCGZzDJXhwBQ24hSa0gMEInuEV3hzpvDjvzseyteDkM6fwR87nD5y6jVQ=</latexit><latexit sha1_base64="BmVm2kBoL99/m27NxkUbvUDufsI=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORksSG0uM8pHAhewte7Bhb++yO2ckF36CjYXG2PqL7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41Yy3WCxj3Q2o4VIo3kKBkncTzWkUSN4JJjfzeueRayNi9YDThPsRHSkRCkbRWvfVp+qgXHFr7kJkHbwcKpCrOSh/9YcxSyOukElqTM9zE/QzqlEwyWelfmp4QtmEjnjPoqIRN362WHVGLqwzJGGs7VNIFu7viYxGxkyjwHZGFMdmtTY3/6v1Ugyv/UyoJEWu2PKjMJUEYzK/mwyF5gzl1AJlWthdCRtTTRnadEo2BG/15HVo12ue5bt6pdHI4yjCGZzDJXhwBQ24hSa0gMEInuEV3hzpvDjvzseyteDkM6fwR87nD5y6jVQ=</latexit><latexit sha1_base64="BmVm2kBoL99/m27NxkUbvUDufsI=">AAAB6nicbZA9TwJBEIbn8AvxC7W02QgmVuSORksSG0uM8pHAhewte7Bhb++yO2ckF36CjYXG2PqL7Pw3LnCFgm+yyZN3ZrIzb5BIYdB1v53CxubW9k5xt7S3f3B4VD4+aZs41Yy3WCxj3Q2o4VIo3kKBkncTzWkUSN4JJjfzeueRayNi9YDThPsRHSkRCkbRWvfVp+qgXHFr7kJkHbwcKpCrOSh/9YcxSyOukElqTM9zE/QzqlEwyWelfmp4QtmEjnjPoqIRN362WHVGLqwzJGGs7VNIFu7viYxGxkyjwHZGFMdmtTY3/6v1Ugyv/UyoJEWu2PKjMJUEYzK/mwyF5gzl1AJlWthdCRtTTRnadEo2BG/15HVo12ue5bt6pdHI4yjCGZzDJXhwBQ24hSa0gMEInuEV3hzpvDjvzseyteDkM6fwR87nD5y6jVQ=</latexit>

kf(x) � f(x?)k2 L2kx � x?k2<latexit sha1_base64="TZXeUxHiiPOPf+dFoMuZUOCv/+8=">AAACGXicbZC7TsMwFIYdrqXcAowsFi1SO7RKygBjJRYGhiLRi9SEyHGd1qpzwXZQq7SvwcKrsDCAECNMvA1OmwFajmTr0/+fI/v8bsSokIbxra2srq1vbOa28ts7u3v7+sFhS4Qxx6SJQxbyjosEYTQgTUklI52IE+S7jLTd4WXqtx8IFzQMbuU4IraP+gH1KEZSSY5uFK2JVxqVK+q6s4REvGxNnBq0GLmH1ylMRpXMSY2ioxeMqjEruAxmBgWQVcPRP61eiGOfBBIzJETXNCJpJ4hLihmZ5q1YkAjhIeqTrsIA+UTYyWyzKTxVSg96IVcnkHCm/p5IkC/E2HdVp4/kQCx6qfif142ld2EnNIhiSQI8f8iLGZQhTGOCPcoJlmysAGFO1V8hHiCOsFRh5lUI5uLKy9CqVc2zqnlTK9TrWRw5cAxOQAmY4BzUwRVogCbA4BE8g1fwpj1pL9q79jFvXdGymSPwp7SvH4KVnr8=</latexit>

activations activations

✏<latexit sha1_base64="i4udT/drOOO6oO7noIEGU/CaHW4=">AAAB8XicbVA9TwJBEJ3DL8Qv1NJmI5hYkTsspCSxscREwAgXsrfMwYa9vcvungm58C9sLDTG1n9j579xgSsUfMkkL+/NZGZekAiujet+O4WNza3tneJuaW//4PCofHzS0XGqGLZZLGL1EFCNgktsG24EPiQKaRQI7AaTm7nffUKleSzvzTRBP6IjyUPOqLHSY7WPieYiltVBueLW3AXIOvFyUoEcrUH5qz+MWRqhNExQrXuemxg/o8pwJnBW6qcaE8omdIQ9SyWNUPvZ4uIZubDKkISxsiUNWai/JzIaaT2NAtsZUTPWq95c/M/rpSZs+BmXSWpQsuWiMBXExGT+PhlyhcyIqSWUKW5vJWxMFWXGhlSyIXirL6+TTr3mXdW8u3ql2cjjKMIZnMMleHANTbiFFrSBgYRneIU3RzsvzrvzsWwtOPnMKfyB8/kDBdqQcw==</latexit> �

<latexit sha1_base64="7vRx4A6LElEKTYMBacNXemCuKD8=">AAAB73icbVA9SwNBEJ3zM8avqKXNYiJYhbtYaCUBG8sI5gOSI+xt5pIle3vn7p4QjvwJGwtFbP07dv4bN8kVmvhg4PHeDDPzgkRwbVz321lb39jc2i7sFHf39g8OS0fHLR2nimGTxSJWnYBqFFxi03AjsJMopFEgsB2Mb2d++wmV5rF8MJME/YgOJQ85o8ZKnUpvgMLQSr9UdqvuHGSVeDkpQ45Gv/TVG8QsjVAaJqjWXc9NjJ9RZTgTOC32Uo0JZWM6xK6lkkao/Wx+75ScW2VAwljZkobM1d8TGY20nkSB7YyoGellbyb+53VTE177GZdJalCyxaIwFcTEZPY8GXCFzIiJJZQpbm8lbEQVZcZGVLQheMsvr5JWrepdVr37Wrl+k8dRgFM4gwvw4ArqcAcNaAIDAc/wCm/Oo/PivDsfi9Y1J585gT9wPn8ASyOPdQ==</latexit>

Proposition 1:

The inequality

L2 ≤1

ε√2

min1≤j≤k, j 6=i?

|fj(x?)− fi?(x?)|,

implies C(x) := argmax1≤i≤kfi(x) = C(x?) for all ‖x− x?‖2 ≤ ε.

• A global measure of robustness 3

Incremental Quadratic Constraints

Definition 1: Incremental Quadratic Constraint [Acıkmese and Corless, 2011]

A function φ : Rn → Rn satisfies the incremental quadratic constraint defined byQ ⊂ S2nif for any Q ∈ Q and x, y ∈ Rn, φ satisfies

[x− y

φ(x)− φ(y)

[x− y

φ(x)− φ(y)

]≥ 0.

I Lipschitz continuity (in `2 norm) is a special case

[x− y

φ(x)− φ(y)

]> [L2In 00 −In

] [x− y

φ(x)− φ(y)

]≥ 0.

Incremental QC for Activation Functions

Lemma 1: Incremental QC for activation functions

Suppose ϕ is slope-restricted on [α, β]. Then φ(x) = [ϕ(x1) · · ·ϕ(xn)]> satisfies

[x− y

φ(x)− φ(y)

]> [ −2αβT (α+ β)T(α+ β)T −2T

] [x− y

φ(x)− φ(y)

]≥ 0,

for all x, y ∈ Rn and all T ∈ Tn, where

Tn = {T ∈ Sn | T =

λiieie>i +

λij(ei−ej)(ei−ej)>, λij ≥ 0}.

Visual “Proof”

-5 0 5-1

(xj ,'(xj))<latexit sha1_base64="7KVgF9Hepxz+AlzTMESMIJp61E0=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQUpSTe6LLhxWcFeoA1hMp20YyeTMDMp1lJ8FTcuFHHre7jzbZy0WWjrDwMf/zmHc+b3Y0alsu1vI7e2vrG5ld8u7Ozu7R+Yh0ctGSUCkyaOWCQ6PpKEUU6aiipGOrEgKPQZafuj67TeHhMhacTv1CQmbogGnAYUI6UtzzwplR+8+4veGIl4SFOuVEqeWbSr9lzWKjgZFCFTwzO/ev0IJyHhCjMkZdexY+VOkVAUMzIr9BJJYoRHaEC6GjkKiXSn8+tn1rl2+lYQCf24subu74kpCqWchL7uDJEayuVaav5X6yYquHKnlMeJIhwvFgUJs1RkpVFYfSoIVmyiAWFB9a0WHiKBsNKBFXQIzvKXV6FVqzqab2vFej2LIw+ncAZlcOAS6nADDWgChkd4hld4M56MF+Pd+Fi05oxs5hj+yPj8AXzPk+0=</latexit><latexit sha1_base64="7KVgF9Hepxz+AlzTMESMIJp61E0=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQUpSTe6LLhxWcFeoA1hMp20YyeTMDMp1lJ8FTcuFHHre7jzbZy0WWjrDwMf/zmHc+b3Y0alsu1vI7e2vrG5ld8u7Ozu7R+Yh0ctGSUCkyaOWCQ6PpKEUU6aiipGOrEgKPQZafuj67TeHhMhacTv1CQmbogGnAYUI6UtzzwplR+8+4veGIl4SFOuVEqeWbSr9lzWKjgZFCFTwzO/ev0IJyHhCjMkZdexY+VOkVAUMzIr9BJJYoRHaEC6GjkKiXSn8+tn1rl2+lYQCf24subu74kpCqWchL7uDJEayuVaav5X6yYquHKnlMeJIhwvFgUJs1RkpVFYfSoIVmyiAWFB9a0WHiKBsNKBFXQIzvKXV6FVqzqab2vFej2LIw+ncAZlcOAS6nADDWgChkd4hld4M56MF+Pd+Fi05oxs5hj+yPj8AXzPk+0=</latexit><latexit sha1_base64="7KVgF9Hepxz+AlzTMESMIJp61E0=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQUpSTe6LLhxWcFeoA1hMp20YyeTMDMp1lJ8FTcuFHHre7jzbZy0WWjrDwMf/zmHc+b3Y0alsu1vI7e2vrG5ld8u7Ozu7R+Yh0ctGSUCkyaOWCQ6PpKEUU6aiipGOrEgKPQZafuj67TeHhMhacTv1CQmbogGnAYUI6UtzzwplR+8+4veGIl4SFOuVEqeWbSr9lzWKjgZFCFTwzO/ev0IJyHhCjMkZdexY+VOkVAUMzIr9BJJYoRHaEC6GjkKiXSn8+tn1rl2+lYQCf24subu74kpCqWchL7uDJEayuVaav5X6yYquHKnlMeJIhwvFgUJs1RkpVFYfSoIVmyiAWFB9a0WHiKBsNKBFXQIzvKXV6FVqzqab2vFej2LIw+ncAZlcOAS6nADDWgChkd4hld4M56MF+Pd+Fi05oxs5hj+yPj8AXzPk+0=</latexit><latexit sha1_base64="7KVgF9Hepxz+AlzTMESMIJp61E0=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQUpSTe6LLhxWcFeoA1hMp20YyeTMDMp1lJ8FTcuFHHre7jzbZy0WWjrDwMf/zmHc+b3Y0alsu1vI7e2vrG5ld8u7Ozu7R+Yh0ctGSUCkyaOWCQ6PpKEUU6aiipGOrEgKPQZafuj67TeHhMhacTv1CQmbogGnAYUI6UtzzwplR+8+4veGIl4SFOuVEqeWbSr9lzWKjgZFCFTwzO/ev0IJyHhCjMkZdexY+VOkVAUMzIr9BJJYoRHaEC6GjkKiXSn8+tn1rl2+lYQCf24subu74kpCqWchL7uDJEayuVaav5X6yYquHKnlMeJIhwvFgUJs1RkpVFYfSoIVmyiAWFB9a0WHiKBsNKBFXQIzvKXV6FVqzqab2vFej2LIw+ncAZlcOAS6nADDWgChkd4hld4M56MF+Pd+Fi05oxs5hj+yPj8AXzPk+0=</latexit>

(xi,'(xi))<latexit sha1_base64="GllpdvBtNW9LH6e2gGOX+rXlP0k=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdFty4rGAv0IYwmU7aoZNJmJkUayi+ihsXirj1Pdz5Nk7aLLT1h4GP/5zDOfP7MaNS2fa3UVhb39jcKm6Xdnb39g/Mw6O2jBKBSQtHLBJdH0nCKCctRRUj3VgQFPqMdPzxTVbvTIiQNOL3ahoTN0RDTgOKkdKWZ55Uqg8evehPkIhHNONareKZZfvSnstaBSeHMuRqeuZXfxDhJCRcYYak7Dl2rNwUCUUxI7NSP5EkRniMhqSnkaOQSDedXz+zzrUzsIJI6MeVNXd/T6QolHIa+rozRGokl2uZ+V+tl6jg2k0pjxNFOF4sChJmqcjKorAGVBCs2FQDwoLqWy08QgJhpQMr6RCc5S+vQrt+6Wi+q5cbjTyOIpzCGVTBgStowC00oQUYHuEZXuHNeDJejHfjY9FaMPKZY/gj4/MHebWT6w==</latexit><latexit sha1_base64="GllpdvBtNW9LH6e2gGOX+rXlP0k=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdFty4rGAv0IYwmU7aoZNJmJkUayi+ihsXirj1Pdz5Nk7aLLT1h4GP/5zDOfP7MaNS2fa3UVhb39jcKm6Xdnb39g/Mw6O2jBKBSQtHLBJdH0nCKCctRRUj3VgQFPqMdPzxTVbvTIiQNOL3ahoTN0RDTgOKkdKWZ55Uqg8evehPkIhHNONareKZZfvSnstaBSeHMuRqeuZXfxDhJCRcYYak7Dl2rNwUCUUxI7NSP5EkRniMhqSnkaOQSDedXz+zzrUzsIJI6MeVNXd/T6QolHIa+rozRGokl2uZ+V+tl6jg2k0pjxNFOF4sChJmqcjKorAGVBCs2FQDwoLqWy08QgJhpQMr6RCc5S+vQrt+6Wi+q5cbjTyOIpzCGVTBgStowC00oQUYHuEZXuHNeDJejHfjY9FaMPKZY/gj4/MHebWT6w==</latexit><latexit sha1_base64="GllpdvBtNW9LH6e2gGOX+rXlP0k=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdFty4rGAv0IYwmU7aoZNJmJkUayi+ihsXirj1Pdz5Nk7aLLT1h4GP/5zDOfP7MaNS2fa3UVhb39jcKm6Xdnb39g/Mw6O2jBKBSQtHLBJdH0nCKCctRRUj3VgQFPqMdPzxTVbvTIiQNOL3ahoTN0RDTgOKkdKWZ55Uqg8evehPkIhHNONareKZZfvSnstaBSeHMuRqeuZXfxDhJCRcYYak7Dl2rNwUCUUxI7NSP5EkRniMhqSnkaOQSDedXz+zzrUzsIJI6MeVNXd/T6QolHIa+rozRGokl2uZ+V+tl6jg2k0pjxNFOF4sChJmqcjKorAGVBCs2FQDwoLqWy08QgJhpQMr6RCc5S+vQrt+6Wi+q5cbjTyOIpzCGVTBgStowC00oQUYHuEZXuHNeDJejHfjY9FaMPKZY/gj4/MHebWT6w==</latexit><latexit sha1_base64="GllpdvBtNW9LH6e2gGOX+rXlP0k=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdFty4rGAv0IYwmU7aoZNJmJkUayi+ihsXirj1Pdz5Nk7aLLT1h4GP/5zDOfP7MaNS2fa3UVhb39jcKm6Xdnb39g/Mw6O2jBKBSQtHLBJdH0nCKCctRRUj3VgQFPqMdPzxTVbvTIiQNOL3ahoTN0RDTgOKkdKWZ55Uqg8evehPkIhHNONareKZZfvSnstaBSeHMuRqeuZXfxDhJCRcYYak7Dl2rNwUCUUxI7NSP5EkRniMhqSnkaOQSDedXz+zzrUzsIJI6MeVNXd/T6QolHIa+rozRGokl2uZ+V+tl6jg2k0pjxNFOF4sChJmqcjKorAGVBCs2FQDwoLqWy08QgJhpQMr6RCc5S+vQrt+6Wi+q5cbjTyOIpzCGVTBgStowC00oQUYHuEZXuHNeDJejHfjY9FaMPKZY/gj4/MHebWT6w==</latexit>

↵ '(xj) � '(xi)

xj � xi �

<latexit sha1_base64="bXpVOSwBhezioWcgI+XxN1XSrkk=">AAACLHicbZDLSgMxFIYzXmu9jbp0E6yCLiwz3eiy0I3LCrYVOqWcSc/Y2MzFJFMsQx/Ija8iiAuLuPU5TNsBrz8EvvznHJLz+4ngSjvOxFpYXFpeWS2sFdc3Nre27Z3dpopTybDBYhHLax8UCh5hQ3Mt8DqRCKEvsOUPatN6a4hS8Ti60qMEOyHcRDzgDLSxunbt0AOR9IF6Au+o1wsksMwbgkz6/Pi+e3ty+nXhJ+PMWKeGxnm/jxoOu3bJKTsz0b/g5lAiuepd+9nrxSwNMdJMgFJt10l0JwOpORM4LnqpwgTYAG6wbTCCEFUnmy07pkfG6dEgluZEms7c7xMZhEqNQt90hqD76ndtav5Xa6c6OO9kPEpSjRGbPxSkguqYTpOjPS6RaTEyAExy81fK+mDy0ibfognB/b3yX2hWyq7hy0qpWs3jKJB9ckCOiUvOSJVckDppEEYeyBN5JRPr0Xqx3qz3eeuClc/skR+yPj4Be/+nuQ==</latexit><latexit sha1_base64="bXpVOSwBhezioWcgI+XxN1XSrkk=">AAACLHicbZDLSgMxFIYzXmu9jbp0E6yCLiwz3eiy0I3LCrYVOqWcSc/Y2MzFJFMsQx/Ija8iiAuLuPU5TNsBrz8EvvznHJLz+4ngSjvOxFpYXFpeWS2sFdc3Nre27Z3dpopTybDBYhHLax8UCh5hQ3Mt8DqRCKEvsOUPatN6a4hS8Ti60qMEOyHcRDzgDLSxunbt0AOR9IF6Au+o1wsksMwbgkz6/Pi+e3ty+nXhJ+PMWKeGxnm/jxoOu3bJKTsz0b/g5lAiuepd+9nrxSwNMdJMgFJt10l0JwOpORM4LnqpwgTYAG6wbTCCEFUnmy07pkfG6dEgluZEms7c7xMZhEqNQt90hqD76ndtav5Xa6c6OO9kPEpSjRGbPxSkguqYTpOjPS6RaTEyAExy81fK+mDy0ibfognB/b3yX2hWyq7hy0qpWs3jKJB9ckCOiUvOSJVckDppEEYeyBN5JRPr0Xqx3qz3eeuClc/skR+yPj4Be/+nuQ==</latexit><latexit sha1_base64="bXpVOSwBhezioWcgI+XxN1XSrkk=">AAACLHicbZDLSgMxFIYzXmu9jbp0E6yCLiwz3eiy0I3LCrYVOqWcSc/Y2MzFJFMsQx/Ija8iiAuLuPU5TNsBrz8EvvznHJLz+4ngSjvOxFpYXFpeWS2sFdc3Nre27Z3dpopTybDBYhHLax8UCh5hQ3Mt8DqRCKEvsOUPatN6a4hS8Ti60qMEOyHcRDzgDLSxunbt0AOR9IF6Au+o1wsksMwbgkz6/Pi+e3ty+nXhJ+PMWKeGxnm/jxoOu3bJKTsz0b/g5lAiuepd+9nrxSwNMdJMgFJt10l0JwOpORM4LnqpwgTYAG6wbTCCEFUnmy07pkfG6dEgluZEms7c7xMZhEqNQt90hqD76ndtav5Xa6c6OO9kPEpSjRGbPxSkguqYTpOjPS6RaTEyAExy81fK+mDy0ibfognB/b3yX2hWyq7hy0qpWs3jKJB9ckCOiUvOSJVckDppEEYeyBN5JRPr0Xqx3qz3eeuClc/skR+yPj4Be/+nuQ==</latexit><latexit sha1_base64="bXpVOSwBhezioWcgI+XxN1XSrkk=">AAACLHicbZDLSgMxFIYzXmu9jbp0E6yCLiwz3eiy0I3LCrYVOqWcSc/Y2MzFJFMsQx/Ija8iiAuLuPU5TNsBrz8EvvznHJLz+4ngSjvOxFpYXFpeWS2sFdc3Nre27Z3dpopTybDBYhHLax8UCh5hQ3Mt8DqRCKEvsOUPatN6a4hS8Ti60qMEOyHcRDzgDLSxunbt0AOR9IF6Au+o1wsksMwbgkz6/Pi+e3ty+nXhJ+PMWKeGxnm/jxoOu3bJKTsz0b/g5lAiuepd+9nrxSwNMdJMgFJt10l0JwOpORM4LnqpwgTYAG6wbTCCEFUnmy07pkfG6dEgluZEms7c7xMZhEqNQt90hqD76ndtav5Xa6c6OO9kPEpSjRGbPxSkguqYTpOjPS6RaTEyAExy81fK+mDy0ibfognB/b3yX2hWyq7hy0qpWs3jKJB9ckCOiUvOSJVckDppEEYeyBN5JRPr0Xqx3qz3eeuClc/skR+yPj4Be/+nuQ==</latexit>

xi<latexit sha1_base64="pmg+gnVv+HU4HLbjrH6Vq/vhhsQ=">AAAB7HicbZA9TwJBEIbn8AvxC7W02QgmVuSORkoSG0tM5COBC9lb9mDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3iCRwqDrfjuFre2d3b3ifung8Oj4pHx61jFxqhlvs1jGuhdQw6VQvI0CJe8lmtMokLwbTG8X9e4j10bE6gFnCfcjOlYiFIyitdrVp6GoDssVt+YuRTbBy6ECuVrD8tdgFLM04gqZpMb0PTdBP6MaBZN8XhqkhieUTemY9y0qGnHjZ8tl5+TKOiMSxto+hWTp/p7IaGTMLApsZ0RxYtZrC/O/Wj/FsOFnQiUpcsVWH4WpJBiTxeVkJDRnKGcWKNPC7krYhGrK0OZTsiF46ydvQqde8yzf1yvNRh5HES7gEq7Bgxtowh20oA0MBDzDK7w5ynlx3p2PVWvByWfO4Y+czx8VwY4o</latexit><latexit sha1_base64="pmg+gnVv+HU4HLbjrH6Vq/vhhsQ=">AAAB7HicbZA9TwJBEIbn8AvxC7W02QgmVuSORkoSG0tM5COBC9lb9mDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3iCRwqDrfjuFre2d3b3ifung8Oj4pHx61jFxqhlvs1jGuhdQw6VQvI0CJe8lmtMokLwbTG8X9e4j10bE6gFnCfcjOlYiFIyitdrVp6GoDssVt+YuRTbBy6ECuVrD8tdgFLM04gqZpMb0PTdBP6MaBZN8XhqkhieUTemY9y0qGnHjZ8tl5+TKOiMSxto+hWTp/p7IaGTMLApsZ0RxYtZrC/O/Wj/FsOFnQiUpcsVWH4WpJBiTxeVkJDRnKGcWKNPC7krYhGrK0OZTsiF46ydvQqde8yzf1yvNRh5HES7gEq7Bgxtowh20oA0MBDzDK7w5ynlx3p2PVWvByWfO4Y+czx8VwY4o</latexit><latexit sha1_base64="pmg+gnVv+HU4HLbjrH6Vq/vhhsQ=">AAAB7HicbZA9TwJBEIbn8AvxC7W02QgmVuSORkoSG0tM5COBC9lb9mDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3iCRwqDrfjuFre2d3b3ifung8Oj4pHx61jFxqhlvs1jGuhdQw6VQvI0CJe8lmtMokLwbTG8X9e4j10bE6gFnCfcjOlYiFIyitdrVp6GoDssVt+YuRTbBy6ECuVrD8tdgFLM04gqZpMb0PTdBP6MaBZN8XhqkhieUTemY9y0qGnHjZ8tl5+TKOiMSxto+hWTp/p7IaGTMLApsZ0RxYtZrC/O/Wj/FsOFnQiUpcsVWH4WpJBiTxeVkJDRnKGcWKNPC7krYhGrK0OZTsiF46ydvQqde8yzf1yvNRh5HES7gEq7Bgxtowh20oA0MBDzDK7w5ynlx3p2PVWvByWfO4Y+czx8VwY4o</latexit><latexit sha1_base64="pmg+gnVv+HU4HLbjrH6Vq/vhhsQ=">AAAB7HicbZA9TwJBEIbn8AvxC7W02QgmVuSORkoSG0tM5COBC9lb9mDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3iCRwqDrfjuFre2d3b3ifung8Oj4pHx61jFxqhlvs1jGuhdQw6VQvI0CJe8lmtMokLwbTG8X9e4j10bE6gFnCfcjOlYiFIyitdrVp6GoDssVt+YuRTbBy6ECuVrD8tdgFLM04gqZpMb0PTdBP6MaBZN8XhqkhieUTemY9y0qGnHjZ8tl5+TKOiMSxto+hWTp/p7IaGTMLApsZ0RxYtZrC/O/Wj/FsOFnQiUpcsVWH4WpJBiTxeVkJDRnKGcWKNPC7krYhGrK0OZTsiF46ydvQqde8yzf1yvNRh5HES7gEq7Bgxtowh20oA0MBDzDK7w5ynlx3p2PVWvByWfO4Y+czx8VwY4o</latexit>

'(xi)<latexit sha1_base64="2RSP5OM1rKvlHPSZBvlymYbyN3E=">AAAB9XicbZDLTgIxFIbP4A3xhrp00wgmuCEzbHRJ4sYlJnJJYCSd0oGGTjtpOyiZ8B5uXGiMW9/FnW9jgVko+CdNvvznnJzTP4g508Z1v53cxubW9k5+t7C3f3B4VDw+aWmZKEKbRHKpOgHWlDNBm4YZTjuxojgKOG0H45t5vT2hSjMp7s00pn6Eh4KFjGBjrYdyb4JVPGKVpz67LPeLJbfqLoTWwcugBJka/eJXbyBJElFhCMdadz03Nn6KlWGE01mhl2gaYzLGQ9q1KHBEtZ8urp6hC+sMUCiVfcKghft7IsWR1tMosJ0RNiO9Wpub/9W6iQmv/ZSJODFUkOWiMOHISDSPAA2YosTwqQVMFLO3IjLCChNjgyrYELzVL69Dq1b1LN/VSvV6FkcezuAcKuDBFdThFhrQBAIKnuEV3pxH58V5dz6WrTknmzmFP3I+fwBh5JHB</latexit><latexit sha1_base64="2RSP5OM1rKvlHPSZBvlymYbyN3E=">AAAB9XicbZDLTgIxFIbP4A3xhrp00wgmuCEzbHRJ4sYlJnJJYCSd0oGGTjtpOyiZ8B5uXGiMW9/FnW9jgVko+CdNvvznnJzTP4g508Z1v53cxubW9k5+t7C3f3B4VDw+aWmZKEKbRHKpOgHWlDNBm4YZTjuxojgKOG0H45t5vT2hSjMp7s00pn6Eh4KFjGBjrYdyb4JVPGKVpz67LPeLJbfqLoTWwcugBJka/eJXbyBJElFhCMdadz03Nn6KlWGE01mhl2gaYzLGQ9q1KHBEtZ8urp6hC+sMUCiVfcKghft7IsWR1tMosJ0RNiO9Wpub/9W6iQmv/ZSJODFUkOWiMOHISDSPAA2YosTwqQVMFLO3IjLCChNjgyrYELzVL69Dq1b1LN/VSvV6FkcezuAcKuDBFdThFhrQBAIKnuEV3pxH58V5dz6WrTknmzmFP3I+fwBh5JHB</latexit><latexit sha1_base64="2RSP5OM1rKvlHPSZBvlymYbyN3E=">AAAB9XicbZDLTgIxFIbP4A3xhrp00wgmuCEzbHRJ4sYlJnJJYCSd0oGGTjtpOyiZ8B5uXGiMW9/FnW9jgVko+CdNvvznnJzTP4g508Z1v53cxubW9k5+t7C3f3B4VDw+aWmZKEKbRHKpOgHWlDNBm4YZTjuxojgKOG0H45t5vT2hSjMp7s00pn6Eh4KFjGBjrYdyb4JVPGKVpz67LPeLJbfqLoTWwcugBJka/eJXbyBJElFhCMdadz03Nn6KlWGE01mhl2gaYzLGQ9q1KHBEtZ8urp6hC+sMUCiVfcKghft7IsWR1tMosJ0RNiO9Wpub/9W6iQmv/ZSJODFUkOWiMOHISDSPAA2YosTwqQVMFLO3IjLCChNjgyrYELzVL69Dq1b1LN/VSvV6FkcezuAcKuDBFdThFhrQBAIKnuEV3pxH58V5dz6WrTknmzmFP3I+fwBh5JHB</latexit><latexit sha1_base64="2RSP5OM1rKvlHPSZBvlymYbyN3E=">AAAB9XicbZDLTgIxFIbP4A3xhrp00wgmuCEzbHRJ4sYlJnJJYCSd0oGGTjtpOyiZ8B5uXGiMW9/FnW9jgVko+CdNvvznnJzTP4g508Z1v53cxubW9k5+t7C3f3B4VDw+aWmZKEKbRHKpOgHWlDNBm4YZTjuxojgKOG0H45t5vT2hSjMp7s00pn6Eh4KFjGBjrYdyb4JVPGKVpz67LPeLJbfqLoTWwcugBJka/eJXbyBJElFhCMdadz03Nn6KlWGE01mhl2gaYzLGQ9q1KHBEtZ8urp6hC+sMUCiVfcKghft7IsWR1tMosJ0RNiO9Wpub/9W6iQmv/ZSJODFUkOWiMOHISDSPAA2YosTwqQVMFLO3IjLCChNjgyrYELzVL69Dq1b1LN/VSvV6FkcezuAcKuDBFdThFhrQBAIKnuEV3pxH58V5dz6WrTknmzmFP3I+fwBh5JHB</latexit>

'(yi)<latexit sha1_base64="RG1iAtaTSYwavALhRg+orYL4LFU=">AAAB9XicbZDLTgIxFIbPeEW8oS7dNIIJbsgMG12SuHGJiVwSGEmndKCh02naDmYy4T3cuNAYt76LO9/GArNQ8E+afPnPOTmnfyA508Z1v52Nza3tnd3CXnH/4PDouHRy2tZxoghtkZjHqhtgTTkTtGWY4bQrFcVRwGknmNzO650pVZrF4sGkkvoRHgkWMoKNtR4r/SlWcsyq6YBdVQalsltzF0Lr4OVQhlzNQemrP4xJElFhCMda9zxXGj/DyjDC6azYTzSVmEzwiPYsChxR7WeLq2fo0jpDFMbKPmHQwv09keFI6zQKbGeEzViv1ubmf7VeYsIbP2NCJoYKslwUJhyZGM0jQEOmKDE8tYCJYvZWRMZYYWJsUEUbgrf65XVo12ue5ft6udHI4yjAOVxAFTy4hgbcQRNaQEDBM7zCm/PkvDjvzseydcPJZ87gj5zPH2NskcI=</latexit><latexit sha1_base64="RG1iAtaTSYwavALhRg+orYL4LFU=">AAAB9XicbZDLTgIxFIbPeEW8oS7dNIIJbsgMG12SuHGJiVwSGEmndKCh02naDmYy4T3cuNAYt76LO9/GArNQ8E+afPnPOTmnfyA508Z1v52Nza3tnd3CXnH/4PDouHRy2tZxoghtkZjHqhtgTTkTtGWY4bQrFcVRwGknmNzO650pVZrF4sGkkvoRHgkWMoKNtR4r/SlWcsyq6YBdVQalsltzF0Lr4OVQhlzNQemrP4xJElFhCMda9zxXGj/DyjDC6azYTzSVmEzwiPYsChxR7WeLq2fo0jpDFMbKPmHQwv09keFI6zQKbGeEzViv1ubmf7VeYsIbP2NCJoYKslwUJhyZGM0jQEOmKDE8tYCJYvZWRMZYYWJsUEUbgrf65XVo12ue5ft6udHI4yjAOVxAFTy4hgbcQRNaQEDBM7zCm/PkvDjvzseydcPJZ87gj5zPH2NskcI=</latexit><latexit sha1_base64="RG1iAtaTSYwavALhRg+orYL4LFU=">AAAB9XicbZDLTgIxFIbPeEW8oS7dNIIJbsgMG12SuHGJiVwSGEmndKCh02naDmYy4T3cuNAYt76LO9/GArNQ8E+afPnPOTmnfyA508Z1v52Nza3tnd3CXnH/4PDouHRy2tZxoghtkZjHqhtgTTkTtGWY4bQrFcVRwGknmNzO650pVZrF4sGkkvoRHgkWMoKNtR4r/SlWcsyq6YBdVQalsltzF0Lr4OVQhlzNQemrP4xJElFhCMda9zxXGj/DyjDC6azYTzSVmEzwiPYsChxR7WeLq2fo0jpDFMbKPmHQwv09keFI6zQKbGeEzViv1ubmf7VeYsIbP2NCJoYKslwUJhyZGM0jQEOmKDE8tYCJYvZWRMZYYWJsUEUbgrf65XVo12ue5ft6udHI4yjAOVxAFTy4hgbcQRNaQEDBM7zCm/PkvDjvzseydcPJZ87gj5zPH2NskcI=</latexit><latexit sha1_base64="RG1iAtaTSYwavALhRg+orYL4LFU=">AAAB9XicbZDLTgIxFIbPeEW8oS7dNIIJbsgMG12SuHGJiVwSGEmndKCh02naDmYy4T3cuNAYt76LO9/GArNQ8E+afPnPOTmnfyA508Z1v52Nza3tnd3CXnH/4PDouHRy2tZxoghtkZjHqhtgTTkTtGWY4bQrFcVRwGknmNzO650pVZrF4sGkkvoRHgkWMoKNtR4r/SlWcsyq6YBdVQalsltzF0Lr4OVQhlzNQemrP4xJElFhCMda9zxXGj/DyjDC6azYTzSVmEzwiPYsChxR7WeLq2fo0jpDFMbKPmHQwv09keFI6zQKbGeEzViv1ubmf7VeYsIbP2NCJoYKslwUJhyZGM0jQEOmKDE8tYCJYvZWRMZYYWJsUEUbgrf65XVo12ue5ft6udHI4yjAOVxAFTy4hgbcQRNaQEDBM7zCm/PkvDjvzseydcPJZ87gj5zPH2NskcI=</latexit>

yi<latexit sha1_base64="WmgSYaT4KZFZh7wSYgzHlBn+laY=">AAAB7HicbZBNT8JAEIan+IX4hXr0shFMPJGWix5JvHjExAIJNGS7bGHDdtvsTk1Iw2/w4kFjvPqDvPlvXKAHBd9kkyfvzGRn3jCVwqDrfjulre2d3b3yfuXg8Oj4pHp61jFJphn3WSIT3Qup4VIo7qNAyXup5jQOJe+G07tFvfvEtRGJesRZyoOYjpWIBKNoLb8+G4r6sFpzG+5SZBO8AmpQqD2sfg1GCctirpBJakzfc1MMcqpRMMnnlUFmeErZlI5536KiMTdBvlx2Tq6sMyJRou1TSJbu74mcxsbM4tB2xhQnZr22MP+r9TOMboNcqDRDrtjqoyiTBBOyuJyMhOYM5cwCZVrYXQmbUE0Z2nwqNgRv/eRN6DQbnuWHZq3VKuIowwVcwjV4cAMtuIc2+MBAwDO8wpujnBfn3flYtZacYuYc/sj5/AEZsI4x</latexit><latexit sha1_base64="WmgSYaT4KZFZh7wSYgzHlBn+laY=">AAAB7HicbZBNT8JAEIan+IX4hXr0shFMPJGWix5JvHjExAIJNGS7bGHDdtvsTk1Iw2/w4kFjvPqDvPlvXKAHBd9kkyfvzGRn3jCVwqDrfjulre2d3b3yfuXg8Oj4pHp61jFJphn3WSIT3Qup4VIo7qNAyXup5jQOJe+G07tFvfvEtRGJesRZyoOYjpWIBKNoLb8+G4r6sFpzG+5SZBO8AmpQqD2sfg1GCctirpBJakzfc1MMcqpRMMnnlUFmeErZlI5536KiMTdBvlx2Tq6sMyJRou1TSJbu74mcxsbM4tB2xhQnZr22MP+r9TOMboNcqDRDrtjqoyiTBBOyuJyMhOYM5cwCZVrYXQmbUE0Z2nwqNgRv/eRN6DQbnuWHZq3VKuIowwVcwjV4cAMtuIc2+MBAwDO8wpujnBfn3flYtZacYuYc/sj5/AEZsI4x</latexit><latexit sha1_base64="WmgSYaT4KZFZh7wSYgzHlBn+laY=">AAAB7HicbZBNT8JAEIan+IX4hXr0shFMPJGWix5JvHjExAIJNGS7bGHDdtvsTk1Iw2/w4kFjvPqDvPlvXKAHBd9kkyfvzGRn3jCVwqDrfjulre2d3b3yfuXg8Oj4pHp61jFJphn3WSIT3Qup4VIo7qNAyXup5jQOJe+G07tFvfvEtRGJesRZyoOYjpWIBKNoLb8+G4r6sFpzG+5SZBO8AmpQqD2sfg1GCctirpBJakzfc1MMcqpRMMnnlUFmeErZlI5536KiMTdBvlx2Tq6sMyJRou1TSJbu74mcxsbM4tB2xhQnZr22MP+r9TOMboNcqDRDrtjqoyiTBBOyuJyMhOYM5cwCZVrYXQmbUE0Z2nwqNgRv/eRN6DQbnuWHZq3VKuIowwVcwjV4cAMtuIc2+MBAwDO8wpujnBfn3flYtZacYuYc/sj5/AEZsI4x</latexit><latexit sha1_base64="WmgSYaT4KZFZh7wSYgzHlBn+laY=">AAAB7HicbZBNT8JAEIan+IX4hXr0shFMPJGWix5JvHjExAIJNGS7bGHDdtvsTk1Iw2/w4kFjvPqDvPlvXKAHBd9kkyfvzGRn3jCVwqDrfjulre2d3b3yfuXg8Oj4pHp61jFJphn3WSIT3Qup4VIo7qNAyXup5jQOJe+G07tFvfvEtRGJesRZyoOYjpWIBKNoLb8+G4r6sFpzG+5SZBO8AmpQqD2sfg1GCctirpBJakzfc1MMcqpRMMnnlUFmeErZlI5536KiMTdBvlx2Tq6sMyJRou1TSJbu74mcxsbM4tB2xhQnZr22MP+r9TOMboNcqDRDrtjqoyiTBBOyuJyMhOYM5cwCZVrYXQmbUE0Z2nwqNgRv/eRN6DQbnuWHZq3VKuIowwVcwjV4cAMtuIc2+MBAwDO8wpujnBfn3flYtZacYuYc/sj5/AEZsI4x</latexit>

-5 0 5-1

(xi,'(xi))<latexit sha1_base64="GllpdvBtNW9LH6e2gGOX+rXlP0k=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdFty4rGAv0IYwmU7aoZNJmJkUayi+ihsXirj1Pdz5Nk7aLLT1h4GP/5zDOfP7MaNS2fa3UVhb39jcKm6Xdnb39g/Mw6O2jBKBSQtHLBJdH0nCKCctRRUj3VgQFPqMdPzxTVbvTIiQNOL3ahoTN0RDTgOKkdKWZ55Uqg8evehPkIhHNONareKZZfvSnstaBSeHMuRqeuZXfxDhJCRcYYak7Dl2rNwUCUUxI7NSP5EkRniMhqSnkaOQSDedXz+zzrUzsIJI6MeVNXd/T6QolHIa+rozRGokl2uZ+V+tl6jg2k0pjxNFOF4sChJmqcjKorAGVBCs2FQDwoLqWy08QgJhpQMr6RCc5S+vQrt+6Wi+q5cbjTyOIpzCGVTBgStowC00oQUYHuEZXuHNeDJejHfjY9FaMPKZY/gj4/MHebWT6w==</latexit><latexit sha1_base64="GllpdvBtNW9LH6e2gGOX+rXlP0k=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdFty4rGAv0IYwmU7aoZNJmJkUayi+ihsXirj1Pdz5Nk7aLLT1h4GP/5zDOfP7MaNS2fa3UVhb39jcKm6Xdnb39g/Mw6O2jBKBSQtHLBJdH0nCKCctRRUj3VgQFPqMdPzxTVbvTIiQNOL3ahoTN0RDTgOKkdKWZ55Uqg8evehPkIhHNONareKZZfvSnstaBSeHMuRqeuZXfxDhJCRcYYak7Dl2rNwUCUUxI7NSP5EkRniMhqSnkaOQSDedXz+zzrUzsIJI6MeVNXd/T6QolHIa+rozRGokl2uZ+V+tl6jg2k0pjxNFOF4sChJmqcjKorAGVBCs2FQDwoLqWy08QgJhpQMr6RCc5S+vQrt+6Wi+q5cbjTyOIpzCGVTBgStowC00oQUYHuEZXuHNeDJejHfjY9FaMPKZY/gj4/MHebWT6w==</latexit><latexit sha1_base64="GllpdvBtNW9LH6e2gGOX+rXlP0k=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdFty4rGAv0IYwmU7aoZNJmJkUayi+ihsXirj1Pdz5Nk7aLLT1h4GP/5zDOfP7MaNS2fa3UVhb39jcKm6Xdnb39g/Mw6O2jBKBSQtHLBJdH0nCKCctRRUj3VgQFPqMdPzxTVbvTIiQNOL3ahoTN0RDTgOKkdKWZ55Uqg8evehPkIhHNONareKZZfvSnstaBSeHMuRqeuZXfxDhJCRcYYak7Dl2rNwUCUUxI7NSP5EkRniMhqSnkaOQSDedXz+zzrUzsIJI6MeVNXd/T6QolHIa+rozRGokl2uZ+V+tl6jg2k0pjxNFOF4sChJmqcjKorAGVBCs2FQDwoLqWy08QgJhpQMr6RCc5S+vQrt+6Wi+q5cbjTyOIpzCGVTBgStowC00oQUYHuEZXuHNeDJejHfjY9FaMPKZY/gj4/MHebWT6w==</latexit><latexit sha1_base64="GllpdvBtNW9LH6e2gGOX+rXlP0k=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdFty4rGAv0IYwmU7aoZNJmJkUayi+ihsXirj1Pdz5Nk7aLLT1h4GP/5zDOfP7MaNS2fa3UVhb39jcKm6Xdnb39g/Mw6O2jBKBSQtHLBJdH0nCKCctRRUj3VgQFPqMdPzxTVbvTIiQNOL3ahoTN0RDTgOKkdKWZ55Uqg8evehPkIhHNONareKZZfvSnstaBSeHMuRqeuZXfxDhJCRcYYak7Dl2rNwUCUUxI7NSP5EkRniMhqSnkaOQSDedXz+zzrUzsIJI6MeVNXd/T6QolHIa+rozRGokl2uZ+V+tl6jg2k0pjxNFOF4sChJmqcjKorAGVBCs2FQDwoLqWy08QgJhpQMr6RCc5S+vQrt+6Wi+q5cbjTyOIpzCGVTBgStowC00oQUYHuEZXuHNeDJejHfjY9FaMPKZY/gj4/MHebWT6w==</latexit>

(yi,'(yi))<latexit sha1_base64="OwRczwPm0a80lXMROfDF4JnscN8=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdScGNywr2Am0Ik+mkHTqZhJlJIZbiq7hxoYhb38Odb+OkzUJbfxj4+M85nDO/HzMqlW1/G4W19Y3NreJ2aWd3b//APDxqyygRmLRwxCLR9ZEkjHLSUlQx0o0FQaHPSMcf32b1zoQISSP+oNKYuCEachpQjJS2PPOkUk09etGfIBGPaMa1WsUzy/alPZe1Ck4OZcjV9Myv/iDCSUi4wgxJ2XPsWLlTJBTFjMxK/USSGOExGpKeRo5CIt3p/PqZda6dgRVEQj+urLn7e2KKQinT0NedIVIjuVzLzP9qvUQF1+6U8jhRhOPFoiBhloqsLAprQAXBiqUaEBZU32rhERIIKx1YSYfgLH95Fdr1S0fzfb3cuMnjKMIpnEEVHLiCBtxBE1qA4RGe4RXejCfjxXg3PhatBSOfOYY/Mj5/AHw5k+s=</latexit><latexit sha1_base64="OwRczwPm0a80lXMROfDF4JnscN8=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdScGNywr2Am0Ik+mkHTqZhJlJIZbiq7hxoYhb38Odb+OkzUJbfxj4+M85nDO/HzMqlW1/G4W19Y3NreJ2aWd3b//APDxqyygRmLRwxCLR9ZEkjHLSUlQx0o0FQaHPSMcf32b1zoQISSP+oNKYuCEachpQjJS2PPOkUk09etGfIBGPaMa1WsUzy/alPZe1Ck4OZcjV9Myv/iDCSUi4wgxJ2XPsWLlTJBTFjMxK/USSGOExGpKeRo5CIt3p/PqZda6dgRVEQj+urLn7e2KKQinT0NedIVIjuVzLzP9qvUQF1+6U8jhRhOPFoiBhloqsLAprQAXBiqUaEBZU32rhERIIKx1YSYfgLH95Fdr1S0fzfb3cuMnjKMIpnEEVHLiCBtxBE1qA4RGe4RXejCfjxXg3PhatBSOfOYY/Mj5/AHw5k+s=</latexit><latexit sha1_base64="OwRczwPm0a80lXMROfDF4JnscN8=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdScGNywr2Am0Ik+mkHTqZhJlJIZbiq7hxoYhb38Odb+OkzUJbfxj4+M85nDO/HzMqlW1/G4W19Y3NreJ2aWd3b//APDxqyygRmLRwxCLR9ZEkjHLSUlQx0o0FQaHPSMcf32b1zoQISSP+oNKYuCEachpQjJS2PPOkUk09etGfIBGPaMa1WsUzy/alPZe1Ck4OZcjV9Myv/iDCSUi4wgxJ2XPsWLlTJBTFjMxK/USSGOExGpKeRo5CIt3p/PqZda6dgRVEQj+urLn7e2KKQinT0NedIVIjuVzLzP9qvUQF1+6U8jhRhOPFoiBhloqsLAprQAXBiqUaEBZU32rhERIIKx1YSYfgLH95Fdr1S0fzfb3cuMnjKMIpnEEVHLiCBtxBE1qA4RGe4RXejCfjxXg3PhatBSOfOYY/Mj5/AHw5k+s=</latexit><latexit sha1_base64="OwRczwPm0a80lXMROfDF4JnscN8=">AAAB/XicbZDLSsNAFIZP6q3WW7zs3ARboQWRpBtdScGNywr2Am0Ik+mkHTqZhJlJIZbiq7hxoYhb38Odb+OkzUJbfxj4+M85nDO/HzMqlW1/G4W19Y3NreJ2aWd3b//APDxqyygRmLRwxCLR9ZEkjHLSUlQx0o0FQaHPSMcf32b1zoQISSP+oNKYuCEachpQjJS2PPOkUk09etGfIBGPaMa1WsUzy/alPZe1Ck4OZcjV9Myv/iDCSUi4wgxJ2XPsWLlTJBTFjMxK/USSGOExGpKeRo5CIt3p/PqZda6dgRVEQj+urLn7e2KKQinT0NedIVIjuVzLzP9qvUQF1+6U8jhRhOPFoiBhloqsLAprQAXBiqUaEBZU32rhERIIKx1YSYfgLH95Fdr1S0fzfb3cuMnjKMIpnEEVHLiCBtxBE1qA4RGe4RXejCfjxXg3PhatBSOfOYY/Mj5/AHw5k+s=</latexit>

↵ '(yi) � '(xi)

yi � xi �

<latexit sha1_base64="71vqPKjY+TK9gITBZZq4AVlX2HI=">AAACLHicbZBNS8NAEIY3flu/qh69LFZBD0rSix4LXjxWsLbQlDLZTpqlmw93N8US8oO8+FcE8WARr/4OtzWCtg4sPPPODLPzeongStv22FpYXFpeWV1bL21sbm3vlHf37lScSoYNFotYtjxQKHiEDc21wFYiEUJPYNMbXE3qzSFKxePoVo8S7ITQj7jPGWgjdctXRy6IJADqCrynbs+XwDJ3CDIJ+Mmoy0/PfpIHk+SZkc4M5UW/hxqOuuWKfW5Pg86DU0CFFFHvll/cXszSECPNBCjVduxEdzKQmjOBeclNFSbABtDHtsEIQlSdbHpsTo+N0qN+LM2LNJ2qvycyCJUahZ7pDEEHarY2Ef+rtVPtX3YyHiWpxoh9L/JTQXVMJ87RHpfItBgZACa5+StlARi/tPG3ZExwZk+eh7vquWP4plqp1Qo71sgBOSQnxCEXpEauSZ00CCOP5Jm8kbH1ZL1a79bHd+uCVczskz9hfX4BfAOnuQ==</latexit><latexit sha1_base64="71vqPKjY+TK9gITBZZq4AVlX2HI=">AAACLHicbZBNS8NAEIY3flu/qh69LFZBD0rSix4LXjxWsLbQlDLZTpqlmw93N8US8oO8+FcE8WARr/4OtzWCtg4sPPPODLPzeongStv22FpYXFpeWV1bL21sbm3vlHf37lScSoYNFotYtjxQKHiEDc21wFYiEUJPYNMbXE3qzSFKxePoVo8S7ITQj7jPGWgjdctXRy6IJADqCrynbs+XwDJ3CDIJ+Mmoy0/PfpIHk+SZkc4M5UW/hxqOuuWKfW5Pg86DU0CFFFHvll/cXszSECPNBCjVduxEdzKQmjOBeclNFSbABtDHtsEIQlSdbHpsTo+N0qN+LM2LNJ2qvycyCJUahZ7pDEEHarY2Ef+rtVPtX3YyHiWpxoh9L/JTQXVMJ87RHpfItBgZACa5+StlARi/tPG3ZExwZk+eh7vquWP4plqp1Qo71sgBOSQnxCEXpEauSZ00CCOP5Jm8kbH1ZL1a79bHd+uCVczskz9hfX4BfAOnuQ==</latexit><latexit sha1_base64="71vqPKjY+TK9gITBZZq4AVlX2HI=">AAACLHicbZBNS8NAEIY3flu/qh69LFZBD0rSix4LXjxWsLbQlDLZTpqlmw93N8US8oO8+FcE8WARr/4OtzWCtg4sPPPODLPzeongStv22FpYXFpeWV1bL21sbm3vlHf37lScSoYNFotYtjxQKHiEDc21wFYiEUJPYNMbXE3qzSFKxePoVo8S7ITQj7jPGWgjdctXRy6IJADqCrynbs+XwDJ3CDIJ+Mmoy0/PfpIHk+SZkc4M5UW/hxqOuuWKfW5Pg86DU0CFFFHvll/cXszSECPNBCjVduxEdzKQmjOBeclNFSbABtDHtsEIQlSdbHpsTo+N0qN+LM2LNJ2qvycyCJUahZ7pDEEHarY2Ef+rtVPtX3YyHiWpxoh9L/JTQXVMJ87RHpfItBgZACa5+StlARi/tPG3ZExwZk+eh7vquWP4plqp1Qo71sgBOSQnxCEXpEauSZ00CCOP5Jm8kbH1ZL1a79bHd+uCVczskz9hfX4BfAOnuQ==</latexit><latexit sha1_base64="71vqPKjY+TK9gITBZZq4AVlX2HI=">AAACLHicbZBNS8NAEIY3flu/qh69LFZBD0rSix4LXjxWsLbQlDLZTpqlmw93N8US8oO8+FcE8WARr/4OtzWCtg4sPPPODLPzeongStv22FpYXFpeWV1bL21sbm3vlHf37lScSoYNFotYtjxQKHiEDc21wFYiEUJPYNMbXE3qzSFKxePoVo8S7ITQj7jPGWgjdctXRy6IJADqCrynbs+XwDJ3CDIJ+Mmoy0/PfpIHk+SZkc4M5UW/hxqOuuWKfW5Pg86DU0CFFFHvll/cXszSECPNBCjVduxEdzKQmjOBeclNFSbABtDHtsEIQlSdbHpsTo+N0qN+LM2LNJ2qvycyCJUahZ7pDEEHarY2Ef+rtVPtX3YyHiWpxoh9L/JTQXVMJ87RHpfItBgZACa5+StlARi/tPG3ZExwZk+eh7vquWP4plqp1Qo71sgBOSQnxCEXpEauSZ00CCOP5Jm8kbH1ZL1a79bHd+uCVczskz9hfX4BfAOnuQ==</latexit>

xj<latexit sha1_base64="Qmcs1an4x+++o9Cy9ZfnlIj1FDM=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmjkZLExhITD0iAkL1lDlb29i67e0Zy4TfYWGiMrT/Izn/jAlco+CabPHlnJjvzBong2rjut7OxubW9s1vYK+4fHB4dl05OWzpOFUOfxSJWnYBqFFyib7gR2EkU0igQ2A4mN/N6+xGV5rG8N9ME+xEdSR5yRo21/MrT4KEyKJXdqrsQWQcvhzLkag5KX71hzNIIpWGCat313MT0M6oMZwJnxV6qMaFsQkfYtShphLqfLZadkUvrDEkYK/ukIQv390RGI62nUWA7I2rGerU2N/+rdVMT1vsZl0lqULLlR2EqiInJ/HIy5AqZEVMLlCludyVsTBVlxuZTtCF4qyevQ6tW9Szf1cqNeh5HAc7hAq7Ag2towC00wQcGHJ7hFd4c6bw4787HsnXDyWfO4I+czx8XRo4p</latexit><latexit sha1_base64="Qmcs1an4x+++o9Cy9ZfnlIj1FDM=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmjkZLExhITD0iAkL1lDlb29i67e0Zy4TfYWGiMrT/Izn/jAlco+CabPHlnJjvzBong2rjut7OxubW9s1vYK+4fHB4dl05OWzpOFUOfxSJWnYBqFFyib7gR2EkU0igQ2A4mN/N6+xGV5rG8N9ME+xEdSR5yRo21/MrT4KEyKJXdqrsQWQcvhzLkag5KX71hzNIIpWGCat313MT0M6oMZwJnxV6qMaFsQkfYtShphLqfLZadkUvrDEkYK/ukIQv390RGI62nUWA7I2rGerU2N/+rdVMT1vsZl0lqULLlR2EqiInJ/HIy5AqZEVMLlCludyVsTBVlxuZTtCF4qyevQ6tW9Szf1cqNeh5HAc7hAq7Ag2towC00wQcGHJ7hFd4c6bw4787HsnXDyWfO4I+czx8XRo4p</latexit><latexit sha1_base64="Qmcs1an4x+++o9Cy9ZfnlIj1FDM=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmjkZLExhITD0iAkL1lDlb29i67e0Zy4TfYWGiMrT/Izn/jAlco+CabPHlnJjvzBong2rjut7OxubW9s1vYK+4fHB4dl05OWzpOFUOfxSJWnYBqFFyib7gR2EkU0igQ2A4mN/N6+xGV5rG8N9ME+xEdSR5yRo21/MrT4KEyKJXdqrsQWQcvhzLkag5KX71hzNIIpWGCat313MT0M6oMZwJnxV6qMaFsQkfYtShphLqfLZadkUvrDEkYK/ukIQv390RGI62nUWA7I2rGerU2N/+rdVMT1vsZl0lqULLlR2EqiInJ/HIy5AqZEVMLlCludyVsTBVlxuZTtCF4qyevQ6tW9Szf1cqNeh5HAc7hAq7Ag2towC00wQcGHJ7hFd4c6bw4787HsnXDyWfO4I+czx8XRo4p</latexit><latexit sha1_base64="Qmcs1an4x+++o9Cy9ZfnlIj1FDM=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmjkZLExhITD0iAkL1lDlb29i67e0Zy4TfYWGiMrT/Izn/jAlco+CabPHlnJjvzBong2rjut7OxubW9s1vYK+4fHB4dl05OWzpOFUOfxSJWnYBqFFyib7gR2EkU0igQ2A4mN/N6+xGV5rG8N9ME+xEdSR5yRo21/MrT4KEyKJXdqrsQWQcvhzLkag5KX71hzNIIpWGCat313MT0M6oMZwJnxV6qMaFsQkfYtShphLqfLZadkUvrDEkYK/ukIQv390RGI62nUWA7I2rGerU2N/+rdVMT1vsZl0lqULLlR2EqiInJ/HIy5AqZEVMLlCludyVsTBVlxuZTtCF4qyevQ6tW9Szf1cqNeh5HAc7hAq7Ag2towC00wQcGHJ7hFd4c6bw4787HsnXDyWfO4I+czx8XRo4p</latexit>

yj<latexit sha1_base64="GXeTI3hCKBcAvNjSYFW2ns3xRoM=">AAAB7HicbZDNTgIxFIVv8Q/xD3XpphFMXJEZNrIkceMSEwdIYEI6pQOVTmfSdkwmE57BjQuNcesDufNtLDALBU/S5Mu596b3niARXBvH+Ualre2d3b3yfuXg8Oj4pHp61tVxqijzaCxi1Q+IZoJL5hluBOsnipEoEKwXzG4X9d4TU5rH8sFkCfMjMpE85JQYa3n1bPRYH1VrTsNZCm+CW0ANCnVG1a/hOKZpxKShgmg9cJ3E+DlRhlPB5pVhqllC6IxM2MCiJBHTfr5cdo6vrDPGYazskwYv3d8TOYm0zqLAdkbETPV6bWH+VxukJmz5OZdJapikq4/CVGAT48XleMwVo0ZkFghV3O6K6ZQoQo3Np2JDcNdP3oRus+Favm/W2q0ijjJcwCVcgws30IY76IAHFDg8wyu8IYle0Dv6WLWWUDFzDn+EPn8AGM2OKg==</latexit><latexit sha1_base64="GXeTI3hCKBcAvNjSYFW2ns3xRoM=">AAAB7HicbZDNTgIxFIVv8Q/xD3XpphFMXJEZNrIkceMSEwdIYEI6pQOVTmfSdkwmE57BjQuNcesDufNtLDALBU/S5Mu596b3niARXBvH+Ualre2d3b3yfuXg8Oj4pHp61tVxqijzaCxi1Q+IZoJL5hluBOsnipEoEKwXzG4X9d4TU5rH8sFkCfMjMpE85JQYa3n1bPRYH1VrTsNZCm+CW0ANCnVG1a/hOKZpxKShgmg9cJ3E+DlRhlPB5pVhqllC6IxM2MCiJBHTfr5cdo6vrDPGYazskwYv3d8TOYm0zqLAdkbETPV6bWH+VxukJmz5OZdJapikq4/CVGAT48XleMwVo0ZkFghV3O6K6ZQoQo3Np2JDcNdP3oRus+Favm/W2q0ijjJcwCVcgws30IY76IAHFDg8wyu8IYle0Dv6WLWWUDFzDn+EPn8AGM2OKg==</latexit><latexit sha1_base64="GXeTI3hCKBcAvNjSYFW2ns3xRoM=">AAAB7HicbZDNTgIxFIVv8Q/xD3XpphFMXJEZNrIkceMSEwdIYEI6pQOVTmfSdkwmE57BjQuNcesDufNtLDALBU/S5Mu596b3niARXBvH+Ualre2d3b3yfuXg8Oj4pHp61tVxqijzaCxi1Q+IZoJL5hluBOsnipEoEKwXzG4X9d4TU5rH8sFkCfMjMpE85JQYa3n1bPRYH1VrTsNZCm+CW0ANCnVG1a/hOKZpxKShgmg9cJ3E+DlRhlPB5pVhqllC6IxM2MCiJBHTfr5cdo6vrDPGYazskwYv3d8TOYm0zqLAdkbETPV6bWH+VxukJmz5OZdJapikq4/CVGAT48XleMwVo0ZkFghV3O6K6ZQoQo3Np2JDcNdP3oRus+Favm/W2q0ijjJcwCVcgws30IY76IAHFDg8wyu8IYle0Dv6WLWWUDFzDn+EPn8AGM2OKg==</latexit><latexit sha1_base64="GXeTI3hCKBcAvNjSYFW2ns3xRoM=">AAAB7HicbZDNTgIxFIVv8Q/xD3XpphFMXJEZNrIkceMSEwdIYEI6pQOVTmfSdkwmE57BjQuNcesDufNtLDALBU/S5Mu596b3niARXBvH+Ualre2d3b3yfuXg8Oj4pHp61tVxqijzaCxi1Q+IZoJL5hluBOsnipEoEKwXzG4X9d4TU5rH8sFkCfMjMpE85JQYa3n1bPRYH1VrTsNZCm+CW0ANCnVG1a/hOKZpxKShgmg9cJ3E+DlRhlPB5pVhqllC6IxM2MCiJBHTfr5cdo6vrDPGYazskwYv3d8TOYm0zqLAdkbETPV6bWH+VxukJmz5OZdJapikq4/CVGAT48XleMwVo0ZkFghV3O6K6ZQoQo3Np2JDcNdP3oRus+Favm/W2q0ijjJcwCVcgws30IY76IAHFDg8wyu8IYle0Dv6WLWWUDFzDn+EPn8AGM2OKg==</latexit>

'(xj)<latexit sha1_base64="oI6G4J0ZwbR7XFQ0Ze3vRM+PxDI=">AAAB9XicbZDLTsJAFIZP8YZ4Q126mQgmuCEtG1mSuHGJiVwSqGQ6TGFkOm1mpihpeA83LjTGre/izrdxCl0o+CeTfPnPOTlnfi/iTGnb/rZyG5tb2zv53cLe/sHhUfH4pK3CWBLaIiEPZdfDinImaEszzWk3khQHHqcdb3Kd1jtTKhULxZ2eRdQN8EgwnxGsjXVf7k+xjMas8jR4uCwPiiW7ai+E1sHJoASZmoPiV38YkjigQhOOleo5dqTdBEvNCKfzQj9WNMJkgke0Z1DggCo3WVw9RxfGGSI/lOYJjRbu74kEB0rNAs90BliP1WotNf+r9WLt192EiSjWVJDlIj/mSIcojQANmaRE85kBTCQztyIyxhITbYIqmBCc1S+vQ7tWdQzf1kqNehZHHs7gHCrgwBU04Aaa0AICEp7hFd6sR+vFerc+lq05K5s5hT+yPn8AYQKRug==</latexit><latexit sha1_base64="oI6G4J0ZwbR7XFQ0Ze3vRM+PxDI=">AAAB9XicbZDLTsJAFIZP8YZ4Q126mQgmuCEtG1mSuHGJiVwSqGQ6TGFkOm1mpihpeA83LjTGre/izrdxCl0o+CeTfPnPOTlnfi/iTGnb/rZyG5tb2zv53cLe/sHhUfH4pK3CWBLaIiEPZdfDinImaEszzWk3khQHHqcdb3Kd1jtTKhULxZ2eRdQN8EgwnxGsjXVf7k+xjMas8jR4uCwPiiW7ai+E1sHJoASZmoPiV38YkjigQhOOleo5dqTdBEvNCKfzQj9WNMJkgke0Z1DggCo3WVw9RxfGGSI/lOYJjRbu74kEB0rNAs90BliP1WotNf+r9WLt192EiSjWVJDlIj/mSIcojQANmaRE85kBTCQztyIyxhITbYIqmBCc1S+vQ7tWdQzf1kqNehZHHs7gHCrgwBU04Aaa0AICEp7hFd6sR+vFerc+lq05K5s5hT+yPn8AYQKRug==</latexit><latexit sha1_base64="oI6G4J0ZwbR7XFQ0Ze3vRM+PxDI=">AAAB9XicbZDLTsJAFIZP8YZ4Q126mQgmuCEtG1mSuHGJiVwSqGQ6TGFkOm1mpihpeA83LjTGre/izrdxCl0o+CeTfPnPOTlnfi/iTGnb/rZyG5tb2zv53cLe/sHhUfH4pK3CWBLaIiEPZdfDinImaEszzWk3khQHHqcdb3Kd1jtTKhULxZ2eRdQN8EgwnxGsjXVf7k+xjMas8jR4uCwPiiW7ai+E1sHJoASZmoPiV38YkjigQhOOleo5dqTdBEvNCKfzQj9WNMJkgke0Z1DggCo3WVw9RxfGGSI/lOYJjRbu74kEB0rNAs90BliP1WotNf+r9WLt192EiSjWVJDlIj/mSIcojQANmaRE85kBTCQztyIyxhITbYIqmBCc1S+vQ7tWdQzf1kqNehZHHs7gHCrgwBU04Aaa0AICEp7hFd6sR+vFerc+lq05K5s5hT+yPn8AYQKRug==</latexit><latexit sha1_base64="oI6G4J0ZwbR7XFQ0Ze3vRM+PxDI=">AAAB9XicbZDLTsJAFIZP8YZ4Q126mQgmuCEtG1mSuHGJiVwSqGQ6TGFkOm1mpihpeA83LjTGre/izrdxCl0o+CeTfPnPOTlnfi/iTGnb/rZyG5tb2zv53cLe/sHhUfH4pK3CWBLaIiEPZdfDinImaEszzWk3khQHHqcdb3Kd1jtTKhULxZ2eRdQN8EgwnxGsjXVf7k+xjMas8jR4uCwPiiW7ai+E1sHJoASZmoPiV38YkjigQhOOleo5dqTdBEvNCKfzQj9WNMJkgke0Z1DggCo3WVw9RxfGGSI/lOYJjRbu74kEB0rNAs90BliP1WotNf+r9WLt192EiSjWVJDlIj/mSIcojQANmaRE85kBTCQztyIyxhITbYIqmBCc1S+vQ7tWdQzf1kqNehZHHs7gHCrgwBU04Aaa0AICEp7hFd6sR+vFerc+lq05K5s5hT+yPn8AYQKRug==</latexit>

'(yj)<latexit sha1_base64="I7R4a32uvuL7gDjagUmI6UlCRsU=">AAAB9XicbZDLTsJAFIZP8YZ4Q126mQgmuCEtG1iSuHGJiVwSqGQ6TGFkOm1mppim4T3cuNAYt76LO9/GAbpQ8E8m+fKfc3LO/F7EmdK2/W3ltrZ3dvfy+4WDw6Pjk+LpWUeFsSS0TUIeyp6HFeVM0LZmmtNeJCkOPE673vRmUe/OqFQsFPc6iagb4LFgPiNYG+uhPJhhGU1YJRk+XpeHxZJdtZdCm+BkUIJMrWHxazAKSRxQoQnHSvUdO9JuiqVmhNN5YRArGmEyxWPaNyhwQJWbLq+eoyvjjJAfSvOERkv390SKA6WSwDOdAdYTtV5bmP/V+rH2G27KRBRrKshqkR9zpEO0iACNmKRE88QAJpKZWxGZYImJNkEVTAjO+pc3oVOrOobvaqVmI4sjDxdwCRVwoA5NuIUWtIGAhGd4hTfryXqx3q2PVWvOymbO4Y+szx9iipG7</latexit><latexit sha1_base64="I7R4a32uvuL7gDjagUmI6UlCRsU=">AAAB9XicbZDLTsJAFIZP8YZ4Q126mQgmuCEtG1iSuHGJiVwSqGQ6TGFkOm1mppim4T3cuNAYt76LO9/GAbpQ8E8m+fKfc3LO/F7EmdK2/W3ltrZ3dvfy+4WDw6Pjk+LpWUeFsSS0TUIeyp6HFeVM0LZmmtNeJCkOPE673vRmUe/OqFQsFPc6iagb4LFgPiNYG+uhPJhhGU1YJRk+XpeHxZJdtZdCm+BkUIJMrWHxazAKSRxQoQnHSvUdO9JuiqVmhNN5YRArGmEyxWPaNyhwQJWbLq+eoyvjjJAfSvOERkv390SKA6WSwDOdAdYTtV5bmP/V+rH2G27KRBRrKshqkR9zpEO0iACNmKRE88QAJpKZWxGZYImJNkEVTAjO+pc3oVOrOobvaqVmI4sjDxdwCRVwoA5NuIUWtIGAhGd4hTfryXqx3q2PVWvOymbO4Y+szx9iipG7</latexit><latexit sha1_base64="I7R4a32uvuL7gDjagUmI6UlCRsU=">AAAB9XicbZDLTsJAFIZP8YZ4Q126mQgmuCEtG1iSuHGJiVwSqGQ6TGFkOm1mppim4T3cuNAYt76LO9/GAbpQ8E8m+fKfc3LO/F7EmdK2/W3ltrZ3dvfy+4WDw6Pjk+LpWUeFsSS0TUIeyp6HFeVM0LZmmtNeJCkOPE673vRmUe/OqFQsFPc6iagb4LFgPiNYG+uhPJhhGU1YJRk+XpeHxZJdtZdCm+BkUIJMrWHxazAKSRxQoQnHSvUdO9JuiqVmhNN5YRArGmEyxWPaNyhwQJWbLq+eoyvjjJAfSvOERkv390SKA6WSwDOdAdYTtV5bmP/V+rH2G27KRBRrKshqkR9zpEO0iACNmKRE88QAJpKZWxGZYImJNkEVTAjO+pc3oVOrOobvaqVmI4sjDxdwCRVwoA5NuIUWtIGAhGd4hTfryXqx3q2PVWvOymbO4Y+szx9iipG7</latexit><latexit sha1_base64="I7R4a32uvuL7gDjagUmI6UlCRsU=">AAAB9XicbZDLTsJAFIZP8YZ4Q126mQgmuCEtG1iSuHGJiVwSqGQ6TGFkOm1mppim4T3cuNAYt76LO9/GAbpQ8E8m+fKfc3LO/F7EmdK2/W3ltrZ3dvfy+4WDw6Pjk+LpWUeFsSS0TUIeyp6HFeVM0LZmmtNeJCkOPE673vRmUe/OqFQsFPc6iagb4LFgPiNYG+uhPJhhGU1YJRk+XpeHxZJdtZdCm+BkUIJMrWHxazAKSRxQoQnHSvUdO9JuiqVmhNN5YRArGmEyxWPaNyhwQJWbLq+eoyvjjJAfSvOERkv390SKA6WSwDOdAdYTtV5bmP/V+rH2G27KRBRrKshqkR9zpEO0iACNmKRE88QAJpKZWxGZYImJNkEVTAjO+pc3oVOrOobvaqVmI4sjDxdwCRVwoA5NuIUWtIGAhGd4hTfryXqx3q2PVWvOymbO4Y+szx9iipG7</latexit>

nonnegative combination

�(x) =

264'(x1)

...'(xn)

<latexit sha1_base64="guxvl0q2bU+xVoPoliEsRkzbL0M=">AAACNXicbZDLSgMxFIYzXmu9VV26CVbBbmSmG7sRCm5cuKhgL9CUksmctsFMZkgypWXoS7nxPVzpwoUibn0F04uirQcCH/9/Djnn92PBtXHdZ2dpeWV1bT2zkd3c2t7Zze3t13SUKAZVFolINXyqQXAJVcONgEasgIa+gLp/dzn2631Qmkfy1gxjaIW0K3mHM2qs1M5dH5O4x08HhQviQ5fL1A+pUXwwIn2qJk7bK2BCMOkHkdFT+nZkgYAMfkaO27m8e+ZOCi+CN4M8mlWlnXskQcSSEKRhgmrd9NzYtFKqDGcCRlmSaIgpu6NdaFqUNATdSidXj/CJVQLciZR90uCJ+nsipaHWw9C3nXbBnp73xuJ/XjMxnVIr5TJODEg2/aiTCGwiPI4QB1wBM2JogTLF7a6Y9aiizNigszYEb/7kRagVzzzLN8V8uTSLI4MO0RE6RR46R2V0hSqoihi6R0/oFb05D86L8+58TFuXnNnMAfpTzucX/a+riw==</latexit><latexit sha1_base64="guxvl0q2bU+xVoPoliEsRkzbL0M=">AAACNXicbZDLSgMxFIYzXmu9VV26CVbBbmSmG7sRCm5cuKhgL9CUksmctsFMZkgypWXoS7nxPVzpwoUibn0F04uirQcCH/9/Djnn92PBtXHdZ2dpeWV1bT2zkd3c2t7Zze3t13SUKAZVFolINXyqQXAJVcONgEasgIa+gLp/dzn2631Qmkfy1gxjaIW0K3mHM2qs1M5dH5O4x08HhQviQ5fL1A+pUXwwIn2qJk7bK2BCMOkHkdFT+nZkgYAMfkaO27m8e+ZOCi+CN4M8mlWlnXskQcSSEKRhgmrd9NzYtFKqDGcCRlmSaIgpu6NdaFqUNATdSidXj/CJVQLciZR90uCJ+nsipaHWw9C3nXbBnp73xuJ/XjMxnVIr5TJODEg2/aiTCGwiPI4QB1wBM2JogTLF7a6Y9aiizNigszYEb/7kRagVzzzLN8V8uTSLI4MO0RE6RR46R2V0hSqoihi6R0/oFb05D86L8+58TFuXnNnMAfpTzucX/a+riw==</latexit><latexit sha1_base64="guxvl0q2bU+xVoPoliEsRkzbL0M=">AAACNXicbZDLSgMxFIYzXmu9VV26CVbBbmSmG7sRCm5cuKhgL9CUksmctsFMZkgypWXoS7nxPVzpwoUibn0F04uirQcCH/9/Djnn92PBtXHdZ2dpeWV1bT2zkd3c2t7Zze3t13SUKAZVFolINXyqQXAJVcONgEasgIa+gLp/dzn2631Qmkfy1gxjaIW0K3mHM2qs1M5dH5O4x08HhQviQ5fL1A+pUXwwIn2qJk7bK2BCMOkHkdFT+nZkgYAMfkaO27m8e+ZOCi+CN4M8mlWlnXskQcSSEKRhgmrd9NzYtFKqDGcCRlmSaIgpu6NdaFqUNATdSidXj/CJVQLciZR90uCJ+nsipaHWw9C3nXbBnp73xuJ/XjMxnVIr5TJODEg2/aiTCGwiPI4QB1wBM2JogTLF7a6Y9aiizNigszYEb/7kRagVzzzLN8V8uTSLI4MO0RE6RR46R2V0hSqoihi6R0/oFb05D86L8+58TFuXnNnMAfpTzucX/a+riw==</latexit><latexit sha1_base64="guxvl0q2bU+xVoPoliEsRkzbL0M=">AAACNXicbZDLSgMxFIYzXmu9VV26CVbBbmSmG7sRCm5cuKhgL9CUksmctsFMZkgypWXoS7nxPVzpwoUibn0F04uirQcCH/9/Djnn92PBtXHdZ2dpeWV1bT2zkd3c2t7Zze3t13SUKAZVFolINXyqQXAJVcONgEasgIa+gLp/dzn2631Qmkfy1gxjaIW0K3mHM2qs1M5dH5O4x08HhQviQ5fL1A+pUXwwIn2qJk7bK2BCMOkHkdFT+nZkgYAMfkaO27m8e+ZOCi+CN4M8mlWlnXskQcSSEKRhgmrd9NzYtFKqDGcCRlmSaIgpu6NdaFqUNATdSidXj/CJVQLciZR90uCJ+nsipaHWw9C3nXbBnp73xuJ/XjMxnVIr5TJODEg2/aiTCGwiPI4QB1wBM2JogTLF7a6Y9aiizNigszYEb/7kRagVzzzLN8V8uTSLI4MO0RE6RR46R2V0hSqoihi6R0/oFb05D86L8+58TFuXnNnMAfpTzucX/a+riw==</latexit>

x � y

�(x) � �(y)

�> �2↵�T (↵ + �)T

(↵ + �)T �2T

� x � y

�(x) � �(y)

�� 0

<latexit sha1_base64="xE4TKIfpLj8g4gjosp4+9y8sOeo=">AAACtHiclVHBThsxEPVuaUtDW9Jy7MUiFCWqgnZzaDkicekRpIQgxSGddSaJhdfr2t4qq1W+sLfe+Jt6NxGCwIWRLD+9N288nkm0FNZF0V0Qvtp5/ebt7rvG3vsPH/ebnz5f2Sw3HAc8k5m5TsCiFAoHTjiJ19ogpInEYXJ7XunDP2isyFTfFRrHKcyVmAkOzlOT5t8jluBcqDJJwRmxXC27BWWMMr0Q7WWnW99Fh6Ga3qfcMJdpuuWj3R4DqRdQCQ5onx7T9pr5VjOdflV3mzr2vv6j6i/vh7I5/qbR0aTZik6iOuhTEG9Ai2ziYtL8x6YZz1NUjkuwdhRH2o1LME5wiasGyy1q4Lcwx5GHClK047Ie+op+9cyUzjLjj3K0Zh86SkitLdLEZ/o+F3Zbq8jntFHuZqfjUiidO1R8/dAsl9RltNognQqD3MnCA+BG+F4pX4AB7vyeG34I8faXn4Kr3kns8WWvdXa6Gccu+UIOSZvE5Ac5Iz/JBRkQHsTBMPgVQPg9ZCEPcZ0aBhvPAXkUofoPmdvT4g==</latexit><latexit sha1_base64="xE4TKIfpLj8g4gjosp4+9y8sOeo=">AAACtHiclVHBThsxEPVuaUtDW9Jy7MUiFCWqgnZzaDkicekRpIQgxSGddSaJhdfr2t4qq1W+sLfe+Jt6NxGCwIWRLD+9N288nkm0FNZF0V0Qvtp5/ebt7rvG3vsPH/ebnz5f2Sw3HAc8k5m5TsCiFAoHTjiJ19ogpInEYXJ7XunDP2isyFTfFRrHKcyVmAkOzlOT5t8jluBcqDJJwRmxXC27BWWMMr0Q7WWnW99Fh6Ga3qfcMJdpuuWj3R4DqRdQCQ5onx7T9pr5VjOdflV3mzr2vv6j6i/vh7I5/qbR0aTZik6iOuhTEG9Ai2ziYtL8x6YZz1NUjkuwdhRH2o1LME5wiasGyy1q4Lcwx5GHClK047Ie+op+9cyUzjLjj3K0Zh86SkitLdLEZ/o+F3Zbq8jntFHuZqfjUiidO1R8/dAsl9RltNognQqD3MnCA+BG+F4pX4AB7vyeG34I8faXn4Kr3kns8WWvdXa6Gccu+UIOSZvE5Ac5Iz/JBRkQHsTBMPgVQPg9ZCEPcZ0aBhvPAXkUofoPmdvT4g==</latexit><latexit sha1_base64="xE4TKIfpLj8g4gjosp4+9y8sOeo=">AAACtHiclVHBThsxEPVuaUtDW9Jy7MUiFCWqgnZzaDkicekRpIQgxSGddSaJhdfr2t4qq1W+sLfe+Jt6NxGCwIWRLD+9N288nkm0FNZF0V0Qvtp5/ebt7rvG3vsPH/ebnz5f2Sw3HAc8k5m5TsCiFAoHTjiJ19ogpInEYXJ7XunDP2isyFTfFRrHKcyVmAkOzlOT5t8jluBcqDJJwRmxXC27BWWMMr0Q7WWnW99Fh6Ga3qfcMJdpuuWj3R4DqRdQCQ5onx7T9pr5VjOdflV3mzr2vv6j6i/vh7I5/qbR0aTZik6iOuhTEG9Ai2ziYtL8x6YZz1NUjkuwdhRH2o1LME5wiasGyy1q4Lcwx5GHClK047Ie+op+9cyUzjLjj3K0Zh86SkitLdLEZ/o+F3Zbq8jntFHuZqfjUiidO1R8/dAsl9RltNognQqD3MnCA+BG+F4pX4AB7vyeG34I8faXn4Kr3kns8WWvdXa6Gccu+UIOSZvE5Ac5Iz/JBRkQHsTBMPgVQPg9ZCEPcZ0aBhvPAXkUofoPmdvT4g==</latexit><latexit sha1_base64="xE4TKIfpLj8g4gjosp4+9y8sOeo=">AAACtHiclVHBThsxEPVuaUtDW9Jy7MUiFCWqgnZzaDkicekRpIQgxSGddSaJhdfr2t4qq1W+sLfe+Jt6NxGCwIWRLD+9N288nkm0FNZF0V0Qvtp5/ebt7rvG3vsPH/ebnz5f2Sw3HAc8k5m5TsCiFAoHTjiJ19ogpInEYXJ7XunDP2isyFTfFRrHKcyVmAkOzlOT5t8jluBcqDJJwRmxXC27BWWMMr0Q7WWnW99Fh6Ga3qfcMJdpuuWj3R4DqRdQCQ5onx7T9pr5VjOdflV3mzr2vv6j6i/vh7I5/qbR0aTZik6iOuhTEG9Ai2ziYtL8x6YZz1NUjkuwdhRH2o1LME5wiasGyy1q4Lcwx5GHClK047Ie+op+9cyUzjLjj3K0Zh86SkitLdLEZ/o+F3Zbq8jntFHuZqfjUiidO1R8/dAsl9RltNognQqD3MnCA+BG+F4pX4AB7vyeG34I8faXn4Kr3kns8WWvdXa6Gccu+UIOSZvE5Ac5Iz/JBRkQHsTBMPgVQPg9ZCEPcZ0aBhvPAXkUofoPmdvT4g==</latexit>

Main Idea

I Original network with Lipschitz constant L2

Linear Linear

y1<latexit sha1_base64="eUbFnEfyWMjYTAUgCeR9bsRZVvk=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pJhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3S0TBWhPpFcql6INeVMUN8ww2kvURTHIafdcHo7r3efqNJMigeTJTSI8ViwiBFsrOXXs0evPqzW3Ia7EFoHr4AaFGoPq1+DkSRpTIUhHGvd99zEBDlWhhFOZ5VBqmmCyRSPad+iwDHVQb5YdoYurDNCkVT2CYMW7u+JHMdaZ3FoO2NsJnq1Njf/q/VTE10HORNJaqggy4+ilCMj0fxyNGKKEsMzC5goZndFZIIVJsbmU7EheKsnr0On2fAs3zdrrZsijjKcwTlcggdX0II7aIMPBBg8wyu8OcJ5cd6dj2VrySlmTuGPnM8fwmmN9g==</latexit><latexit sha1_base64="eUbFnEfyWMjYTAUgCeR9bsRZVvk=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pJhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3S0TBWhPpFcql6INeVMUN8ww2kvURTHIafdcHo7r3efqNJMigeTJTSI8ViwiBFsrOXXs0evPqzW3Ia7EFoHr4AaFGoPq1+DkSRpTIUhHGvd99zEBDlWhhFOZ5VBqmmCyRSPad+iwDHVQb5YdoYurDNCkVT2CYMW7u+JHMdaZ3FoO2NsJnq1Njf/q/VTE10HORNJaqggy4+ilCMj0fxyNGKKEsMzC5goZndFZIIVJsbmU7EheKsnr0On2fAs3zdrrZsijjKcwTlcggdX0II7aIMPBBg8wyu8OcJ5cd6dj2VrySlmTuGPnM8fwmmN9g==</latexit><latexit sha1_base64="eUbFnEfyWMjYTAUgCeR9bsRZVvk=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pJhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3S0TBWhPpFcql6INeVMUN8ww2kvURTHIafdcHo7r3efqNJMigeTJTSI8ViwiBFsrOXXs0evPqzW3Ia7EFoHr4AaFGoPq1+DkSRpTIUhHGvd99zEBDlWhhFOZ5VBqmmCyRSPad+iwDHVQb5YdoYurDNCkVT2CYMW7u+JHMdaZ3FoO2NsJnq1Njf/q/VTE10HORNJaqggy4+ilCMj0fxyNGKKEsMzC5goZndFZIIVJsbmU7EheKsnr0On2fAs3zdrrZsijjKcwTlcggdX0II7aIMPBBg8wyu8OcJ5cd6dj2VrySlmTuGPnM8fwmmN9g==</latexit><latexit sha1_base64="eUbFnEfyWMjYTAUgCeR9bsRZVvk=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pJhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3S0TBWhPpFcql6INeVMUN8ww2kvURTHIafdcHo7r3efqNJMigeTJTSI8ViwiBFsrOXXs0evPqzW3Ia7EFoHr4AaFGoPq1+DkSRpTIUhHGvd99zEBDlWhhFOZ5VBqmmCyRSPad+iwDHVQb5YdoYurDNCkVT2CYMW7u+JHMdaZ3FoO2NsJnq1Njf/q/VTE10HORNJaqggy4+ilCMj0fxyNGKKEsMzC5goZndFZIIVJsbmU7EheKsnr0On2fAs3zdrrZsijjKcwTlcggdX0II7aIMPBBg8wyu8OcJ5cd6dj2VrySlmTuGPnM8fwmmN9g==</latexit>

x1<latexit sha1_base64="GLHp/tQR2hCLFqJ7sxcm2CFPS9U=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04NX6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHwOKN9Q==</latexit><latexit sha1_base64="GLHp/tQR2hCLFqJ7sxcm2CFPS9U=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04NX6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHwOKN9Q==</latexit><latexit sha1_base64="GLHp/tQR2hCLFqJ7sxcm2CFPS9U=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04NX6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHwOKN9Q==</latexit><latexit sha1_base64="GLHp/tQR2hCLFqJ7sxcm2CFPS9U=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04NX6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHwOKN9Q==</latexit>x0

<latexit sha1_base64="C6tOYRQ7vOtaRX8BQQC0iQJnarU=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04Nb6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHv12N9A==</latexit><latexit sha1_base64="C6tOYRQ7vOtaRX8BQQC0iQJnarU=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04Nb6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHv12N9A==</latexit><latexit sha1_base64="C6tOYRQ7vOtaRX8BQQC0iQJnarU=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04Nb6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHv12N9A==</latexit><latexit sha1_base64="C6tOYRQ7vOtaRX8BQQC0iQJnarU=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04Nb6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHv12N9A==</latexit>

y2<latexit sha1_base64="yxxj+G8OPCSpuuevqkaSKw6jqu0=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pkhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3R0nCpCfRLzWPVCrClnkvqGGU57iaJYhJx2w+ntvN59okqzWD6YLKGBwGPJIkawsZZfzx6b9WG15jbchdA6eAXUoFB7WP0ajGKSCioN4VjrvucmJsixMoxwOqsMUk0TTKZ4TPsWJRZUB/li2Rm6sM4IRbGyTxq0cH9P5FhonYnQdgpsJnq1Njf/q/VTE10HOZNJaqgky4+ilCMTo/nlaMQUJYZnFjBRzO6KyAQrTIzNp2JD8FZPXodOs+FZvm/WWjdFHGU4g3O4BA+uoAV30AYfCDB4hld4c6Tz4rw7H8vWklPMnMIfOZ8/w+6N9w==</latexit><latexit sha1_base64="yxxj+G8OPCSpuuevqkaSKw6jqu0=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pkhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3R0nCpCfRLzWPVCrClnkvqGGU57iaJYhJx2w+ntvN59okqzWD6YLKGBwGPJIkawsZZfzx6b9WG15jbchdA6eAXUoFB7WP0ajGKSCioN4VjrvucmJsixMoxwOqsMUk0TTKZ4TPsWJRZUB/li2Rm6sM4IRbGyTxq0cH9P5FhonYnQdgpsJnq1Njf/q/VTE10HOZNJaqgky4+ilCMTo/nlaMQUJYZnFjBRzO6KyAQrTIzNp2JD8FZPXodOs+FZvm/WWjdFHGU4g3O4BA+uoAV30AYfCDB4hld4c6Tz4rw7H8vWklPMnMIfOZ8/w+6N9w==</latexit><latexit sha1_base64="yxxj+G8OPCSpuuevqkaSKw6jqu0=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pkhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3R0nCpCfRLzWPVCrClnkvqGGU57iaJYhJx2w+ntvN59okqzWD6YLKGBwGPJIkawsZZfzx6b9WG15jbchdA6eAXUoFB7WP0ajGKSCioN4VjrvucmJsixMoxwOqsMUk0TTKZ4TPsWJRZUB/li2Rm6sM4IRbGyTxq0cH9P5FhonYnQdgpsJnq1Njf/q/VTE10HOZNJaqgky4+ilCMTo/nlaMQUJYZnFjBRzO6KyAQrTIzNp2JD8FZPXodOs+FZvm/WWjdFHGU4g3O4BA+uoAV30AYfCDB4hld4c6Tz4rw7H8vWklPMnMIfOZ8/w+6N9w==</latexit><latexit sha1_base64="yxxj+G8OPCSpuuevqkaSKw6jqu0=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pkhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3R0nCpCfRLzWPVCrClnkvqGGU57iaJYhJx2w+ntvN59okqzWD6YLKGBwGPJIkawsZZfzx6b9WG15jbchdA6eAXUoFB7WP0ajGKSCioN4VjrvucmJsixMoxwOqsMUk0TTKZ4TPsWJRZUB/li2Rm6sM4IRbGyTxq0cH9P5FhonYnQdgpsJnq1Njf/q/VTE10HOZNJaqgky4+ilCMTo/nlaMQUJYZnFjBRzO6KyAQrTIzNp2JD8FZPXodOs+FZvm/WWjdFHGU4g3O4BA+uoAV30AYfCDB4hld4c6Tz4rw7H8vWklPMnMIfOZ8/w+6N9w==</latexit>

x0<latexit sha1_base64="9J+ynOBFN0XXGcAqpkKrLN982n8=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRguLgI1lBHOB7BpmZ88mQ2YvzMyKYclr2FgoYuvL2Pk2TpItNPGHgY//nMM58/up4Erb9re1tr6xubVd2inv7u0fHFaOjjsqySTDNktEIns+VSh4jG3NtcBeKpFGvsCuP76Z1buPKBVP4ns9SdGL6DDmIWdUG8utuZqLAMnTg10bVKp23Z6LrIJTQBUKtQaVLzdIWBZhrJmgSvUdO9VeTqXmTOC07GYKU8rGdIh9gzGNUHn5/OYpOTdOQMJEmhdrMnd/T+Q0UmoS+aYzonqklmsz879aP9PhlZfzOM00xmyxKMwE0QmZBUACLpFpMTFAmeTmVsJGVFKmTUxlE4Kz/OVV6DTqjuG7RrV5XcRRglM4gwtw4BKacAstaAODFJ7hFd6szHqx3q2PReuaVcycwB9Znz+0mZDG</latexit><latexit sha1_base64="9J+ynOBFN0XXGcAqpkKrLN982n8=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRguLgI1lBHOB7BpmZ88mQ2YvzMyKYclr2FgoYuvL2Pk2TpItNPGHgY//nMM58/up4Erb9re1tr6xubVd2inv7u0fHFaOjjsqySTDNktEIns+VSh4jG3NtcBeKpFGvsCuP76Z1buPKBVP4ns9SdGL6DDmIWdUG8utuZqLAMnTg10bVKp23Z6LrIJTQBUKtQaVLzdIWBZhrJmgSvUdO9VeTqXmTOC07GYKU8rGdIh9gzGNUHn5/OYpOTdOQMJEmhdrMnd/T+Q0UmoS+aYzonqklmsz879aP9PhlZfzOM00xmyxKMwE0QmZBUACLpFpMTFAmeTmVsJGVFKmTUxlE4Kz/OVV6DTqjuG7RrV5XcRRglM4gwtw4BKacAstaAODFJ7hFd6szHqx3q2PReuaVcycwB9Znz+0mZDG</latexit><latexit sha1_base64="9J+ynOBFN0XXGcAqpkKrLN982n8=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRguLgI1lBHOB7BpmZ88mQ2YvzMyKYclr2FgoYuvL2Pk2TpItNPGHgY//nMM58/up4Erb9re1tr6xubVd2inv7u0fHFaOjjsqySTDNktEIns+VSh4jG3NtcBeKpFGvsCuP76Z1buPKBVP4ns9SdGL6DDmIWdUG8utuZqLAMnTg10bVKp23Z6LrIJTQBUKtQaVLzdIWBZhrJmgSvUdO9VeTqXmTOC07GYKU8rGdIh9gzGNUHn5/OYpOTdOQMJEmhdrMnd/T+Q0UmoS+aYzonqklmsz879aP9PhlZfzOM00xmyxKMwE0QmZBUACLpFpMTFAmeTmVsJGVFKmTUxlE4Kz/OVV6DTqjuG7RrV5XcRRglM4gwtw4BKacAstaAODFJ7hFd6szHqx3q2PReuaVcycwB9Znz+0mZDG</latexit><latexit sha1_base64="9J+ynOBFN0XXGcAqpkKrLN982n8=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRguLgI1lBHOB7BpmZ88mQ2YvzMyKYclr2FgoYuvL2Pk2TpItNPGHgY//nMM58/up4Erb9re1tr6xubVd2inv7u0fHFaOjjsqySTDNktEIns+VSh4jG3NtcBeKpFGvsCuP76Z1buPKBVP4ns9SdGL6DDmIWdUG8utuZqLAMnTg10bVKp23Z6LrIJTQBUKtQaVLzdIWBZhrJmgSvUdO9VeTqXmTOC07GYKU8rGdIh9gzGNUHn5/OYpOTdOQMJEmhdrMnd/T+Q0UmoS+aYzonqklmsz879aP9PhlZfzOM00xmyxKMwE0QmZBUACLpFpMTFAmeTmVsJGVFKmTUxlE4Kz/OVV6DTqjuG7RrV5XcRRglM4gwtw4BKacAstaAODFJ7hFd6szHqx3q2PReuaVcycwB9Znz+0mZDG</latexit> y1

<latexit sha1_base64="JPSKS7ln6VHlWD+W3XtlejUo+3w=">AAAB83icbZC7SgNBFIZn4y3GW9TSZjARrMJuGq0kYGMZwVwgu4bZ2bPJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmeLQ4YlMVN9nGqSIoYMCJfRTBSzyJfT8ye283nsCpUUSP2CeghexUSxCwRkay627KGQANH906sNqzW7YC9F1cAqokULtYfXLDRKeRRAjl0zrgWOn6E2ZQsElzCpupiFlfMJGMDAYswi0N13cPKMXxglomCjzYqQL9/fElEVa55FvOiOGY71am5v/1QYZhtfeVMRphhDz5aIwkxQTOg+ABkIBR5kbYFwJcyvlY6YYRxNTxYTgrH55HbrNhmP4vllr3RRxlMkZOSeXxCFXpEXuSJt0CCcpeSav5M3KrBfr3fpYtpasYuaU/JH1+QO4P5DK</latexit><latexit sha1_base64="JPSKS7ln6VHlWD+W3XtlejUo+3w=">AAAB83icbZC7SgNBFIZn4y3GW9TSZjARrMJuGq0kYGMZwVwgu4bZ2bPJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmeLQ4YlMVN9nGqSIoYMCJfRTBSzyJfT8ye283nsCpUUSP2CeghexUSxCwRkay627KGQANH906sNqzW7YC9F1cAqokULtYfXLDRKeRRAjl0zrgWOn6E2ZQsElzCpupiFlfMJGMDAYswi0N13cPKMXxglomCjzYqQL9/fElEVa55FvOiOGY71am5v/1QYZhtfeVMRphhDz5aIwkxQTOg+ABkIBR5kbYFwJcyvlY6YYRxNTxYTgrH55HbrNhmP4vllr3RRxlMkZOSeXxCFXpEXuSJt0CCcpeSav5M3KrBfr3fpYtpasYuaU/JH1+QO4P5DK</latexit><latexit sha1_base64="JPSKS7ln6VHlWD+W3XtlejUo+3w=">AAAB83icbZC7SgNBFIZn4y3GW9TSZjARrMJuGq0kYGMZwVwgu4bZ2bPJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmeLQ4YlMVN9nGqSIoYMCJfRTBSzyJfT8ye283nsCpUUSP2CeghexUSxCwRkay627KGQANH906sNqzW7YC9F1cAqokULtYfXLDRKeRRAjl0zrgWOn6E2ZQsElzCpupiFlfMJGMDAYswi0N13cPKMXxglomCjzYqQL9/fElEVa55FvOiOGY71am5v/1QYZhtfeVMRphhDz5aIwkxQTOg+ABkIBR5kbYFwJcyvlY6YYRxNTxYTgrH55HbrNhmP4vllr3RRxlMkZOSeXxCFXpEXuSJt0CCcpeSav5M3KrBfr3fpYtpasYuaU/JH1+QO4P5DK</latexit><latexit sha1_base64="JPSKS7ln6VHlWD+W3XtlejUo+3w=">AAAB83icbZC7SgNBFIZn4y3GW9TSZjARrMJuGq0kYGMZwVwgu4bZ2bPJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmeLQ4YlMVN9nGqSIoYMCJfRTBSzyJfT8ye283nsCpUUSP2CeghexUSxCwRkay627KGQANH906sNqzW7YC9F1cAqokULtYfXLDRKeRRAjl0zrgWOn6E2ZQsElzCpupiFlfMJGMDAYswi0N13cPKMXxglomCjzYqQL9/fElEVa55FvOiOGY71am5v/1QYZhtfeVMRphhDz5aIwkxQTOg+ABkIBR5kbYFwJcyvlY6YYRxNTxYTgrH55HbrNhmP4vllr3RRxlMkZOSeXxCFXpEXuSJt0CCcpeSav5M3KrBfr3fpYtpasYuaU/JH1+QO4P5DK</latexit>

x1<latexit sha1_base64="SVxPp3MhBV5bWqUypPfvEiT2R0o=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRisJ2FhGMBfIrmF29mwyZPbCzKwYlryGjYUitr6MnW/jJNlCE38Y+PjPOZwzv58KrrRtf1tr6xubW9ulnfLu3v7BYeXouKOSTDJss0QksudThYLH2NZcC+ylEmnkC+z645tZvfuIUvEkvteTFL2IDmMecka1sdyaq7kIkDw9OLVBpWrX7bnIKjgFVKFQa1D5coOEZRHGmgmqVN+xU+3lVGrOBE7LbqYwpWxMh9g3GNMIlZfPb56Sc+MEJEykebEmc/f3RE4jpSaRbzojqkdquTYz/6v1Mx1eeTmP00xjzBaLwkwQnZBZACTgEpkWEwOUSW5uJWxEJWXaxFQ2ITjLX16FTqPuGL5rVJvXRRwlOIUzuAAHLqEJt9CCNjBI4Rle4c3KrBfr3fpYtK5ZxcwJ/JH1+QO2uJDJ</latexit><latexit sha1_base64="SVxPp3MhBV5bWqUypPfvEiT2R0o=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRisJ2FhGMBfIrmF29mwyZPbCzKwYlryGjYUitr6MnW/jJNlCE38Y+PjPOZwzv58KrrRtf1tr6xubW9ulnfLu3v7BYeXouKOSTDJss0QksudThYLH2NZcC+ylEmnkC+z645tZvfuIUvEkvteTFL2IDmMecka1sdyaq7kIkDw9OLVBpWrX7bnIKjgFVKFQa1D5coOEZRHGmgmqVN+xU+3lVGrOBE7LbqYwpWxMh9g3GNMIlZfPb56Sc+MEJEykebEmc/f3RE4jpSaRbzojqkdquTYz/6v1Mx1eeTmP00xjzBaLwkwQnZBZACTgEpkWEwOUSW5uJWxEJWXaxFQ2ITjLX16FTqPuGL5rVJvXRRwlOIUzuAAHLqEJt9CCNjBI4Rle4c3KrBfr3fpYtK5ZxcwJ/JH1+QO2uJDJ</latexit><latexit sha1_base64="SVxPp3MhBV5bWqUypPfvEiT2R0o=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRisJ2FhGMBfIrmF29mwyZPbCzKwYlryGjYUitr6MnW/jJNlCE38Y+PjPOZwzv58KrrRtf1tr6xubW9ulnfLu3v7BYeXouKOSTDJss0QksudThYLH2NZcC+ylEmnkC+z645tZvfuIUvEkvteTFL2IDmMecka1sdyaq7kIkDw9OLVBpWrX7bnIKjgFVKFQa1D5coOEZRHGmgmqVN+xU+3lVGrOBE7LbqYwpWxMh9g3GNMIlZfPb56Sc+MEJEykebEmc/f3RE4jpSaRbzojqkdquTYz/6v1Mx1eeTmP00xjzBaLwkwQnZBZACTgEpkWEwOUSW5uJWxEJWXaxFQ2ITjLX16FTqPuGL5rVJvXRRwlOIUzuAAHLqEJt9CCNjBI4Rle4c3KrBfr3fpYtK5ZxcwJ/JH1+QO2uJDJ</latexit><latexit sha1_base64="SVxPp3MhBV5bWqUypPfvEiT2R0o=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRisJ2FhGMBfIrmF29mwyZPbCzKwYlryGjYUitr6MnW/jJNlCE38Y+PjPOZwzv58KrrRtf1tr6xubW9ulnfLu3v7BYeXouKOSTDJss0QksudThYLH2NZcC+ylEmnkC+z645tZvfuIUvEkvteTFL2IDmMecka1sdyaq7kIkDw9OLVBpWrX7bnIKjgFVKFQa1D5coOEZRHGmgmqVN+xU+3lVGrOBE7LbqYwpWxMh9g3GNMIlZfPb56Sc+MEJEykebEmc/f3RE4jpSaRbzojqkdquTYz/6v1Mx1eeTmP00xjzBaLwkwQnZBZACTgEpkWEwOUSW5uJWxEJWXaxFQ2ITjLX16FTqPuGL5rVJvXRRwlOIUzuAAHLqEJt9CCNjBI4Rle4c3KrBfr3fpYtK5ZxcwJ/JH1+QO2uJDJ</latexit>

y2<latexit sha1_base64="65sCX0O1Vp7SoObv8xiFdvgtIZ4=">AAAB83icbZC7SgNBFIbPxluMt6ilzWAiWIXdNFpJwMYygrlAdg2zs7PJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmWK8wxKZqL5PNZci5h0UKHk/VZxGvuQ9f3I7r/eeuNIiiR8wT7kX0VEsQsEoGsutuyhkwEn+2KwPqzW7YS9E1sEpoAaF2sPqlxskLIt4jExSrQeOnaI3pQoFk3xWcTPNU8omdMQHBmMace1NFzfPyIVxAhImyrwYycL9PTGlkdZ55JvOiOJYr9bm5n+1QYbhtTcVcZohj9lyUZhJggmZB0ACoThDmRugTAlzK2FjqihDE1PFhOCsfnkdus2GY/i+WWvdFHGU4QzO4RIcuIIW3EEbOsAghWd4hTcrs16sd+tj2VqyiplT+CPr8we5xJDL</latexit><latexit sha1_base64="65sCX0O1Vp7SoObv8xiFdvgtIZ4=">AAAB83icbZC7SgNBFIbPxluMt6ilzWAiWIXdNFpJwMYygrlAdg2zs7PJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmWK8wxKZqL5PNZci5h0UKHk/VZxGvuQ9f3I7r/eeuNIiiR8wT7kX0VEsQsEoGsutuyhkwEn+2KwPqzW7YS9E1sEpoAaF2sPqlxskLIt4jExSrQeOnaI3pQoFk3xWcTPNU8omdMQHBmMace1NFzfPyIVxAhImyrwYycL9PTGlkdZ55JvOiOJYr9bm5n+1QYbhtTcVcZohj9lyUZhJggmZB0ACoThDmRugTAlzK2FjqihDE1PFhOCsfnkdus2GY/i+WWvdFHGU4QzO4RIcuIIW3EEbOsAghWd4hTcrs16sd+tj2VqyiplT+CPr8we5xJDL</latexit><latexit sha1_base64="65sCX0O1Vp7SoObv8xiFdvgtIZ4=">AAAB83icbZC7SgNBFIbPxluMt6ilzWAiWIXdNFpJwMYygrlAdg2zs7PJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmWK8wxKZqL5PNZci5h0UKHk/VZxGvuQ9f3I7r/eeuNIiiR8wT7kX0VEsQsEoGsutuyhkwEn+2KwPqzW7YS9E1sEpoAaF2sPqlxskLIt4jExSrQeOnaI3pQoFk3xWcTPNU8omdMQHBmMace1NFzfPyIVxAhImyrwYycL9PTGlkdZ55JvOiOJYr9bm5n+1QYbhtTcVcZohj9lyUZhJggmZB0ACoThDmRugTAlzK2FjqihDE1PFhOCsfnkdus2GY/i+WWvdFHGU4QzO4RIcuIIW3EEbOsAghWd4hTcrs16sd+tj2VqyiplT+CPr8we5xJDL</latexit><latexit sha1_base64="65sCX0O1Vp7SoObv8xiFdvgtIZ4=">AAAB83icbZC7SgNBFIbPxluMt6ilzWAiWIXdNFpJwMYygrlAdg2zs7PJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmWK8wxKZqL5PNZci5h0UKHk/VZxGvuQ9f3I7r/eeuNIiiR8wT7kX0VEsQsEoGsutuyhkwEn+2KwPqzW7YS9E1sEpoAaF2sPqlxskLIt4jExSrQeOnaI3pQoFk3xWcTPNU8omdMQHBmMace1NFzfPyIVxAhImyrwYycL9PTGlkdZ55JvOiOJYr9bm5n+1QYbhtTcVcZohj9lyUZhJggmZB0ACoThDmRugTAlzK2FjqihDE1PFhOCsfnkdus2GY/i+WWvdFHGU4QzO4RIcuIIW3EEbOsAghWd4hTcrs16sd+tj2VqyiplT+CPr8we5xJDL</latexit>

I Incrementally Quadratically Constrained Linear Network (IQCLN)

Linear Linear

x0<latexit sha1_base64="C6tOYRQ7vOtaRX8BQQC0iQJnarU=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04Nb6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHv12N9A==</latexit><latexit sha1_base64="C6tOYRQ7vOtaRX8BQQC0iQJnarU=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04Nb6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHv12N9A==</latexit><latexit sha1_base64="C6tOYRQ7vOtaRX8BQQC0iQJnarU=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04Nb6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHv12N9A==</latexit><latexit sha1_base64="C6tOYRQ7vOtaRX8BQQC0iQJnarU=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04Nb6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHv12N9A==</latexit>

y2<latexit sha1_base64="yxxj+G8OPCSpuuevqkaSKw6jqu0=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pkhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3R0nCpCfRLzWPVCrClnkvqGGU57iaJYhJx2w+ntvN59okqzWD6YLKGBwGPJIkawsZZfzx6b9WG15jbchdA6eAXUoFB7WP0ajGKSCioN4VjrvucmJsixMoxwOqsMUk0TTKZ4TPsWJRZUB/li2Rm6sM4IRbGyTxq0cH9P5FhonYnQdgpsJnq1Njf/q/VTE10HOZNJaqgky4+ilCMTo/nlaMQUJYZnFjBRzO6KyAQrTIzNp2JD8FZPXodOs+FZvm/WWjdFHGU4g3O4BA+uoAV30AYfCDB4hld4c6Tz4rw7H8vWklPMnMIfOZ8/w+6N9w==</latexit><latexit sha1_base64="yxxj+G8OPCSpuuevqkaSKw6jqu0=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pkhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3R0nCpCfRLzWPVCrClnkvqGGU57iaJYhJx2w+ntvN59okqzWD6YLKGBwGPJIkawsZZfzx6b9WG15jbchdA6eAXUoFB7WP0ajGKSCioN4VjrvucmJsixMoxwOqsMUk0TTKZ4TPsWJRZUB/li2Rm6sM4IRbGyTxq0cH9P5FhonYnQdgpsJnq1Njf/q/VTE10HOZNJaqgky4+ilCMTo/nlaMQUJYZnFjBRzO6KyAQrTIzNp2JD8FZPXodOs+FZvm/WWjdFHGU4g3O4BA+uoAV30AYfCDB4hld4c6Tz4rw7H8vWklPMnMIfOZ8/w+6N9w==</latexit><latexit sha1_base64="yxxj+G8OPCSpuuevqkaSKw6jqu0=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pkhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3R0nCpCfRLzWPVCrClnkvqGGU57iaJYhJx2w+ntvN59okqzWD6YLKGBwGPJIkawsZZfzx6b9WG15jbchdA6eAXUoFB7WP0ajGKSCioN4VjrvucmJsixMoxwOqsMUk0TTKZ4TPsWJRZUB/li2Rm6sM4IRbGyTxq0cH9P5FhonYnQdgpsJnq1Njf/q/VTE10HOZNJaqgky4+ilCMTo/nlaMQUJYZnFjBRzO6KyAQrTIzNp2JD8FZPXodOs+FZvm/WWjdFHGU4g3O4BA+uoAV30AYfCDB4hld4c6Tz4rw7H8vWklPMnMIfOZ8/w+6N9w==</latexit><latexit sha1_base64="yxxj+G8OPCSpuuevqkaSKw6jqu0=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pkhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3R0nCpCfRLzWPVCrClnkvqGGU57iaJYhJx2w+ntvN59okqzWD6YLKGBwGPJIkawsZZfzx6b9WG15jbchdA6eAXUoFB7WP0ajGKSCioN4VjrvucmJsixMoxwOqsMUk0TTKZ4TPsWJRZUB/li2Rm6sM4IRbGyTxq0cH9P5FhonYnQdgpsJnq1Njf/q/VTE10HOZNJaqgky4+ilCMTo/nlaMQUJYZnFjBRzO6KyAQrTIzNp2JD8FZPXodOs+FZvm/WWjdFHGU4g3O4BA+uoAV30AYfCDB4hld4c6Tz4rw7H8vWklPMnMIfOZ8/w+6N9w==</latexit>

x0<latexit sha1_base64="9J+ynOBFN0XXGcAqpkKrLN982n8=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRguLgI1lBHOB7BpmZ88mQ2YvzMyKYclr2FgoYuvL2Pk2TpItNPGHgY//nMM58/up4Erb9re1tr6xubVd2inv7u0fHFaOjjsqySTDNktEIns+VSh4jG3NtcBeKpFGvsCuP76Z1buPKBVP4ns9SdGL6DDmIWdUG8utuZqLAMnTg10bVKp23Z6LrIJTQBUKtQaVLzdIWBZhrJmgSvUdO9VeTqXmTOC07GYKU8rGdIh9gzGNUHn5/OYpOTdOQMJEmhdrMnd/T+Q0UmoS+aYzonqklmsz879aP9PhlZfzOM00xmyxKMwE0QmZBUACLpFpMTFAmeTmVsJGVFKmTUxlE4Kz/OVV6DTqjuG7RrV5XcRRglM4gwtw4BKacAstaAODFJ7hFd6szHqx3q2PReuaVcycwB9Znz+0mZDG</latexit><latexit sha1_base64="9J+ynOBFN0XXGcAqpkKrLN982n8=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRguLgI1lBHOB7BpmZ88mQ2YvzMyKYclr2FgoYuvL2Pk2TpItNPGHgY//nMM58/up4Erb9re1tr6xubVd2inv7u0fHFaOjjsqySTDNktEIns+VSh4jG3NtcBeKpFGvsCuP76Z1buPKBVP4ns9SdGL6DDmIWdUG8utuZqLAMnTg10bVKp23Z6LrIJTQBUKtQaVLzdIWBZhrJmgSvUdO9VeTqXmTOC07GYKU8rGdIh9gzGNUHn5/OYpOTdOQMJEmhdrMnd/T+Q0UmoS+aYzonqklmsz879aP9PhlZfzOM00xmyxKMwE0QmZBUACLpFpMTFAmeTmVsJGVFKmTUxlE4Kz/OVV6DTqjuG7RrV5XcRRglM4gwtw4BKacAstaAODFJ7hFd6szHqx3q2PReuaVcycwB9Znz+0mZDG</latexit><latexit sha1_base64="9J+ynOBFN0XXGcAqpkKrLN982n8=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRguLgI1lBHOB7BpmZ88mQ2YvzMyKYclr2FgoYuvL2Pk2TpItNPGHgY//nMM58/up4Erb9re1tr6xubVd2inv7u0fHFaOjjsqySTDNktEIns+VSh4jG3NtcBeKpFGvsCuP76Z1buPKBVP4ns9SdGL6DDmIWdUG8utuZqLAMnTg10bVKp23Z6LrIJTQBUKtQaVLzdIWBZhrJmgSvUdO9VeTqXmTOC07GYKU8rGdIh9gzGNUHn5/OYpOTdOQMJEmhdrMnd/T+Q0UmoS+aYzonqklmsz879aP9PhlZfzOM00xmyxKMwE0QmZBUACLpFpMTFAmeTmVsJGVFKmTUxlE4Kz/OVV6DTqjuG7RrV5XcRRglM4gwtw4BKacAstaAODFJ7hFd6szHqx3q2PReuaVcycwB9Znz+0mZDG</latexit><latexit sha1_base64="9J+ynOBFN0XXGcAqpkKrLN982n8=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRguLgI1lBHOB7BpmZ88mQ2YvzMyKYclr2FgoYuvL2Pk2TpItNPGHgY//nMM58/up4Erb9re1tr6xubVd2inv7u0fHFaOjjsqySTDNktEIns+VSh4jG3NtcBeKpFGvsCuP76Z1buPKBVP4ns9SdGL6DDmIWdUG8utuZqLAMnTg10bVKp23Z6LrIJTQBUKtQaVLzdIWBZhrJmgSvUdO9VeTqXmTOC07GYKU8rGdIh9gzGNUHn5/OYpOTdOQMJEmhdrMnd/T+Q0UmoS+aYzonqklmsz879aP9PhlZfzOM00xmyxKMwE0QmZBUACLpFpMTFAmeTmVsJGVFKmTUxlE4Kz/OVV6DTqjuG7RrV5XcRRglM4gwtw4BKacAstaAODFJ7hFd6szHqx3q2PReuaVcycwB9Znz+0mZDG</latexit>

y2<latexit sha1_base64="65sCX0O1Vp7SoObv8xiFdvgtIZ4=">AAAB83icbZC7SgNBFIbPxluMt6ilzWAiWIXdNFpJwMYygrlAdg2zs7PJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmWK8wxKZqL5PNZci5h0UKHk/VZxGvuQ9f3I7r/eeuNIiiR8wT7kX0VEsQsEoGsutuyhkwEn+2KwPqzW7YS9E1sEpoAaF2sPqlxskLIt4jExSrQeOnaI3pQoFk3xWcTPNU8omdMQHBmMace1NFzfPyIVxAhImyrwYycL9PTGlkdZ55JvOiOJYr9bm5n+1QYbhtTcVcZohj9lyUZhJggmZB0ACoThDmRugTAlzK2FjqihDE1PFhOCsfnkdus2GY/i+WWvdFHGU4QzO4RIcuIIW3EEbOsAghWd4hTcrs16sd+tj2VqyiplT+CPr8we5xJDL</latexit><latexit sha1_base64="65sCX0O1Vp7SoObv8xiFdvgtIZ4=">AAAB83icbZC7SgNBFIbPxluMt6ilzWAiWIXdNFpJwMYygrlAdg2zs7PJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmWK8wxKZqL5PNZci5h0UKHk/VZxGvuQ9f3I7r/eeuNIiiR8wT7kX0VEsQsEoGsutuyhkwEn+2KwPqzW7YS9E1sEpoAaF2sPqlxskLIt4jExSrQeOnaI3pQoFk3xWcTPNU8omdMQHBmMace1NFzfPyIVxAhImyrwYycL9PTGlkdZ55JvOiOJYr9bm5n+1QYbhtTcVcZohj9lyUZhJggmZB0ACoThDmRugTAlzK2FjqihDE1PFhOCsfnkdus2GY/i+WWvdFHGU4QzO4RIcuIIW3EEbOsAghWd4hTcrs16sd+tj2VqyiplT+CPr8we5xJDL</latexit><latexit sha1_base64="65sCX0O1Vp7SoObv8xiFdvgtIZ4=">AAAB83icbZC7SgNBFIbPxluMt6ilzWAiWIXdNFpJwMYygrlAdg2zs7PJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmWK8wxKZqL5PNZci5h0UKHk/VZxGvuQ9f3I7r/eeuNIiiR8wT7kX0VEsQsEoGsutuyhkwEn+2KwPqzW7YS9E1sEpoAaF2sPqlxskLIt4jExSrQeOnaI3pQoFk3xWcTPNU8omdMQHBmMace1NFzfPyIVxAhImyrwYycL9PTGlkdZ55JvOiOJYr9bm5n+1QYbhtTcVcZohj9lyUZhJggmZB0ACoThDmRugTAlzK2FjqihDE1PFhOCsfnkdus2GY/i+WWvdFHGU4QzO4RIcuIIW3EEbOsAghWd4hTcrs16sd+tj2VqyiplT+CPr8we5xJDL</latexit><latexit sha1_base64="65sCX0O1Vp7SoObv8xiFdvgtIZ4=">AAAB83icbZC7SgNBFIbPxluMt6ilzWAiWIXdNFpJwMYygrlAdg2zs7PJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmWK8wxKZqL5PNZci5h0UKHk/VZxGvuQ9f3I7r/eeuNIiiR8wT7kX0VEsQsEoGsutuyhkwEn+2KwPqzW7YS9E1sEpoAaF2sPqlxskLIt4jExSrQeOnaI3pQoFk3xWcTPNU8omdMQHBmMace1NFzfPyIVxAhImyrwYycL9PTGlkdZ55JvOiOJYr9bm5n+1QYbhtTcVcZohj9lyUZhJggmZB0ACoThDmRugTAlzK2FjqihDE1PFhOCsfnkdus2GY/i+WWvdFHGU4QzO4RIcuIIW3EEbOsAghWd4hTcrs16sd+tj2VqyiplT+CPr8we5xJDL</latexit>

y1<latexit sha1_base64="eUbFnEfyWMjYTAUgCeR9bsRZVvk=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pJhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3S0TBWhPpFcql6INeVMUN8ww2kvURTHIafdcHo7r3efqNJMigeTJTSI8ViwiBFsrOXXs0evPqzW3Ia7EFoHr4AaFGoPq1+DkSRpTIUhHGvd99zEBDlWhhFOZ5VBqmmCyRSPad+iwDHVQb5YdoYurDNCkVT2CYMW7u+JHMdaZ3FoO2NsJnq1Njf/q/VTE10HORNJaqggy4+ilCMj0fxyNGKKEsMzC5goZndFZIIVJsbmU7EheKsnr0On2fAs3zdrrZsijjKcwTlcggdX0II7aIMPBBg8wyu8OcJ5cd6dj2VrySlmTuGPnM8fwmmN9g==</latexit><latexit sha1_base64="eUbFnEfyWMjYTAUgCeR9bsRZVvk=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pJhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3S0TBWhPpFcql6INeVMUN8ww2kvURTHIafdcHo7r3efqNJMigeTJTSI8ViwiBFsrOXXs0evPqzW3Ia7EFoHr4AaFGoPq1+DkSRpTIUhHGvd99zEBDlWhhFOZ5VBqmmCyRSPad+iwDHVQb5YdoYurDNCkVT2CYMW7u+JHMdaZ3FoO2NsJnq1Njf/q/VTE10HORNJaqggy4+ilCMj0fxyNGKKEsMzC5goZndFZIIVJsbmU7EheKsnr0On2fAs3zdrrZsijjKcwTlcggdX0II7aIMPBBg8wyu8OcJ5cd6dj2VrySlmTuGPnM8fwmmN9g==</latexit><latexit sha1_base64="eUbFnEfyWMjYTAUgCeR9bsRZVvk=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pJhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3S0TBWhPpFcql6INeVMUN8ww2kvURTHIafdcHo7r3efqNJMigeTJTSI8ViwiBFsrOXXs0evPqzW3Ia7EFoHr4AaFGoPq1+DkSRpTIUhHGvd99zEBDlWhhFOZ5VBqmmCyRSPad+iwDHVQb5YdoYurDNCkVT2CYMW7u+JHMdaZ3FoO2NsJnq1Njf/q/VTE10HORNJaqggy4+ilCMj0fxyNGKKEsMzC5goZndFZIIVJsbmU7EheKsnr0On2fAs3zdrrZsijjKcwTlcggdX0II7aIMPBBg8wyu8OcJ5cd6dj2VrySlmTuGPnM8fwmmN9g==</latexit><latexit sha1_base64="eUbFnEfyWMjYTAUgCeR9bsRZVvk=">AAAB7HicbZDNSgMxFIXv1L9a/6ou3QRbwVWZ6UZXUnDjsoLTFtqxZNJMG5pJhiQjDEOfwY0LRdz6QO58G9N2Ftp6IPBx7r3k3hMmnGnjut9OaWNza3unvFvZ2z84PKoen3S0TBWhPpFcql6INeVMUN8ww2kvURTHIafdcHo7r3efqNJMigeTJTSI8ViwiBFsrOXXs0evPqzW3Ia7EFoHr4AaFGoPq1+DkSRpTIUhHGvd99zEBDlWhhFOZ5VBqmmCyRSPad+iwDHVQb5YdoYurDNCkVT2CYMW7u+JHMdaZ3FoO2NsJnq1Njf/q/VTE10HORNJaqggy4+ilCMj0fxyNGKKEsMzC5goZndFZIIVJsbmU7EheKsnr0On2fAs3zdrrZsijjKcwTlcggdX0II7aIMPBBg8wyu8OcJ5cd6dj2VrySlmTuGPnM8fwmmN9g==</latexit>

y1<latexit sha1_base64="JPSKS7ln6VHlWD+W3XtlejUo+3w=">AAAB83icbZC7SgNBFIZn4y3GW9TSZjARrMJuGq0kYGMZwVwgu4bZ2bPJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmeLQ4YlMVN9nGqSIoYMCJfRTBSzyJfT8ye283nsCpUUSP2CeghexUSxCwRkay627KGQANH906sNqzW7YC9F1cAqokULtYfXLDRKeRRAjl0zrgWOn6E2ZQsElzCpupiFlfMJGMDAYswi0N13cPKMXxglomCjzYqQL9/fElEVa55FvOiOGY71am5v/1QYZhtfeVMRphhDz5aIwkxQTOg+ABkIBR5kbYFwJcyvlY6YYRxNTxYTgrH55HbrNhmP4vllr3RRxlMkZOSeXxCFXpEXuSJt0CCcpeSav5M3KrBfr3fpYtpasYuaU/JH1+QO4P5DK</latexit><latexit sha1_base64="JPSKS7ln6VHlWD+W3XtlejUo+3w=">AAAB83icbZC7SgNBFIZn4y3GW9TSZjARrMJuGq0kYGMZwVwgu4bZ2bPJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmeLQ4YlMVN9nGqSIoYMCJfRTBSzyJfT8ye283nsCpUUSP2CeghexUSxCwRkay627KGQANH906sNqzW7YC9F1cAqokULtYfXLDRKeRRAjl0zrgWOn6E2ZQsElzCpupiFlfMJGMDAYswi0N13cPKMXxglomCjzYqQL9/fElEVa55FvOiOGY71am5v/1QYZhtfeVMRphhDz5aIwkxQTOg+ABkIBR5kbYFwJcyvlY6YYRxNTxYTgrH55HbrNhmP4vllr3RRxlMkZOSeXxCFXpEXuSJt0CCcpeSav5M3KrBfr3fpYtpasYuaU/JH1+QO4P5DK</latexit><latexit sha1_base64="JPSKS7ln6VHlWD+W3XtlejUo+3w=">AAAB83icbZC7SgNBFIZn4y3GW9TSZjARrMJuGq0kYGMZwVwgu4bZ2bPJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmeLQ4YlMVN9nGqSIoYMCJfRTBSzyJfT8ye283nsCpUUSP2CeghexUSxCwRkay627KGQANH906sNqzW7YC9F1cAqokULtYfXLDRKeRRAjl0zrgWOn6E2ZQsElzCpupiFlfMJGMDAYswi0N13cPKMXxglomCjzYqQL9/fElEVa55FvOiOGY71am5v/1QYZhtfeVMRphhDz5aIwkxQTOg+ABkIBR5kbYFwJcyvlY6YYRxNTxYTgrH55HbrNhmP4vllr3RRxlMkZOSeXxCFXpEXuSJt0CCcpeSav5M3KrBfr3fpYtpasYuaU/JH1+QO4P5DK</latexit><latexit sha1_base64="JPSKS7ln6VHlWD+W3XtlejUo+3w=">AAAB83icbZC7SgNBFIZn4y3GW9TSZjARrMJuGq0kYGMZwVwgu4bZ2bPJkNkLM2eFJeQ1bCwUsfVl7HwbJ8kWmvjDwMd/zuGc+f1UCo22/W2VNja3tnfKu5W9/YPDo+rxSVcnmeLQ4YlMVN9nGqSIoYMCJfRTBSzyJfT8ye283nsCpUUSP2CeghexUSxCwRkay627KGQANH906sNqzW7YC9F1cAqokULtYfXLDRKeRRAjl0zrgWOn6E2ZQsElzCpupiFlfMJGMDAYswi0N13cPKMXxglomCjzYqQL9/fElEVa55FvOiOGY71am5v/1QYZhtfeVMRphhDz5aIwkxQTOg+ABkIBR5kbYFwJcyvlY6YYRxNTxYTgrH55HbrNhmP4vllr3RRxlMkZOSeXxCFXpEXuSJt0CCcpeSav5M3KrBfr3fpYtpasYuaU/JH1+QO4P5DK</latexit>

x1<latexit sha1_base64="GLHp/tQR2hCLFqJ7sxcm2CFPS9U=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04NX6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHwOKN9Q==</latexit><latexit sha1_base64="GLHp/tQR2hCLFqJ7sxcm2CFPS9U=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04NX6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHwOKN9Q==</latexit><latexit sha1_base64="GLHp/tQR2hCLFqJ7sxcm2CFPS9U=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04NX6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHwOKN9Q==</latexit><latexit sha1_base64="GLHp/tQR2hCLFqJ7sxcm2CFPS9U=">AAAB7HicbZA9TwJBEIbn/ET8Qi1tNoKJFbmj0cqQ2Fhi4gEJnGRvWWDD3t5ld85ILvwGGwuNsfUH2flvXOAKBd9kkyfvzGRn3jCRwqDrfjtr6xubW9uFneLu3v7BYenouGniVDPus1jGuh1Sw6VQ3EeBkrcTzWkUSt4KxzezeuuRayNidY+ThAcRHSoxEIyitfzK04NX6ZXKbtWdi6yCl0MZcjV6pa9uP2ZpxBUySY3peG6CQUY1Cib5tNhNDU8oG9Mh71hUNOImyObLTsm5dfpkEGv7FJK5+3sio5Exkyi0nRHFkVmuzcz/ap0UB1dBJlSSIlds8dEglQRjMruc9IXmDOXEAmVa2F0JG1FNGdp8ijYEb/nkVWjWqp7lu1q5fp3HUYBTOIML8OAS6nALDfCBgYBneIU3RzkvzrvzsWhdc/KZE/gj5/MHwOKN9Q==</latexit>

x1<latexit sha1_base64="SVxPp3MhBV5bWqUypPfvEiT2R0o=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRisJ2FhGMBfIrmF29mwyZPbCzKwYlryGjYUitr6MnW/jJNlCE38Y+PjPOZwzv58KrrRtf1tr6xubW9ulnfLu3v7BYeXouKOSTDJss0QksudThYLH2NZcC+ylEmnkC+z645tZvfuIUvEkvteTFL2IDmMecka1sdyaq7kIkDw9OLVBpWrX7bnIKjgFVKFQa1D5coOEZRHGmgmqVN+xU+3lVGrOBE7LbqYwpWxMh9g3GNMIlZfPb56Sc+MEJEykebEmc/f3RE4jpSaRbzojqkdquTYz/6v1Mx1eeTmP00xjzBaLwkwQnZBZACTgEpkWEwOUSW5uJWxEJWXaxFQ2ITjLX16FTqPuGL5rVJvXRRwlOIUzuAAHLqEJt9CCNjBI4Rle4c3KrBfr3fpYtK5ZxcwJ/JH1+QO2uJDJ</latexit><latexit sha1_base64="SVxPp3MhBV5bWqUypPfvEiT2R0o=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRisJ2FhGMBfIrmF29mwyZPbCzKwYlryGjYUitr6MnW/jJNlCE38Y+PjPOZwzv58KrrRtf1tr6xubW9ulnfLu3v7BYeXouKOSTDJss0QksudThYLH2NZcC+ylEmnkC+z645tZvfuIUvEkvteTFL2IDmMecka1sdyaq7kIkDw9OLVBpWrX7bnIKjgFVKFQa1D5coOEZRHGmgmqVN+xU+3lVGrOBE7LbqYwpWxMh9g3GNMIlZfPb56Sc+MEJEykebEmc/f3RE4jpSaRbzojqkdquTYz/6v1Mx1eeTmP00xjzBaLwkwQnZBZACTgEpkWEwOUSW5uJWxEJWXaxFQ2ITjLX16FTqPuGL5rVJvXRRwlOIUzuAAHLqEJt9CCNjBI4Rle4c3KrBfr3fpYtK5ZxcwJ/JH1+QO2uJDJ</latexit><latexit sha1_base64="SVxPp3MhBV5bWqUypPfvEiT2R0o=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRisJ2FhGMBfIrmF29mwyZPbCzKwYlryGjYUitr6MnW/jJNlCE38Y+PjPOZwzv58KrrRtf1tr6xubW9ulnfLu3v7BYeXouKOSTDJss0QksudThYLH2NZcC+ylEmnkC+z645tZvfuIUvEkvteTFL2IDmMecka1sdyaq7kIkDw9OLVBpWrX7bnIKjgFVKFQa1D5coOEZRHGmgmqVN+xU+3lVGrOBE7LbqYwpWxMh9g3GNMIlZfPb56Sc+MEJEykebEmc/f3RE4jpSaRbzojqkdquTYz/6v1Mx1eeTmP00xjzBaLwkwQnZBZACTgEpkWEwOUSW5uJWxEJWXaxFQ2ITjLX16FTqPuGL5rVJvXRRwlOIUzuAAHLqEJt9CCNjBI4Rle4c3KrBfr3fpYtK5ZxcwJ/JH1+QO2uJDJ</latexit><latexit sha1_base64="SVxPp3MhBV5bWqUypPfvEiT2R0o=">AAAB83icbZC7SgNBFIbPeo3xFrW0GUwEq7CbRisJ2FhGMBfIrmF29mwyZPbCzKwYlryGjYUitr6MnW/jJNlCE38Y+PjPOZwzv58KrrRtf1tr6xubW9ulnfLu3v7BYeXouKOSTDJss0QksudThYLH2NZcC+ylEmnkC+z645tZvfuIUvEkvteTFL2IDmMecka1sdyaq7kIkDw9OLVBpWrX7bnIKjgFVKFQa1D5coOEZRHGmgmqVN+xU+3lVGrOBE7LbqYwpWxMh9g3GNMIlZfPb56Sc+MEJEykebEmc/f3RE4jpSaRbzojqkdquTYz/6v1Mx1eeTmP00xjzBaLwkwQnZBZACTgEpkWEwOUSW5uJWxEJWXaxFQ2ITjLX16FTqPuGL5rVJvXRRwlOIUzuAAHLqEJt9CCNjBI4Rle4c3KrBfr3fpYtK5ZxcwJ/JH1+QO2uJDJ</latexit>

y1 � y1

x1 � x1

y1 � y1

x1 � x1

�� 0

<latexit sha1_base64="1stDpQy7SxKa9NLTSeAdhEKIWLI=">AAACd3icpZE9T8MwEIad8FXCV4ENBixaEAsoYYEJIbEwFokCUl0qx7kWC8cJ9gVRRf0L/Dg2/gcLG26p+N44ydKj996zz3dxrqTFMHz2/InJqemZymwwN7+wuFRdXrmwWWEENEWmMnMVcwtKamiiRAVXuQGexgou49uTYf7yHoyVmT7Hfg7tlPe07ErB0Umd6mOdxdCTuoxTjkY+DIL+dbTLUKoEqEPKGH34VBwGDHTyYb9mmOX0jP7vFsp6cEfDoN6p1sK9cBT0N0RjqJFxNDrVJ5ZkokhBo1Dc2lYU5tguuUEpFAwCVljIubjlPWg51DwF2y5HcxvQLacktJsZdzTSkfq1ouSptf00dk7X6I39mRuKf+VaBXYP26XUeYGgxftD3UJRzOhwCTSRBgSqvgMujHS9UnHDDRfoVhW4IUQ/v/wbLvb3Isdn+7Xjo/E4KmSdbJIdEpEDckxOSYM0iSAv3ppX8+req7/hb/s771bfG9eskm/hR2/5qr0V</latexit><latexit sha1_base64="1stDpQy7SxKa9NLTSeAdhEKIWLI=">AAACd3icpZE9T8MwEIad8FXCV4ENBixaEAsoYYEJIbEwFokCUl0qx7kWC8cJ9gVRRf0L/Dg2/gcLG26p+N44ydKj996zz3dxrqTFMHz2/InJqemZymwwN7+wuFRdXrmwWWEENEWmMnMVcwtKamiiRAVXuQGexgou49uTYf7yHoyVmT7Hfg7tlPe07ErB0Umd6mOdxdCTuoxTjkY+DIL+dbTLUKoEqEPKGH34VBwGDHTyYb9mmOX0jP7vFsp6cEfDoN6p1sK9cBT0N0RjqJFxNDrVJ5ZkokhBo1Dc2lYU5tguuUEpFAwCVljIubjlPWg51DwF2y5HcxvQLacktJsZdzTSkfq1ouSptf00dk7X6I39mRuKf+VaBXYP26XUeYGgxftD3UJRzOhwCTSRBgSqvgMujHS9UnHDDRfoVhW4IUQ/v/wbLvb3Isdn+7Xjo/E4KmSdbJIdEpEDckxOSYM0iSAv3ppX8+req7/hb/s771bfG9eskm/hR2/5qr0V</latexit><latexit sha1_base64="1stDpQy7SxKa9NLTSeAdhEKIWLI=">AAACd3icpZE9T8MwEIad8FXCV4ENBixaEAsoYYEJIbEwFokCUl0qx7kWC8cJ9gVRRf0L/Dg2/gcLG26p+N44ydKj996zz3dxrqTFMHz2/InJqemZymwwN7+wuFRdXrmwWWEENEWmMnMVcwtKamiiRAVXuQGexgou49uTYf7yHoyVmT7Hfg7tlPe07ErB0Umd6mOdxdCTuoxTjkY+DIL+dbTLUKoEqEPKGH34VBwGDHTyYb9mmOX0jP7vFsp6cEfDoN6p1sK9cBT0N0RjqJFxNDrVJ5ZkokhBo1Dc2lYU5tguuUEpFAwCVljIubjlPWg51DwF2y5HcxvQLacktJsZdzTSkfq1ouSptf00dk7X6I39mRuKf+VaBXYP26XUeYGgxftD3UJRzOhwCTSRBgSqvgMujHS9UnHDDRfoVhW4IUQ/v/wbLvb3Isdn+7Xjo/E4KmSdbJIdEpEDckxOSYM0iSAv3ppX8+req7/hb/s771bfG9eskm/hR2/5qr0V</latexit><latexit sha1_base64="1stDpQy7SxKa9NLTSeAdhEKIWLI=">AAACd3icpZE9T8MwEIad8FXCV4ENBixaEAsoYYEJIbEwFokCUl0qx7kWC8cJ9gVRRf0L/Dg2/gcLG26p+N44ydKj996zz3dxrqTFMHz2/InJqemZymwwN7+wuFRdXrmwWWEENEWmMnMVcwtKamiiRAVXuQGexgou49uTYf7yHoyVmT7Hfg7tlPe07ErB0Umd6mOdxdCTuoxTjkY+DIL+dbTLUKoEqEPKGH34VBwGDHTyYb9mmOX0jP7vFsp6cEfDoN6p1sK9cBT0N0RjqJFxNDrVJ5ZkokhBo1Dc2lYU5tguuUEpFAwCVljIubjlPWg51DwF2y5HcxvQLacktJsZdzTSkfq1ouSptf00dk7X6I39mRuKf+VaBXYP26XUeYGgxftD3UJRzOhwCTSRBgSqvgMujHS9UnHDDRfoVhW4IUQ/v/wbLvb3Isdn+7Xjo/E4KmSdbJIdEpEDckxOSYM0iSAv3ppX8+req7/hb/s771bfG9eskm/hR2/5qr0V</latexit>

infQ∈Q

LIQCLN2 (Q) ≥ L2 (1)

I Main result: we can pose (1) as a semidefinite program

Main Result

Theorem 1: LipSDP for one-layer neural networks

Consider a single-layer neural network described by f(x) =W 1φ(W 0x+b0)+b1. Supposeφ(x) = [ϕ(x1) · · ·ϕ(xn)], where ϕ is slope-restricted on [α, β]. Suppose there exists aρ > 0 such that the matrix inequality

M(ρ, T ) :=

[−2αβW 0>TW 0 − ρIn0 (α+ β)W 0>T

(α+ β)TW 0 −2T +W 1>W 1

]� 0,

holds for some T ∈ Tn. Then ‖f(x)− f(y)‖2 ≤ √ρ‖x− y‖2 for all x, y ∈ Rn0 .

A semidefinite program for finding the best upper bound:

minimize ρ subject to M(ρ, T ) � 0, T ∈ Tn. (2)

Multi-layer Case

x0 = x, xk+1 = φ(W kxk + bk) k = 0, · · · , `− 1, f(x) =W `x` + b`. (3)

I Introduce x = [x0> · · ·x`>]>. We can write (3) compactly as

x = Dx, Bx = φ(Ax+ b), f(x) = Cx+ b`,

W 0 0 · · · 0 00 W 1 · · · 0 0...

.... . .

......

0 0 · · · W `−1 0

...b`−1

0 In1 · · · 0 0...

.... . .

......

0 0 · · · In`−1 00 0 · · · 0 In`

[0 0 · · · 0 W `

[In0 0 · · · 0 0

Lipschitz Certificates for Multi-layer Neural Networks

I For multi-layer neural networks

M(ρ, T )=

]> [ −2αβT (α+ β)T(α+ β)T −2T

−ρI0 0 . . . 00 0 . . . 0...

.... . .

...0 0 . . . (W `)>W `

I We can find the smallest ρ by semidefinite programming!

minimize ρ subject to M(ρ, T ) � 0 and T ∈ Tn,

Complexity Analysis

I Parameters: nx input dimension, n total number of neurons, ` number of layers

I Number of decision variables:

– LipSDP-Network imposes constraints on all possible pairs of activation functions and has O(n2)decision variables. It is the least scalable but the most accurate method.

– LipSDP-Neuron ignores the cross coupling constraints among different neurons and has O(n)decision variables. It is more scalable and less accurate than LipSDP-Network. For this case, wehave T = diag(λ11, · · · , λnn).

– LipSDP-Layer considers only one constraint per layer, resulting in O(`) decision variables. It isthe most scalable and least accurate method. For this variant, we haveT = blkdiag(λ1In1 , · · · , λ`In`).

I A hierarchy of SDP relaxations

I Size of the PSD cone: nx + n

Experiments

I Platform: MATLAB, CVX toolbox, and MOSEK on a 9-core CPU with 16GB of RAM

I Methods:

– LipSDP:I LipSDP-NetworkI LipSDP-NeuronI LipSDP-Layer

– CPLip: Combettes, Patrick L., and Jean-Christophe Pesquet. ”Lipschitz Certificates for NeuralNetwork Structures Driven by Averaged Activation Operators.”arXiv preprintarXiv:1903.01014(2019).

– SeqLip: Virmaux, Aladin, and Kevin Scaman. ”Lipschitz regularity of deep neural networks:analysis and efficient estimation.”Advances in Neural Information Processing Systems. 2018.

Experiments

I Training Methods: two robust and one standard training procedures

– Adam: Kingma, Diederik P., and Jimmy Ba. ”Adam: A method for stochasticoptimization.”arXiv preprint arXiv:1412.6980(2014).

– LP-Train: Wong, Eric, and J. Zico Kolter. ”Provable defenses against adversarial examples viathe convex outer adversarial polytope.”arXiv preprint arXiv:1711.00851(2017).

– PGD-Train: Madry, Aleksander, et al. ”Towards deep learning models resistant to adversarialattacks.”arXiv preprint arXiv:1706.06083(2017).

Tightness of the Bounds

Figure: Comparison of Lipschitz bounds found by various methods for five-hidden-layer networkstrained on MNIST with the Adam optimizer. Each network had a test accuracy above 97%.

• Our bounds are the tightest in the literature14

Run Time

nLipSDP-Neuron

LipSDP-Layer

500 5.22 2.851000 27.91 17.881500 82.12 58.612000 200.88 146.092500 376.07 245.943000 734.63 473.25

Table: Computation time in seconds forevaluating Lipschitz bounds ofone-hidden-layer neural networks with avarying number of hidden units. A plotshowing the Lipschitz constant for eachnetwork tested in this table has beenprovided in the Appendix.

`LipSDP-Neuron

LipSDP-Layer

5 20.33 3.4110 32.18 7.0650 87.45 25.88

100 135.85 40.39200 221.2 64.90500 707.56 216.49

Table: Computation time in seconds forcomputing Lipschitz bounds of`-hidden-layer neural networks with 100activation functions per layer. ForLipSDP-Neuron and LipSDP-Layer, wesplit each network up into 5-layersub-networks.

Reconciling Accuracy and Efficiency

Figure: LipSDP-Network Lipschitz bounds and computation time for a one-hidden-layer network with100 neurons. The weights for this network were obtained by sampling from a normal distribution.

Effect of Robust Training

Figure: (Left) Lipschitz bounds for a one-hidden-layer neural networks trained on the MNIST datasetwith the Adam optimizer and LP-Train and PGD-Train for two values of the robustness parameter ε.Each network reached an accuracy of 95% or higher. (Right) Histograms showing the local robustness(in `∞ norm) around each correctly-classified test instance from the MNIST dataset. The neuralnetworks had three hidden layers with 100, 50, 20 neurons, respectively. All classifiers had a testaccuracy of 97%.

Conclusion and Future Work

I Proposed a unified framework based on convex optimization for reachability analysis,robustness analysis, and safety verification of feedforward neural networks

I Ongoing work

– Extension to Convolutional Nets– Improve scalability– Robust training

Efficient and Accurate Estimation ofLipschitz Constants for Deep Neural Networks

Mahyar Fazlyab, Alexander RobeyHamed Hassani, Manfred Morari, George J. Pappas

NeurIPS 2019

References I

Acıkmese, B. and Corless, M. (2011).Observers for systems with nonlinearities satisfying incremental quadratic constraints.Automatica, 47(7):1339–1348.

Efficient and Accurate Estimation of Lipschitz …mahyarfa/files/Slides...E cient and Accurate...

Documents

Transcript of Efficient and Accurate Estimation of Lipschitz …mahyarfa/files/Slides...E cient and Accurate...

Accurate estimation of microbial communities using 16S tags

Fast and Accurate Stochastic Gradient Estimation · Fast and Accurate Stochastic Gradient Estimation Beidi Chen Rice University Houston, Texas beidi.chen@rice.edu Yingchen Xu Rice

Fast and Accurate Permittivity Estimation Algorithm … and Accurate Permittivity Estimation Algorithm for UWB Internal Imaging Radar Ryunosuke Souma, Shouhei Kidera, and Tetsuo Kirimoto

Lidar Essential Beam Model for Accurate Width Estimation ...

SPE 95897 Efficient and Accurate Estimation of ... · Efficient and Accurate Estimation of Permeability and Permeability Anisotropy from Straddle-Packer Formation ... with wireline

GEOMETRY OF LIPSCHITZ PERCOLATION 1. Lipschitz percolation ...

ACCURATE OPTIMAL DOPPLER CENTROID ESTIMATION FOR SAR DATA

Fast and Accurate Skew Estimation Based on Distance Transform

Accurate range estimation for an electric vehicle ...

Accurate Closed-form Estimation of Local Afﬁne ...

Polarimetric Scattering Feature Estimation For Accurate Wetland Boundary Classification

A Power Estimation Technique for Cycle-Accurate …users.auth.gr/ksiop/pdf/samos2015_power.pdf · A Power Estimation Technique for Cycle-Accurate Higher-Abstraction SystemC-based

Heterogeneous Sensor Fusion for Accurate State Estimation ... · Heterogeneous Sensor Fusion for Accurate State Estimation of Dynamic Legged Robots Simona Nobili1, Marco Camurri 2,

VTrack : Accurate, Energy-aware Road Traffic Delay Estimation

Accurate Disparity Estimation for Plenoptic Imagesvigir.missouri.edu/~gdesouza/Research/Conference_CDs/...Accurate Disparity Estimation for Plenoptic Images Neus Sabater, Mozhdeh Seiﬁ,

Accurate estimation of microscopic diffusion anisotropy ...shemeshlab.org/wp-content/uploads/2019/02/2018... · Accurate estimation of microscopic diffusion anisotropy and its time

Higher Order Accurate k-eigenvalue Sensitivity Estimation ...rampages.us/.../34693/2019/12/45-MC2019_present.pdf · Higher Order Accurate k-eigenvalue Sensitivity Estimation Using

Accurate Estimation of Center of Gravity of Spacecraft ...

Highly accurate attitude estimation via horizon detection

ACCURATE ESTIMATION OF ORIENTATION PARAMETERS OF …