Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data...

of 20/20
Processing Big Data Compressed sensing
  • date post

    14-Jul-2020
  • Category

    Documents

  • view

    0
  • download

    0

Embed Size (px)

Transcript of Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/... · Processing Big Data...

  • Processing Big DataCompressed sensing

  • Dimensionality reduction• Linear

    • Principal Component Analysis: SVD-based

    • Compressed sensing

    • Matrix sketching

    • Non-linear

    • Kernel PCA

    • Isometric mapping

  • Compressed sensing

    Consider a sparse vector

    i.e., a d-dimensional vector with only m non-zero entries

    d = 19, m = 4

    f = (0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0)AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=

  • Compressed sensing

    Let a measurement be a random vector xi 2 {�1, 0, 1}dAAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=AAAB/HicbVBNS8NAEJ34WetX/Lh5WSyCh1oSEdRb0YvHCsYWmlg22227dLMJuxuxhuJf8eJBxas/xJv/xm2bg7Y+GHi8N8PMvDDhTGnH+bbm5hcWl5YLK8XVtfWNTXtr+1bFqSTUIzGPZSPEinImqKeZ5rSRSIqjkNN62L8c+fV7KhWLxY0eJDSIcFewDiNYG6ll7z60GPKZQH525JadsusP79otu+RUnDHQLHFzUoIctZb95bdjkkZUaMKxUk3XSXSQYakZ4XRY9FNFE0z6uEubhgocURVk4+uH6MAobdSJpSmh0Vj9PZHhSKlBFJrOCOuemvZG4n9eM9WdsyBjIkk1FWSyqJNypGM0igK1maRE84EhmEhmbkWkhyUm2gRWNCG40y/PEu+4cl5xr09K1Ys8jQLswT4cggunUIUrqIEHBB7hGV7hzXqyXqx362PSOmflMzvwB9bnD4AFk4M=

    xi = (0,�1, 0, 1, 0, 0, 1, 0, 0,�1, 1, 0, 0, 0, 1, 0, 0, 1, 0)AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=AAACIHicbVDLSgMxFM34rPU16tJNsAgVxpIRwboQim5cVnBsoR2GTJq2oZkHSUYsQ3/Fjb/ixoWK7vRrzExnoa2X5N7DOfeS3OPHnEmF0JexsLi0vLJaWiuvb2xubZs7u3cySgShDol4JNo+lpSzkDqKKU7bsaA48Dlt+aOrTG/dUyFZFN6qcUzdAA9C1mcEK015Zv3BY/ACVpEFj20L6pJnZNlWljWXAZgfu6i6BR15ZgXVUB5wHtgFqIAimp752e1FJAloqAjHUnZsFCs3xUIxwumk3E0kjTEZ4QHtaBjigEo3zTecwEPN9GA/EvqGCubs74kUB1KOA193BlgN5ayWkf9pnUT1627KwjhRNCTTh/oJhyqCmV2wxwQlio81wEQw/VdIhlhgorSpZW2CPbvyPHBOauc1++a00rgs3CiBfXAAqsAGZ6ABrkETOICAR/AMXsGb8WS8GO/Gx7R1wShm9sCfML5/ALLOmOY=

    Taking a measurement is

    f = (0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0)AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=AAACHHicbVBNSwMxEJ2tX7V+rXr0EixCBSm7UlAPQtGLxwquLbRLyabZNjT7QZIVytI/4sW/4sWDihcPgv/GbLsFbZ3JhMd7MyTzvJgzqSzr2ygsLa+srhXXSxubW9s75u7evYwSQahDIh6Jlocl5SykjmKK01YsKA48Tpve8DrTmw9USBaFd2oUUzfA/ZD5jGClqa5Z89ElqlgnaHrs7J6lrXOm/Oqwjrtm2apak0CLwM5BGfJodM3PTi8iSUBDRTiWsm1bsXJTLBQjnI5LnUTSGJMh7tO2hiEOqHTTyXZjdKSZHvIjoStUaML+nkhxIOUo8HRngNVAzmsZ+Z/WTpR/7qYsjBNFQzJ9yE84UhHKrEI9JihRfKQBJoLpvyIywAITpQ0taRPs+ZUXgXNavajat7Vy/Sp3owgHcAgVsOEM6nADDXCAwCM8wyu8GU/Gi/FufExbC0Y+sw9/wvj6AQV6l4c=

    hf, xii = 2AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=AAACAnicbVBNS8NAEJ3Ur1q/ot70slgED1KSIqgHoejFYwVjC00pm+2mXbrZhN2NWELBi3/FiwcVr/4Kb/4bt20O2vpg4O17M+zMCxLOlHacb6uwsLi0vFJcLa2tb2xu2ds7dypOJaEeiXksmwFWlDNBPc00p81EUhwFnDaCwdXYb9xTqVgsbvUwoe0I9wQLGcHaSB17z+dY9DhF4TF66DDky+nzAlU7dtmpOBOgeeLmpAw56h37y+/GJI2o0IRjpVquk+h2hqVmhNNRyU8VTTAZ4B5tGSpwRFU7m9wwQodG6aIwlqaERhP190SGI6WGUWA6I6z7atYbi/95rVSHZ+2MiSTVVJDpR2HKkY7ROBDUZZISzYeGYCKZ2RWRPpaYaBNbyYTgzp48T7xq5bzi3pyUa5d5GkXYhwM4AhdOoQbXUAcPCDzCM7zCm/VkvVjv1se0tWDlM7vwB9bnDwhplgg=

    X = [x1 x2 · · · xN ]AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==

    in general, we have a measurement matrix

  • Compressed sensing

    X = [x1 x2 · · · xN ]AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==AAACCXicbVDLSsNAFJ3UV62vqEs3o0VwVZIiqIhQdONKKhhbSEKYTCbt0MmDmYm0hK7d+CtuXKi49Q/c+TdO2iy09cDlHs65l5l7/JRRIQ3jW6ssLC4tr1RXa2vrG5tb+vbOvUgyjomFE5bwro8EYTQmlqSSkW7KCYp8Rjr+4KrwOw+EC5rEd3KUEjdCvZiGFCOpJE/f78ILaA89EzrncOg1i+bgIJFiKty4nl43GsYEcJ6YJamDEm1P/3KCBGcRiSVmSAjbNFLp5ohLihkZ15xMkBThAeoRW9EYRUS4+eSUMTxUSgDDhKuKJZyovzdyFAkxinw1GSHZF7NeIf7n2ZkMT92cxmkmSYynD4UZgzKBRS4woJxgyUaKIMyp+ivEfcQRliq9mgrBnD15nljNxlnDvD2uty7LNKpgDxyAI2CCE9AC16ANLIDBI3gGr+BNe9JetHftYzpa0cqdXfAH2ucPDJiXig==

    in general, we have a measurement matrix

    y = XfAAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCqYttKFstpt27WY37G6EEPofvHhQ8eoP8ua/cdvmoK0PBh7vzTAzL0w408Z1v53Syura+kZ5s7K1vbO7V90/aGmZKkJ9IrlUnRBrypmgvmGG006iKI5DTtvh+Hbqt5+o0kyKB5MlNIjxULCIEWys1MrQNepE/WrNrbszoGXiFaQGBZr96ldvIEkaU2EIx1p3PTcxQY6VYYTTSaWXappgMsZD2rVU4JjqIJ9dO0EnVhmgSCpbwqCZ+nsix7HWWRzazhibkV70puJ/Xjc10WWQM5GkhgoyXxSlHBmJpq+jAVOUGJ5Zgoli9lZERlhhYmxAFRuCt/jyMvHP6ld17/681rgp0ijDERzDKXhwAQ24gyb4QOARnuEV3hzpvDjvzse8teQUM4fwB87nD+8gjj4=

    and a measurement

    N = Km log

    ✓d

    m

    AAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjlAAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjlAAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjlAAACCXicbVBNS8NAEN34WetX1KOX1SLUS0lEUA9C0YsgSAVjC00om82mXbr5YHcilNCzF/+KFw8qXv0H3vw3btsctPXBwOO9GWbm+angCizr25ibX1hcWi6tlFfX1jc2za3te5VkkjKHJiKRLZ8oJnjMHOAgWCuVjES+YE2/fznymw9MKp7EdzBImReRbsxDTgloqWPu3eBzfB1hVyRdV7AQqm4oCcWBliTv9uCwY1asmjUGniV2QSqoQKNjfrlBQrOIxUAFUaptWyl4OZHAqWDDspsplhLaJ13W1jQmEVNePn5liA+0EuAwkbpiwGP190ROIqUGka87IwI9Ne2NxP+8dgbhqZfzOM2AxXSyKMwEhgSPcsEBl4yCGGhCqOT6Vkx7REcBOr2yDsGefnmWOEe1s5p9e1ypXxRplNAu2kdVZKMTVEdXqIEcRNEjekav6M14Ml6Md+Nj0jpnFDM76A+Mzx8vFpjl

    Our goal is to make only N measurements

    where K in some settings is 4 or max 20

  • Compressed sensingCompression: y takes N logN

    AAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8YAAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8YAAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8YAAAB7nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lEUG9FL55KBWMLbSib7aZdutmNuxuhhP4JLx5UvPp7vPlv3LY5aOuDgcd7M8zMi1LOtPG8b6e0srq2vlHerGxt7+zuufsHD1pmitCASC5VO8KaciZoYJjhtJ0qipOI01Y0upn6rSeqNJPi3oxTGiZ4IFjMCDZWajdQl8sBavTcqlfzZkDLxC9IFQo0e+5Xty9JllBhCMdad3wvNWGOlWGE00mlm2maYjLCA9qxVOCE6jCf3TtBJ1bpo1gqW8Kgmfp7IseJ1uMksp0JNkO96E3F/7xOZuLLMGcizQwVZL4ozjgyEk2fR32mKDF8bAkmitlbERlihYmxEVVsCP7iy8skOKtd1fy782r9ukijDEdwDKfgwwXU4RaaEAABDs/wCm/Oo/PivDsf89aSU8wcwh84nz9o2Y8Y

    Application: Hubble telescope

    Has high-res camera in space, for less interference. But communication to Earth is expensive. With an initially random matrix X in space used as a mask, the images are compressed. Matrix X is known on Earth.

    Data is often noisy, and have more attributes than the real underlying process. CS helps finding the true structure of the data.

    Noisy Data

  • How to recover the compressed signal?

    One of the simplest ways: Orthogonal Matching Pursuit

    Simple and greedy method

    Greedy algorithm: makes the locally optimal choice at each stage, hoping to find a global solution

    Example: reaching the path of largest sum

    https://en.wikipedia.org/wiki/Greedy_algorithm

    A greedy strategy is not guaranteed to reach the optimal solution.

  • Orthogonal Matching pursuit

    XjAAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF272YTdjVBCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zCZXmibw3oxSDmPYljzijxkp3re5jt1J1a+4UZJF4BalCgUa38tXpJSyLURomqNZtz01NkFNlOBM4LncyjSllQ9rHtqWSxqiDfHrqmBxbpUeiRNmShkzV3xM5jbUexaHtjKkZ6HlvIv7ntTMTXQQ5l2lmULLZoigTxCRk8jfpcYXMiJEllClubyVsQBVlxqZTtiF48y8vEv+0dlnzbs+q9asijRIcwhGcgAfnUIcbaIAPDPrwDK/w5gjnxXl3PmatS04xcwB/4Hz+AJ8AjY0=

    Find the measurement column

    Note: we are not finding the row xiAAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpl262YTdiVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPJsk04z5LZKJbITVcCsV9FCh5K9WcxqHkzXB4PfGbj1wbkah7HKU8iGlfiUgwila6e+qKbqXq1twpyCLxClKFAo1u5avTS1gWc4VMUmPanptikFONgkk+Lncyw1PKhrTP25YqGnMT5NNTx+TYKj0SJdqWQjJVf0/kNDZmFIe2M6Y4MPPeRPzPa2cYXQS5UGmGXLHZoiiTBBMy+Zv0hOYM5cgSyrSwtxI2oJoytOmUbQje/MuLxD+tXda827Nq/apIowSHcAQn4ME51OEGGuADgz48wyu8OdJ5cd6dj1nrklPMHMAfOJ8/zh2NrA==

    j is the index of f that better explains y

    Step 1

  • Orthogonal Matching pursuit

    Reconstruction residual between measurement and what is our estimate for the entry j of f

    Step 2

  • Orthogonal Matching pursuit

    this is what remains to be explained by other elements of f

    Step 3

    compute the residual

    and return to step 1, until r is small (there is not that much to explain) oris small (further explanation is not useful)

    �AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7pAAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7pAAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7pAAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvQi8cIbhJIljA7mU3GzGOZmRXCkn/w4kHFqx/kzb9xkuxBEwsaiqpuurvilDNjff/bW1ldW9/YLG2Vt3d29/YrB4dNozJNaEgUV7odY0M5kzS0zHLaTjXFIua0FY9up37riWrDlHyw45RGAg8kSxjB1knN7gALgXuVql/zZ0DLJChIFQo0epWvbl+RTFBpCcfGdAI/tVGOtWWE00m5mxmaYjLCA9pxVGJBTZTPrp2gU6f0UaK0K2nRTP09kWNhzFjErlNgOzSL3lT8z+tkNrmKcibTzFJJ5ouSjCOr0PR11GeaEsvHjmCimbsVkSHWmFgXUNmFECy+vEzC89p1Lbi/qNZvijRKcAwncAYBXEId7qABIRB4hGd4hTdPeS/eu/cxb13xipkj+APv8wfzVo7p

    y = rAAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43oAAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43oAAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43oAAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8sfkmuhetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdWA43o

    do

  • Orthogonal Matching pursuit

    r = yAAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543oAAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543oAAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543oAAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2lA22027dLMJuxOhhP4GLx5UvPqHvPlv3LY5aOuDgcd7M8zMC1MpDLrut1NaWV1b3yhvVra2d3b3qvsHjybJNOM+S2Si2yE1XArFfRQoeTvVnMah5K1wdDv1W09cG5GoBxynPIjpQIlIMIpW8jW5JuNetebW3RnIMvEKUoMCzV71q9tPWBZzhUxSYzqem2KQU42CST6pdDPDU8pGdMA7lioacxPks2Mn5MQqfRIl2pZCMlN/T+Q0NmYch7Yzpjg0i95U/M/rZBhdBrlQaYZcsfmiKJMEEzL9nPSF5gzl2BLKtLC3EjakmjK0+VRsCN7iy8vEP6tf1b3781rjpkijDEdwDKfgwQU04A6a4AMDAc/wCm+Ocl6cd+dj3lpyiplD+APn8wdV543o

    Xj = argmaxXj0

    |hr,Xj0i|AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=AAACM3icbVDLSgMxFM34tr6qLt0Ei+BCyowI6kIQ3QhuFKwWOmXIpLc1mmSG5I5YxvkoN36IGxFcqLj1H0wfC18HAodz7uXmnDiVwqLvP3sjo2PjE5NT06WZ2bn5hfLi0rlNMsOhxhOZmHrMLEihoYYCJdRTA0zFEi7i68Oef3EDxopEn2E3haZiHS3agjN0UlQ+rkdXdI+GiuFlkuYhwi3mzHQUuy2KKK9H+VWYGqGgKOgdDSXTHQnUbHwzQjMQ76Jyxa/6fdC/JBiSChniJCo/hq2EZwo0csmsbQR+ik13HgWXUJTCzELK+DXrQMNRzRTYZt4PXdA1p7RoOzHuaaR99ftGzpS1XRW7yV44+9vrif95jQzbO81c6DRD0HxwqJ1JigntNUhbwgBH2XWEcSPcXym/ZIZxdD2XXAnB78h/SW2zulsNTrcq+wfDNqbIClkl6yQg22SfHJETUiOc3JMn8krevAfvxXv3PgajI95wZ5n8gPf5BVz3rLY=

    �j = argmin�

    kr �Xj�kAAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==AAACKXicbVDLSsQwFE3H9/iqunQTHAQ3Dq0I6kIYdeNSwdGBaSlpJh3jJGlJbsWh9nvc+CtuXPja+iNmHgtfBwKHc+7l5pw4E9yA5707lYnJqemZ2bnq/MLi0rK7snpp0lxT1qSpSHUrJoYJrlgTOAjWyjQjMhbsKu6dDPyrW6YNT9UF9DMWStJVPOGUgJUi9yjoEilJdIMPcSAJXKdZEQC7g4LoruSqLKNiNFLi4B5jjbdxKypuypFotciteXVvCPyX+GNSQ2OcRe5z0ElpLpkCKogxbd/LILT3gFPBymqQG5YR2iNd1rZUEclMWAyjlnjTKh2cpNo+BXioft8oiDSmL2M7OUhjfnsD8T+vnUOyHxZcZTkwRUeHklxgSPGgN9zhmlEQfUsI1dz+FdNrogkF227VluD/jvyXNHfqB3X/fLfWOB63MYvW0QbaQj7aQw10is5QE1H0gJ7QC3p1Hp1n5835GI1WnPHOGvoB5/MLSmKnXw==

    r = r �Xj�jAAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=AAAB/XicbVBNS8NAEJ34WetXVDx5WSyCF0signoQil48VjC20Iaw2W7abXeTsLsRSij4V7x4UPHq//Dmv3Hb5qCtDwYe780wMy9MOVPacb6thcWl5ZXV0lp5fWNza9ve2X1QSSYJ9UjCE9kMsaKcxdTTTHPaTCXFIuS0EQ5uxn7jkUrFkvheD1PqC9yNWcQI1kYK7H2JrpBEJ6gZ5P1Ru4uFwEE/sCtO1ZkAzRO3IBUoUA/sr3YnIZmgsSYcK9VynVT7OZaaEU5H5XamaIrJAHdpy9AYC6r8fHL+CB0ZpYOiRJqKNZqovydyLJQaitB0Cqx7atYbi/95rUxHF37O4jTTNCbTRVHGkU7QOAvUYZISzYeGYCKZuRWRHpaYaJNY2YTgzr48T7zT6mXVvTur1K6LNEpwAIdwDC6cQw1uoQ4eEMjhGV7hzXqyXqx362PaumAVM3vwB9bnD+WLlGA=

    gj = �jAAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGvAAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGvAAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGvAAAB9HicbVBNS8NAEN3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW2hgm20267W4SdjdKCf0fXjyoePXHePPfuG1z0NYHA4/3ZpiZF6ScKW3b31ZpaXllda28XtnY3Nreqe7u3askk4S6JOGJbAegKGcxdTXTnLZTSUEEnLaC4fXEbz1SqVgS3+lRSj0BUcxCRkAb6SHyB/gSdyMQAvyBX63ZdXsKvEicgtRQgaZf/er2EpIJGmvCQamOY6fay0FqRjgdV7qZoimQIUS0Y2gMgiovn149xkdG6eEwkaZijafq74kchFIjEZhOAbqv5r2J+J/XyXR47uUsTjNNYzJbFGYc6wRPIsA9JinRfGQIEMnMrZj0QQLRJqiKCcGZf3mRuCf1i7pze1prXBVplNEBOkTHyEFnqIFuUBO5iCCJntErerOerBfr3fqYtZasYmYf/YH1+QPrYpGv

    Set

    Return

    S1:

    S2:

    S3:

  • Examplef = [0, 0, 1, 0, 0, 1, 0, 0, 1, 0]

    AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==AAACCnicbVDLSgMxFL1TX7W+qi7dhBbBhZQZEdSFUHTjsoJjC9OhZNJMG5rJDElGKEP3bvwVNy5U3PoF7vwb0+kstPWQXA7n3EtyT5BwprRtf1ulpeWV1bXyemVjc2t7p7q7d6/iVBLqkpjHshNgRTkT1NVMc9pJJMVRwGk7GF1P/fYDlYrF4k6PE+pHeCBYyAjWRupVayG6RJ59jMxx8jpP/F61bjfsHGiROAWpQ4FWr/rV7cckjajQhGOlPMdOtJ9hqRnhdFLppoommIzwgHqGChxR5Wf5LhN0aJQ+CmNprtAoV39PZDhSahwFpjPCeqjmvan4n+elOjz3MyaSVFNBZg+FKUc6RtNgUJ9JSjQfG4KJZOaviAyxxESb+ComBGd+5UXinjQuGs7tab15VaRRhgOowRE4cAZNuIEWuEDgEZ7hFd6sJ+vFerc+Zq0lq5jZhz+wPn8AmbCT/Q==

    y = XfTAAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqB6EohePFRobaGPZbDft0s0m7m6EEPonvHhQ8erv8ea/cdvmoK0PBh7vzTAzL0g4U9q2v63Syura+kZ5s7K1vbO7V90/uFdxKgl1Scxj6QVYUc4EdTXTnHqJpDgKOO0E45up33miUrFYtHWWUD/CQ8FCRrA2kpehK+SFD+1+tWbX7RnQMnEKUoMCrX71qzeISRpRoQnHSnUdO9F+jqVmhNNJpZcqmmAyxkPaNVTgiCo/n907QSdGGaAwlqaERjP190SOI6WyKDCdEdYjtehNxf+8bqrDCz9nIkk1FWS+KEw50jGaPo8GTFKieWYIJpKZWxEZYYmJNhFVTAjO4svLxD2rX9adu/Na87pIowxHcAyn4EADmnALLXCBAIdneIU369F6sd6tj3lrySpmDuEPrM8fSfaPBA==

    y = [0, 0, 0, 1, 1,�2]TAAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKmAAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKmAAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKmAAACAXicbVDLSsNAFJ3UV62vqCtxM1gEF1qSIqgLoejGZYXGFtJYJtNJO3QyCTMTIYTixl9x40LFrX/hzr9xmmahrYc7cDjnXu7c48eMSmVZ30ZpYXFpeaW8Wllb39jcMrd37mSUCEwcHLFIdHwkCaOcOIoqRjqxICj0GWn7o+uJ334gQtKIt1QaEy9EA04DipHSUs/cS+EldK1jOC07r5O6d9/qmVWrZuWA88QuSBUUaPbMr24/wklIuMIMSenaVqy8DAlFMSPjSjeRJEZ4hAbE1ZSjkEgvy08Yw0Ot9GEQCf24grn6eyJDoZRp6OvOEKmhnPUm4n+em6jg3MsojxNFOJ4uChIGVQQnecA+FQQrlmqCsKD6rxAPkUBY6dQqOgR79uR54tRrFzX79rTauCrSKIN9cACOgA3OQAPcgCZwAAaP4Bm8gjfjyXgx3o2PaWvJKGZ2wR8Ynz+gJJKm

  • Compressed sensing: full reconstructionWe can fully reconstruct any sparse signal if it was compressed by a matrix satisfying the Restricted Isoperimetric Property (RIP)

    RIP guarantees a low distortion of the norm of any sparse representable vector 


    Full reconstruction is done by a solving linear problem: polynomial time

    A random matrix with the characteristics of X is likely to satisfy the RIP condition

  • CS: problem formulation

    f̂ 2 argming:Xg=y

    kgk0AAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKOAAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKOAAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKOAAACG3icbVDLSsNAFJ34rPVVdelmsAiuSqKCDxCKblxWsLbQlDCZTtKhk0mYuRFLmg9x46+4caHiSnDh3zh9LLT1wIXDOfdy7z1+IrgG2/625uYXFpeWCyvF1bX1jc3S1vadjlNFWZ3GIlZNn2gmuGR14CBYM1GMRL5gDb93NfQb90xpHstb6CesHZFQ8oBTAkbySkdul0AW5NjlErsRgW6cZC6wB8iICiMu89zLwvNmeNE3PYPQHXi2VyrbFXsEPEucCSmjCWpe6dPtxDSNmAQqiNYtx06gbRYAp4LlRTfVLCG0R0LWMlSSiOl2Nnoux/tG6eAgVqYk4JH6eyIjkdb9yDedw/P1tDcU//NaKQSn7YzLJAUm6XhRkAoMMR4mhTtcMQqibwihiptbMe0SRSiYPIsmBGf65VlSP6ycVZyb43L1cpJGAe2iPXSAHHSCquga1VAdUfSIntErerOerBfr3foYt85Zk5kd9AfW1w+rgKKO

    If X is RIP

    kfk0 mAAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=AAAB9HicbVBNSwMxEM3Wr1q/qh69BIvgqeyKoN6KXjxWcG2hu5ZsOtuGJtk1ySpl2//hxYOKV3+MN/+N6cdBWx8MPN6bYWZelHKmjet+O4Wl5ZXVteJ6aWNza3unvLt3p5NMUfBpwhPVjIgGziT4hhkOzVQBERGHRtS/GvuNR1CaJfLWDFIIBelKFjNKjJXug2EcDNsuDjg8YNEuV9yqOwFeJN6MVNAM9Xb5K+gkNBMgDeVE65bnpibMiTKMchiVgkxDSmifdKFlqSQCdJhPrh7hI6t0cJwoW9Lgifp7IidC64GIbKcgpqfnvbH4n9fKTHwe5kymmQFJp4vijGOT4HEEuMMUUMMHlhCqmL0V0x5RhBobVMmG4M2/vEj8k+pF1bs5rdQuZ2kU0QE6RMfIQ2eohq5RHfmIIoWe0St6c56cF+fd+Zi2FpzZzD76A+fzBxg4kco=

    )AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==AAAB8XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWMVYwttKJvtpl26yYbdiVJCf4YXDype/Tfe/Ddu2xy09cHA470ZZuaFqRQGXffbWVpeWV1bL22UN7e2d3Yre/sPRmWacZ8pqXQrpIZLkXAfBUreSjWncSh5MxxeT/zmI9dGqOQeRykPYtpPRCQYRSu1O3eiP0CqtXrqVqpuzZ2CLBKvIFUo0OhWvjo9xbKYJ8gkNabtuSkGOdUomOTjciczPKVsSPu8bWlCY26CfHrymBxbpUcipW0lSKbq74mcxsaM4tB2xhQHZt6biP957QyjiyAXSZohT9hsUZRJgopM/ic9oTlDObKEMi3srYQNqKYMbUplG4I3//Ii8U9rlzXv9qxavyrSKMEhHMEJeHAOdbiBBvjAQMEzvMKbg86L8+58zFqXnGLmAP7A+fwB+1uRQQ==

    f̂ = fAAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUA9C0YvHCsYW21A22027dLMJuxOhhP4LLx5UvPpzvPlv3LY5aPXBwOO9GWbmhakUBl33yyktLa+srpXXKxubW9s71d29e5NkmnGfJTLR7ZAaLoXiPgqUvJ1qTuNQ8lY4up76rUeujUjUHY5THsR0oEQkGEUrPXSHFElELknUq9bcujsD+Uu8gtSgQLNX/ez2E5bFXCGT1JiO56YY5FSjYJJPKt3M8JSyER3wjqWKxtwE+eziCTmySp9EibalkMzUnxM5jY0Zx6HtjCkOzaI3Ff/zOhlG50EuVJohV2y+KMokwYRM3yd9oTlDObaEMi3srYQNqaYMbUgVG4K3+PJf4p/UL+re7WmtcVWkUYYDOIRj8OAMGnADTfCBgYIneIFXxzjPzpvzPm8tOcXMPvyC8/ENhtiPtA==

  • CS: convex relaxation is tight

    We can prove, under mild conditions, that

    f = argming:Xg=y

    kgk0 = argming:Xg=y

    kgk1AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==AAACPXicjVA9S8RAEN347fl1ammzeAhWRyKCH3Ag2lgqGD24HGGzt8ktbjZhdyKGXH6Zjb/BztLGQsXW1r0zhZ4WPhh4vDfDzLwgFVyDbT9aE5NT0zOzc/O1hcWl5ZX66tqlTjJFmUsTkah2QDQTXDIXOAjWThUjcSDYVXB9MvSvbpjSPJEXkKesG5NI8pBTAkby626IW9iLCfSTtPCA3UJBVBRzWZZ+ER22o1ZeYm8QeQPf/nen49cbdtMeAf8mTkUaqMKZX3/wegnNYiaBCqJ1x7FT6JoFwKlgZc3LNEsJvSYR6xgqScx0txi9X+Ito/RwmChTEvBI/T5RkFjrPA5M5/B8Pe4Nxb+8TgbhfrfgMs2ASfq1KMwEhgQPs8Q9rhgFkRtCqOLmVkz7RBEKJvGaCcEZf/k3cXeaB03nfLdxdFylMYc20CbaRg7aQ0foFJ0hF1F0h57QC3q17q1n6816/2qdsKqZdfQD1scnGF2wgA==

    More information in Shai Shalev-Shwartz, Compressed Sensing: Basic results and self contained proofs

  • Matrix sketching

    Going back to a dense data matrix A

    We are going to see a way to “improve” PCA and the SVD

  • Matrix sketching: the SVD

    Jeff M. Phillips, University of Utah

  • Matrix sketching: the SVD

    Jeff M. Phillips,

    PCA:

  • Approximation of A using the SVD

    Columns of V are linear combinations of features

    Interpretability of V

    What is a linear combination of purchased books?

    What is a linear combination between number of calls and being male or female?

    Computational efficiency

    Computing the SVD demands loading all A into memory and requires time

    What about really big data?

    What about streamed data?

  • Row sampling

    Interpretability: choose V so that columns of V are columns of A