Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go...

75
Processing Big Data Kernel PCA

Transcript of Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go...

Page 1: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Processing Big DataKernel PCA

Page 2: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Dimensionality reduction• Linear

• Principal Component Analysis: SVD-based

• Compressed sensing

• Matrix sketching

• Non-linear

• Kernel PCA

• Isometric mapping

Page 3: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

High dimensionality can be good

Transforming data to lie in higher dimension spaces can be a good thing!

Page 4: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

High dimensionality can be good

Transforming data to lie in higher dimension spaces can be a good thing!

Statistical learning theory aims to answer:

Page 5: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

High dimensionality can be good

Transforming data to lie in higher dimension spaces can be a good thing!

Statistical learning theory aims to answer:

Under what conditions are high-dimension representations good?

Page 6: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

High dimensionality can be good

Transforming data to lie in higher dimension spaces can be a good thing!

Statistical learning theory aims to answer:

Under what conditions are high-dimension representations good?

Sometimes

Page 7: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

High dimensionality can be good

Transforming data to lie in higher dimension spaces can be a good thing!

Statistical learning theory aims to answer:

Under what conditions are high-dimension representations good?

Sometimes

Higher dimension Increased classification power

Page 8: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Example

Page 9: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

ExampleLet’s go 3D

� : R2 ! R3<latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit><latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit><latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit><latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit>

(x1, x2) ! (x1, x2, x21 + x2

2)<latexit sha1_base64="+mY8jK9SQHkQeVKU86Mp72Rw5IA=">AAACD3icbVDLSgMxFM3UV62vUZdugkVpsZSZQdBl0Y3LCvYB7XTIpGkbmskMSUZahv6BG3/FjQtF3Lp159+YtiNo64WQk3Pu5eYcP2JUKsv6MjIrq2vrG9nN3Nb2zu6euX9Ql2EsMKnhkIWi6SNJGOWkpqhipBkJggKfkYY/vJ7qjXsiJA35nRpHxA1Qn9MexUhpyjNPCyPPLo08pwjbKoQ/rxLUoOPAM307HafomXmrbM0KLgM7BXmQVtUzP9vdEMcB4QozJGXLtiLlJkgoihmZ5NqxJBHCQ9QnLQ05Coh0k5mfCTzRTBf2QqEPV3DG/p5IUCDlOPB1Z4DUQC5qU/I/rRWr3qWbUB7FinA8X9SLGdTOp+HALhUEKzbWAGFB9V8hHiCBsNIR5nQI9qLlZVB3yrZVtm/P85WrNI4sOALHoABscAEq4AZUQQ1g8ACewAt4NR6NZ+PNeJ+3Zox05hD8KePjG73VmKo=</latexit><latexit sha1_base64="+mY8jK9SQHkQeVKU86Mp72Rw5IA=">AAACD3icbVDLSgMxFM3UV62vUZdugkVpsZSZQdBl0Y3LCvYB7XTIpGkbmskMSUZahv6BG3/FjQtF3Lp159+YtiNo64WQk3Pu5eYcP2JUKsv6MjIrq2vrG9nN3Nb2zu6euX9Ql2EsMKnhkIWi6SNJGOWkpqhipBkJggKfkYY/vJ7qjXsiJA35nRpHxA1Qn9MexUhpyjNPCyPPLo08pwjbKoQ/rxLUoOPAM307HafomXmrbM0KLgM7BXmQVtUzP9vdEMcB4QozJGXLtiLlJkgoihmZ5NqxJBHCQ9QnLQ05Coh0k5mfCTzRTBf2QqEPV3DG/p5IUCDlOPB1Z4DUQC5qU/I/rRWr3qWbUB7FinA8X9SLGdTOp+HALhUEKzbWAGFB9V8hHiCBsNIR5nQI9qLlZVB3yrZVtm/P85WrNI4sOALHoABscAEq4AZUQQ1g8ACewAt4NR6NZ+PNeJ+3Zox05hD8KePjG73VmKo=</latexit><latexit sha1_base64="+mY8jK9SQHkQeVKU86Mp72Rw5IA=">AAACD3icbVDLSgMxFM3UV62vUZdugkVpsZSZQdBl0Y3LCvYB7XTIpGkbmskMSUZahv6BG3/FjQtF3Lp159+YtiNo64WQk3Pu5eYcP2JUKsv6MjIrq2vrG9nN3Nb2zu6euX9Ql2EsMKnhkIWi6SNJGOWkpqhipBkJggKfkYY/vJ7qjXsiJA35nRpHxA1Qn9MexUhpyjNPCyPPLo08pwjbKoQ/rxLUoOPAM307HafomXmrbM0KLgM7BXmQVtUzP9vdEMcB4QozJGXLtiLlJkgoihmZ5NqxJBHCQ9QnLQ05Coh0k5mfCTzRTBf2QqEPV3DG/p5IUCDlOPB1Z4DUQC5qU/I/rRWr3qWbUB7FinA8X9SLGdTOp+HALhUEKzbWAGFB9V8hHiCBsNIR5nQI9qLlZVB3yrZVtm/P85WrNI4sOALHoABscAEq4AZUQQ1g8ACewAt4NR6NZ+PNeJ+3Zox05hD8KePjG73VmKo=</latexit>

x !

xkxk2

<latexit sha1_base64="S78ZIv1hWRk1fX6qxO/0wH15ybQ=">AAACGnicbVBNS8NAEN3Ur1q/oh69LBbBU0mKoMeiF48V7Ac0sWy203bpZhN2N9KS9nd48a948aCIN/Hiv3HbBtTWBwOP92aYmRfEnCntOF9WbmV1bX0jv1nY2t7Z3bP3D+oqSiSFGo14JJsBUcCZgJpmmkMzlkDCgEMjGFxN/cY9SMUicatHMfgh6QnWZZRoI7Vtd4g9HWEvgB4TaRASLdlwgo3qYW889MZ3ZeyB6PxYbbvolJwZ8DJxM1JEGapt+8PrRDQJQWjKiVIt14m1nxKpGeUwKXiJgpjQAelBy1BBQlB+Onttgk+M0sHdSJoSGs/U3xMpCZUahYHpNAf21aI3Ff/zWonuXvgpE3GiQdD5om7CsQljmhPuMAlU85EhhEpmbsW0TySh2qRZMCG4iy8vk3q55Dol9+asWLnM4sijI3SMTpGLzlEFXaMqqiGKHtATekGv1qP1bL1Z7/PWnJXNHKI/sD6/ARjToOY=</latexit><latexit sha1_base64="S78ZIv1hWRk1fX6qxO/0wH15ybQ=">AAACGnicbVBNS8NAEN3Ur1q/oh69LBbBU0mKoMeiF48V7Ac0sWy203bpZhN2N9KS9nd48a948aCIN/Hiv3HbBtTWBwOP92aYmRfEnCntOF9WbmV1bX0jv1nY2t7Z3bP3D+oqSiSFGo14JJsBUcCZgJpmmkMzlkDCgEMjGFxN/cY9SMUicatHMfgh6QnWZZRoI7Vtd4g9HWEvgB4TaRASLdlwgo3qYW889MZ3ZeyB6PxYbbvolJwZ8DJxM1JEGapt+8PrRDQJQWjKiVIt14m1nxKpGeUwKXiJgpjQAelBy1BBQlB+Onttgk+M0sHdSJoSGs/U3xMpCZUahYHpNAf21aI3Ff/zWonuXvgpE3GiQdD5om7CsQljmhPuMAlU85EhhEpmbsW0TySh2qRZMCG4iy8vk3q55Dol9+asWLnM4sijI3SMTpGLzlEFXaMqqiGKHtATekGv1qP1bL1Z7/PWnJXNHKI/sD6/ARjToOY=</latexit><latexit sha1_base64="S78ZIv1hWRk1fX6qxO/0wH15ybQ=">AAACGnicbVBNS8NAEN3Ur1q/oh69LBbBU0mKoMeiF48V7Ac0sWy203bpZhN2N9KS9nd48a948aCIN/Hiv3HbBtTWBwOP92aYmRfEnCntOF9WbmV1bX0jv1nY2t7Z3bP3D+oqSiSFGo14JJsBUcCZgJpmmkMzlkDCgEMjGFxN/cY9SMUicatHMfgh6QnWZZRoI7Vtd4g9HWEvgB4TaRASLdlwgo3qYW889MZ3ZeyB6PxYbbvolJwZ8DJxM1JEGapt+8PrRDQJQWjKiVIt14m1nxKpGeUwKXiJgpjQAelBy1BBQlB+Onttgk+M0sHdSJoSGs/U3xMpCZUahYHpNAf21aI3Ff/zWonuXvgpE3GiQdD5om7CsQljmhPuMAlU85EhhEpmbsW0TySh2qRZMCG4iy8vk3q55Dol9+asWLnM4sijI3SMTpGLzlEFXaMqqiGKHtATekGv1qP1bL1Z7/PWnJXNHKI/sD6/ARjToOY=</latexit>

or, more concisely,

Page 10: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Example

� : R2 ! R3<latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit><latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit><latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit><latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit>

x !

xkxk2

<latexit sha1_base64="S78ZIv1hWRk1fX6qxO/0wH15ybQ=">AAACGnicbVBNS8NAEN3Ur1q/oh69LBbBU0mKoMeiF48V7Ac0sWy203bpZhN2N9KS9nd48a948aCIN/Hiv3HbBtTWBwOP92aYmRfEnCntOF9WbmV1bX0jv1nY2t7Z3bP3D+oqSiSFGo14JJsBUcCZgJpmmkMzlkDCgEMjGFxN/cY9SMUicatHMfgh6QnWZZRoI7Vtd4g9HWEvgB4TaRASLdlwgo3qYW889MZ3ZeyB6PxYbbvolJwZ8DJxM1JEGapt+8PrRDQJQWjKiVIt14m1nxKpGeUwKXiJgpjQAelBy1BBQlB+Onttgk+M0sHdSJoSGs/U3xMpCZUahYHpNAf21aI3Ff/zWonuXvgpE3GiQdD5om7CsQljmhPuMAlU85EhhEpmbsW0TySh2qRZMCG4iy8vk3q55Dol9+asWLnM4sijI3SMTpGLzlEFXaMqqiGKHtATekGv1qP1bL1Z7/PWnJXNHKI/sD6/ARjToOY=</latexit><latexit sha1_base64="S78ZIv1hWRk1fX6qxO/0wH15ybQ=">AAACGnicbVBNS8NAEN3Ur1q/oh69LBbBU0mKoMeiF48V7Ac0sWy203bpZhN2N9KS9nd48a948aCIN/Hiv3HbBtTWBwOP92aYmRfEnCntOF9WbmV1bX0jv1nY2t7Z3bP3D+oqSiSFGo14JJsBUcCZgJpmmkMzlkDCgEMjGFxN/cY9SMUicatHMfgh6QnWZZRoI7Vtd4g9HWEvgB4TaRASLdlwgo3qYW889MZ3ZeyB6PxYbbvolJwZ8DJxM1JEGapt+8PrRDQJQWjKiVIt14m1nxKpGeUwKXiJgpjQAelBy1BBQlB+Onttgk+M0sHdSJoSGs/U3xMpCZUahYHpNAf21aI3Ff/zWonuXvgpE3GiQdD5om7CsQljmhPuMAlU85EhhEpmbsW0TySh2qRZMCG4iy8vk3q55Dol9+asWLnM4sijI3SMTpGLzlEFXaMqqiGKHtATekGv1qP1bL1Z7/PWnJXNHKI/sD6/ARjToOY=</latexit><latexit sha1_base64="S78ZIv1hWRk1fX6qxO/0wH15ybQ=">AAACGnicbVBNS8NAEN3Ur1q/oh69LBbBU0mKoMeiF48V7Ac0sWy203bpZhN2N9KS9nd48a948aCIN/Hiv3HbBtTWBwOP92aYmRfEnCntOF9WbmV1bX0jv1nY2t7Z3bP3D+oqSiSFGo14JJsBUcCZgJpmmkMzlkDCgEMjGFxN/cY9SMUicatHMfgh6QnWZZRoI7Vtd4g9HWEvgB4TaRASLdlwgo3qYW889MZ3ZeyB6PxYbbvolJwZ8DJxM1JEGapt+8PrRDQJQWjKiVIt14m1nxKpGeUwKXiJgpjQAelBy1BBQlB+Onttgk+M0sHdSJoSGs/U3xMpCZUahYHpNAf21aI3Ff/zWonuXvgpE3GiQdD5om7CsQljmhPuMAlU85EhhEpmbsW0TySh2qRZMCG4iy8vk3q55Dol9+asWLnM4sijI3SMTpGLzlEFXaMqqiGKHtATekGv1qP1bL1Z7/PWnJXNHKI/sD6/ARjToOY=</latexit>

Page 11: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Problems with high-dimensionalityProduct feature extractor with monomials of degree d=2

� : R2 ! R3<latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit><latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit><latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit><latexit sha1_base64="6UGWcFCkSQYtfoRwCAg0UxNhP1Y=">AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh+w8VdsLBSxtbfzb5xNtoiJBwYO59zL3HO8mDOpLOvHKCwtr6yuFddLG5tb2zvm7l5TRokgtEEiHom2hyXlLKQNxRSn7VhQHHictrzhdea3HqmQLArv1SimboD7IfMZwUpLXfPIqQ8YukROgNXA89K78UMVOSqaFU67ZtmqWBOgRWLnpAw56l3z2+lFJAloqAjHUnZsK1ZuioVihNNxyUkkjTEZ4j7taBrigEo3naQZo2Ot9JAfCf1ChSbq7EaKAylHgacnsxvlvJeJ/3mdRPkXbsrCOFE0JNOP/IQjnTarBvWYoETxkSaYCKZvRWSABSZKF1jSJdjzkRdJs1qxrYp9e1auXeV1FOEADuEEbDiHGtxAHRpA4Ale4A3ejWfj1fgwPqejBSPf2Yc/ML5+AeY3mtI=</latexit>

(x1, x2) ! (x21, x

22, x1x2, x2x1)

<latexit sha1_base64="6raepg/CjlRxtKiXO1OrmPHeF10=">AAACE3icbZC7TsMwFIadcivlFmBksaiQWoSqpEKCsYKFsUj0IrUhclynteo4ke2gVlHfgYVXYWEAIVYWNt4GJ80ALUey9Z3/nCP7/F7EqFSW9W0UVlbX1jeKm6Wt7Z3dPXP/oC3DWGDSwiELRddDkjDKSUtRxUg3EgQFHiMdb3yd1jsPREga8js1jYgToCGnPsVIack1TysT1z6buPUq7KsQptl9Pc2z29aQJlBj1TXLVs3KAi6DnUMZ5NF0za/+IMRxQLjCDEnZs61IOQkSimJGZqV+LEmE8BgNSU8jRwGRTpLtNIMnWhlAPxT6cAUz9fdEggIpp4GnOwOkRnKxlor/1Xqx8i+dhPIoVoTj+UN+zKDePjUIDqggWLGpBoQF1X+FeIQEwkrbWNIm2IsrL0O7XrOtmn17Xm5c5XYUwRE4BhVggwvQADegCVoAg0fwDF7Bm/FkvBjvxse8tWDkM4fgTxifP0n7mqQ=</latexit><latexit sha1_base64="6raepg/CjlRxtKiXO1OrmPHeF10=">AAACE3icbZC7TsMwFIadcivlFmBksaiQWoSqpEKCsYKFsUj0IrUhclynteo4ke2gVlHfgYVXYWEAIVYWNt4GJ80ALUey9Z3/nCP7/F7EqFSW9W0UVlbX1jeKm6Wt7Z3dPXP/oC3DWGDSwiELRddDkjDKSUtRxUg3EgQFHiMdb3yd1jsPREga8js1jYgToCGnPsVIack1TysT1z6buPUq7KsQptl9Pc2z29aQJlBj1TXLVs3KAi6DnUMZ5NF0za/+IMRxQLjCDEnZs61IOQkSimJGZqV+LEmE8BgNSU8jRwGRTpLtNIMnWhlAPxT6cAUz9fdEggIpp4GnOwOkRnKxlor/1Xqx8i+dhPIoVoTj+UN+zKDePjUIDqggWLGpBoQF1X+FeIQEwkrbWNIm2IsrL0O7XrOtmn17Xm5c5XYUwRE4BhVggwvQADegCVoAg0fwDF7Bm/FkvBjvxse8tWDkM4fgTxifP0n7mqQ=</latexit><latexit sha1_base64="6raepg/CjlRxtKiXO1OrmPHeF10=">AAACE3icbZC7TsMwFIadcivlFmBksaiQWoSqpEKCsYKFsUj0IrUhclynteo4ke2gVlHfgYVXYWEAIVYWNt4GJ80ALUey9Z3/nCP7/F7EqFSW9W0UVlbX1jeKm6Wt7Z3dPXP/oC3DWGDSwiELRddDkjDKSUtRxUg3EgQFHiMdb3yd1jsPREga8js1jYgToCGnPsVIack1TysT1z6buPUq7KsQptl9Pc2z29aQJlBj1TXLVs3KAi6DnUMZ5NF0za/+IMRxQLjCDEnZs61IOQkSimJGZqV+LEmE8BgNSU8jRwGRTpLtNIMnWhlAPxT6cAUz9fdEggIpp4GnOwOkRnKxlor/1Xqx8i+dhPIoVoTj+UN+zKDePjUIDqggWLGpBoQF1X+FeIQEwkrbWNIm2IsrL0O7XrOtmn17Xm5c5XYUwRE4BhVggwvQADegCVoAg0fwDF7Bm/FkvBjvxse8tWDkM4fgTxifP0n7mqQ=</latexit><latexit sha1_base64="6raepg/CjlRxtKiXO1OrmPHeF10=">AAACE3icbZC7TsMwFIadcivlFmBksaiQWoSqpEKCsYKFsUj0IrUhclynteo4ke2gVlHfgYVXYWEAIVYWNt4GJ80ALUey9Z3/nCP7/F7EqFSW9W0UVlbX1jeKm6Wt7Z3dPXP/oC3DWGDSwiELRddDkjDKSUtRxUg3EgQFHiMdb3yd1jsPREga8js1jYgToCGnPsVIack1TysT1z6buPUq7KsQptl9Pc2z29aQJlBj1TXLVs3KAi6DnUMZ5NF0za/+IMRxQLjCDEnZs61IOQkSimJGZqV+LEmE8BgNSU8jRwGRTpLtNIMnWhlAPxT6cAUz9fdEggIpp4GnOwOkRnKxlor/1Xqx8i+dhPIoVoTj+UN+zKDePjUIDqggWLGpBoQF1X+FeIQEwkrbWNIm2IsrL0O7XrOtmn17Xm5c5XYUwRE4BhVggwvQADegCVoAg0fwDF7Bm/FkvBjvxse8tWDkM4fgTxifP0n7mqQ=</latexit>

for m-dimensional data the number of monomials is

✓m+ d+ 1

d

◆=

(d+m� 1)!

d!(m� 1)!<latexit sha1_base64="2wc3TY/SE1D2Fcqdd3fVdLwrY4o=">AAACFXicbZDLSgMxFIYz9VbrbdSlm9QiVKplRgTdCEU3LivYC3RKyWQybWiSGZKMUIZ5CTe+ihsXirgV3Pk2ppeFtv4Q+PjPOZyc348ZVdpxvq3c0vLK6lp+vbCxubW9Y+/uNVWUSEwaOGKRbPtIEUYFaWiqGWnHkiDuM9LyhzfjeuuBSEUjca9HMely1Bc0pBhpY/XsE8+nIuIprwQVN0uDDF5BL5QIp+Wgwk/d46Ixi+Up9eySU3UmgovgzqAEZqr37C8viHDCidCYIaU6rhPrboqkppiRrOAlisQID1GfdAwKxInqppOrMnhknACGkTRPaDhxf0+kiCs14r7p5EgP1HxtbP5X6yQ6vOymVMSJJgJPF4UJgzqC44hgQCXBmo0MICyp+SvEA2Qy0SbIggnBnT95EZpnVdepunfnpdr1LI48OACHoAxccAFq4BbUQQNg8AiewSt4s56sF+vd+pi25qzZzD74I+vzB33snHM=</latexit><latexit sha1_base64="2wc3TY/SE1D2Fcqdd3fVdLwrY4o=">AAACFXicbZDLSgMxFIYz9VbrbdSlm9QiVKplRgTdCEU3LivYC3RKyWQybWiSGZKMUIZ5CTe+ihsXirgV3Pk2ppeFtv4Q+PjPOZyc348ZVdpxvq3c0vLK6lp+vbCxubW9Y+/uNVWUSEwaOGKRbPtIEUYFaWiqGWnHkiDuM9LyhzfjeuuBSEUjca9HMely1Bc0pBhpY/XsE8+nIuIprwQVN0uDDF5BL5QIp+Wgwk/d46Ixi+Up9eySU3UmgovgzqAEZqr37C8viHDCidCYIaU6rhPrboqkppiRrOAlisQID1GfdAwKxInqppOrMnhknACGkTRPaDhxf0+kiCs14r7p5EgP1HxtbP5X6yQ6vOymVMSJJgJPF4UJgzqC44hgQCXBmo0MICyp+SvEA2Qy0SbIggnBnT95EZpnVdepunfnpdr1LI48OACHoAxccAFq4BbUQQNg8AiewSt4s56sF+vd+pi25qzZzD74I+vzB33snHM=</latexit><latexit sha1_base64="2wc3TY/SE1D2Fcqdd3fVdLwrY4o=">AAACFXicbZDLSgMxFIYz9VbrbdSlm9QiVKplRgTdCEU3LivYC3RKyWQybWiSGZKMUIZ5CTe+ihsXirgV3Pk2ppeFtv4Q+PjPOZyc348ZVdpxvq3c0vLK6lp+vbCxubW9Y+/uNVWUSEwaOGKRbPtIEUYFaWiqGWnHkiDuM9LyhzfjeuuBSEUjca9HMely1Bc0pBhpY/XsE8+nIuIprwQVN0uDDF5BL5QIp+Wgwk/d46Ixi+Up9eySU3UmgovgzqAEZqr37C8viHDCidCYIaU6rhPrboqkppiRrOAlisQID1GfdAwKxInqppOrMnhknACGkTRPaDhxf0+kiCs14r7p5EgP1HxtbP5X6yQ6vOymVMSJJgJPF4UJgzqC44hgQCXBmo0MICyp+SvEA2Qy0SbIggnBnT95EZpnVdepunfnpdr1LI48OACHoAxccAFq4BbUQQNg8AiewSt4s56sF+vd+pi25qzZzD74I+vzB33snHM=</latexit><latexit sha1_base64="2wc3TY/SE1D2Fcqdd3fVdLwrY4o=">AAACFXicbZDLSgMxFIYz9VbrbdSlm9QiVKplRgTdCEU3LivYC3RKyWQybWiSGZKMUIZ5CTe+ihsXirgV3Pk2ppeFtv4Q+PjPOZyc348ZVdpxvq3c0vLK6lp+vbCxubW9Y+/uNVWUSEwaOGKRbPtIEUYFaWiqGWnHkiDuM9LyhzfjeuuBSEUjca9HMely1Bc0pBhpY/XsE8+nIuIprwQVN0uDDF5BL5QIp+Wgwk/d46Ixi+Up9eySU3UmgovgzqAEZqr37C8viHDCidCYIaU6rhPrboqkppiRrOAlisQID1GfdAwKxInqppOrMnhknACGkTRPaDhxf0+kiCs14r7p5EgP1HxtbP5X6yQ6vOymVMSJJgJPF4UJgzqC44hgQCXBmo0MICyp+SvEA2Qy0SbIggnBnT95EZpnVdepunfnpdr1LI48OACHoAxccAFq4BbUQQNg8AiewSt4s56sF+vd+pi25qzZzD74I+vzB33snHM=</latexit>

16 x 16 pixel images with

d=5 will lead to 1010<latexit sha1_base64="PNDeVSHIe81Hvsgj3BXt8UVmb94=">AAAB7nicdVDLSgMxFL3js9ZX1aWbYBFcDYmI2l3RjcsK9gHtWDJppg3NZIYkI5ShH+HGhSJu/R53/o3pQ/B54MLhnHu5954wlcJYjN+9hcWl5ZXVwlpxfWNza7u0s9swSaYZr7NEJroVUsOlULxuhZW8lWpO41DyZji8nPjNO66NSNSNHaU8iGlfiUgwap3UJPg2J3jcLZWxX8GkckrQb0J8PEUZ5qh1S2+dXsKymCvLJDWmTXBqg5xqK5jk42InMzylbEj7vO2oojE3QT49d4wOndJDUaJdKYum6teJnMbGjOLQdcbUDsxPbyL+5bUzG50HuVBpZrlis0VRJpFN0OR31BOaMytHjlCmhbsVsQHVlFmXUNGF8Pkp+p80jn2CfXJ9Uq5ezOMowD4cwBEQOIMqXEEN6sBgCPfwCE9e6j14z97LrHXBm8/swTd4rx+w9I8m</latexit><latexit sha1_base64="PNDeVSHIe81Hvsgj3BXt8UVmb94=">AAAB7nicdVDLSgMxFL3js9ZX1aWbYBFcDYmI2l3RjcsK9gHtWDJppg3NZIYkI5ShH+HGhSJu/R53/o3pQ/B54MLhnHu5954wlcJYjN+9hcWl5ZXVwlpxfWNza7u0s9swSaYZr7NEJroVUsOlULxuhZW8lWpO41DyZji8nPjNO66NSNSNHaU8iGlfiUgwap3UJPg2J3jcLZWxX8GkckrQb0J8PEUZ5qh1S2+dXsKymCvLJDWmTXBqg5xqK5jk42InMzylbEj7vO2oojE3QT49d4wOndJDUaJdKYum6teJnMbGjOLQdcbUDsxPbyL+5bUzG50HuVBpZrlis0VRJpFN0OR31BOaMytHjlCmhbsVsQHVlFmXUNGF8Pkp+p80jn2CfXJ9Uq5ezOMowD4cwBEQOIMqXEEN6sBgCPfwCE9e6j14z97LrHXBm8/swTd4rx+w9I8m</latexit><latexit sha1_base64="PNDeVSHIe81Hvsgj3BXt8UVmb94=">AAAB7nicdVDLSgMxFL3js9ZX1aWbYBFcDYmI2l3RjcsK9gHtWDJppg3NZIYkI5ShH+HGhSJu/R53/o3pQ/B54MLhnHu5954wlcJYjN+9hcWl5ZXVwlpxfWNza7u0s9swSaYZr7NEJroVUsOlULxuhZW8lWpO41DyZji8nPjNO66NSNSNHaU8iGlfiUgwap3UJPg2J3jcLZWxX8GkckrQb0J8PEUZ5qh1S2+dXsKymCvLJDWmTXBqg5xqK5jk42InMzylbEj7vO2oojE3QT49d4wOndJDUaJdKYum6teJnMbGjOLQdcbUDsxPbyL+5bUzG50HuVBpZrlis0VRJpFN0OR31BOaMytHjlCmhbsVsQHVlFmXUNGF8Pkp+p80jn2CfXJ9Uq5ezOMowD4cwBEQOIMqXEEN6sBgCPfwCE9e6j14z97LrHXBm8/swTd4rx+w9I8m</latexit><latexit sha1_base64="PNDeVSHIe81Hvsgj3BXt8UVmb94=">AAAB7nicdVDLSgMxFL3js9ZX1aWbYBFcDYmI2l3RjcsK9gHtWDJppg3NZIYkI5ShH+HGhSJu/R53/o3pQ/B54MLhnHu5954wlcJYjN+9hcWl5ZXVwlpxfWNza7u0s9swSaYZr7NEJroVUsOlULxuhZW8lWpO41DyZji8nPjNO66NSNSNHaU8iGlfiUgwap3UJPg2J3jcLZWxX8GkckrQb0J8PEUZ5qh1S2+dXsKymCvLJDWmTXBqg5xqK5jk42InMzylbEj7vO2oojE3QT49d4wOndJDUaJdKYum6teJnMbGjOLQdcbUDsxPbyL+5bUzG50HuVBpZrlis0VRJpFN0OR31BOaMytHjlCmhbsVsQHVlFmXUNGF8Pkp+p80jn2CfXJ9Uq5ezOMowD4cwBEQOIMqXEEN6sBgCPfwCE9e6j14z97LrHXBm8/swTd4rx+w9I8m</latexit>

Page 12: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Can we have the best of two worlds?

Yes, we can!

Page 13: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Can we have the best of two worlds?

Yes, we can!

Given any algorithm that only requires dot products

Page 14: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Can we have the best of two worlds?

Yes, we can!

Given any algorithm that only requires dot products

We can work in the high-dimensional space without ever mapping into that space!

Page 15: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

The kernel trick

k(x, x0) = h�(x),�(x0)i<latexit sha1_base64="cXyk4E9OLouOqndDqiTZd/gEeNU=">AAACHnicbVDLSgMxFM3UV62vqks3wSK0UMqMKLoRim5cVrAP6NSSSW/b0ExmSDLSMvRL3PgrblwoIrjSvzGdVtDWAyGHc84luccLOVPatr+s1NLyyupaej2zsbm1vZPd3aupIJIUqjTggWx4RAFnAqqaaQ6NUALxPQ51b3A18ev3IBULxK0ehdDySU+wLqNEG6mdPR3kh8XhnRtK5kMBX2CXE9HjgN1Kn+WHheL0/gm4MnHb2ZxdshPgReLMSA7NUGlnP9xOQCMfhKacKNV07FC3YiI1oxzGGTdSEBI6ID1oGiqID6oVJ+uN8ZFROrgbSHOExon6eyImvlIj3zNJn+i+mvcm4n9eM9Ld81bMRBhpEHT6UDfiWAd40hXuMAlU85EhhEpm/oppn0hCtWk0Y0pw5ldeJLXjkmOXnJuTXPlyVkcaHaBDlEcOOkNldI0qqIooekBP6AW9Wo/Ws/VmvU+jKWs2s4/+wPr8BhSEoTw=</latexit><latexit sha1_base64="cXyk4E9OLouOqndDqiTZd/gEeNU=">AAACHnicbVDLSgMxFM3UV62vqks3wSK0UMqMKLoRim5cVrAP6NSSSW/b0ExmSDLSMvRL3PgrblwoIrjSvzGdVtDWAyGHc84luccLOVPatr+s1NLyyupaej2zsbm1vZPd3aupIJIUqjTggWx4RAFnAqqaaQ6NUALxPQ51b3A18ev3IBULxK0ehdDySU+wLqNEG6mdPR3kh8XhnRtK5kMBX2CXE9HjgN1Kn+WHheL0/gm4MnHb2ZxdshPgReLMSA7NUGlnP9xOQCMfhKacKNV07FC3YiI1oxzGGTdSEBI6ID1oGiqID6oVJ+uN8ZFROrgbSHOExon6eyImvlIj3zNJn+i+mvcm4n9eM9Ld81bMRBhpEHT6UDfiWAd40hXuMAlU85EhhEpm/oppn0hCtWk0Y0pw5ldeJLXjkmOXnJuTXPlyVkcaHaBDlEcOOkNldI0qqIooekBP6AW9Wo/Ws/VmvU+jKWs2s4/+wPr8BhSEoTw=</latexit><latexit sha1_base64="cXyk4E9OLouOqndDqiTZd/gEeNU=">AAACHnicbVDLSgMxFM3UV62vqks3wSK0UMqMKLoRim5cVrAP6NSSSW/b0ExmSDLSMvRL3PgrblwoIrjSvzGdVtDWAyGHc84luccLOVPatr+s1NLyyupaej2zsbm1vZPd3aupIJIUqjTggWx4RAFnAqqaaQ6NUALxPQ51b3A18ev3IBULxK0ehdDySU+wLqNEG6mdPR3kh8XhnRtK5kMBX2CXE9HjgN1Kn+WHheL0/gm4MnHb2ZxdshPgReLMSA7NUGlnP9xOQCMfhKacKNV07FC3YiI1oxzGGTdSEBI6ID1oGiqID6oVJ+uN8ZFROrgbSHOExon6eyImvlIj3zNJn+i+mvcm4n9eM9Ld81bMRBhpEHT6UDfiWAd40hXuMAlU85EhhEpm/oppn0hCtWk0Y0pw5ldeJLXjkmOXnJuTXPlyVkcaHaBDlEcOOkNldI0qqIooekBP6AW9Wo/Ws/VmvU+jKWs2s4/+wPr8BhSEoTw=</latexit><latexit sha1_base64="cXyk4E9OLouOqndDqiTZd/gEeNU=">AAACHnicbVDLSgMxFM3UV62vqks3wSK0UMqMKLoRim5cVrAP6NSSSW/b0ExmSDLSMvRL3PgrblwoIrjSvzGdVtDWAyGHc84luccLOVPatr+s1NLyyupaej2zsbm1vZPd3aupIJIUqjTggWx4RAFnAqqaaQ6NUALxPQ51b3A18ev3IBULxK0ehdDySU+wLqNEG6mdPR3kh8XhnRtK5kMBX2CXE9HjgN1Kn+WHheL0/gm4MnHb2ZxdshPgReLMSA7NUGlnP9xOQCMfhKacKNV07FC3YiI1oxzGGTdSEBI6ID1oGiqID6oVJ+uN8ZFROrgbSHOExon6eyImvlIj3zNJn+i+mvcm4n9eM9Ld81bMRBhpEHT6UDfiWAd40hXuMAlU85EhhEpm/oppn0hCtWk0Y0pw5ldeJLXjkmOXnJuTXPlyVkcaHaBDlEcOOkNldI0qqIooekBP6AW9Wo/Ws/VmvU+jKWs2s4/+wPr8BhSEoTw=</latexit>

Given any algorithm that only requires dot products, we can construct different nonlinear versions of it, by setting different kernel functions

Page 16: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

The kernel trick

Let’s apply the kernel trick to PCA…

k(x, x0) = h�(x),�(x0)i<latexit sha1_base64="cXyk4E9OLouOqndDqiTZd/gEeNU=">AAACHnicbVDLSgMxFM3UV62vqks3wSK0UMqMKLoRim5cVrAP6NSSSW/b0ExmSDLSMvRL3PgrblwoIrjSvzGdVtDWAyGHc84luccLOVPatr+s1NLyyupaej2zsbm1vZPd3aupIJIUqjTggWx4RAFnAqqaaQ6NUALxPQ51b3A18ev3IBULxK0ehdDySU+wLqNEG6mdPR3kh8XhnRtK5kMBX2CXE9HjgN1Kn+WHheL0/gm4MnHb2ZxdshPgReLMSA7NUGlnP9xOQCMfhKacKNV07FC3YiI1oxzGGTdSEBI6ID1oGiqID6oVJ+uN8ZFROrgbSHOExon6eyImvlIj3zNJn+i+mvcm4n9eM9Ld81bMRBhpEHT6UDfiWAd40hXuMAlU85EhhEpm/oppn0hCtWk0Y0pw5ldeJLXjkmOXnJuTXPlyVkcaHaBDlEcOOkNldI0qqIooekBP6AW9Wo/Ws/VmvU+jKWs2s4/+wPr8BhSEoTw=</latexit><latexit sha1_base64="cXyk4E9OLouOqndDqiTZd/gEeNU=">AAACHnicbVDLSgMxFM3UV62vqks3wSK0UMqMKLoRim5cVrAP6NSSSW/b0ExmSDLSMvRL3PgrblwoIrjSvzGdVtDWAyGHc84luccLOVPatr+s1NLyyupaej2zsbm1vZPd3aupIJIUqjTggWx4RAFnAqqaaQ6NUALxPQ51b3A18ev3IBULxK0ehdDySU+wLqNEG6mdPR3kh8XhnRtK5kMBX2CXE9HjgN1Kn+WHheL0/gm4MnHb2ZxdshPgReLMSA7NUGlnP9xOQCMfhKacKNV07FC3YiI1oxzGGTdSEBI6ID1oGiqID6oVJ+uN8ZFROrgbSHOExon6eyImvlIj3zNJn+i+mvcm4n9eM9Ld81bMRBhpEHT6UDfiWAd40hXuMAlU85EhhEpm/oppn0hCtWk0Y0pw5ldeJLXjkmOXnJuTXPlyVkcaHaBDlEcOOkNldI0qqIooekBP6AW9Wo/Ws/VmvU+jKWs2s4/+wPr8BhSEoTw=</latexit><latexit sha1_base64="cXyk4E9OLouOqndDqiTZd/gEeNU=">AAACHnicbVDLSgMxFM3UV62vqks3wSK0UMqMKLoRim5cVrAP6NSSSW/b0ExmSDLSMvRL3PgrblwoIrjSvzGdVtDWAyGHc84luccLOVPatr+s1NLyyupaej2zsbm1vZPd3aupIJIUqjTggWx4RAFnAqqaaQ6NUALxPQ51b3A18ev3IBULxK0ehdDySU+wLqNEG6mdPR3kh8XhnRtK5kMBX2CXE9HjgN1Kn+WHheL0/gm4MnHb2ZxdshPgReLMSA7NUGlnP9xOQCMfhKacKNV07FC3YiI1oxzGGTdSEBI6ID1oGiqID6oVJ+uN8ZFROrgbSHOExon6eyImvlIj3zNJn+i+mvcm4n9eM9Ld81bMRBhpEHT6UDfiWAd40hXuMAlU85EhhEpm/oppn0hCtWk0Y0pw5ldeJLXjkmOXnJuTXPlyVkcaHaBDlEcOOkNldI0qqIooekBP6AW9Wo/Ws/VmvU+jKWs2s4/+wPr8BhSEoTw=</latexit><latexit sha1_base64="cXyk4E9OLouOqndDqiTZd/gEeNU=">AAACHnicbVDLSgMxFM3UV62vqks3wSK0UMqMKLoRim5cVrAP6NSSSW/b0ExmSDLSMvRL3PgrblwoIrjSvzGdVtDWAyGHc84luccLOVPatr+s1NLyyupaej2zsbm1vZPd3aupIJIUqjTggWx4RAFnAqqaaQ6NUALxPQ51b3A18ev3IBULxK0ehdDySU+wLqNEG6mdPR3kh8XhnRtK5kMBX2CXE9HjgN1Kn+WHheL0/gm4MnHb2ZxdshPgReLMSA7NUGlnP9xOQCMfhKacKNV07FC3YiI1oxzGGTdSEBI6ID1oGiqID6oVJ+uN8ZFROrgbSHOExon6eyImvlIj3zNJn+i+mvcm4n9eM9Ld81bMRBhpEHT6UDfiWAd40hXuMAlU85EhhEpm/oppn0hCtWk0Y0pw5ldeJLXjkmOXnJuTXPlyVkcaHaBDlEcOOkNldI0qqIooekBP6AW9Wo/Ws/VmvU+jKWs2s4/+wPr8BhSEoTw=</latexit>

Given any algorithm that only requires dot products, we can construct different nonlinear versions of it, by setting different kernel functions

Page 17: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

PCA

ai 2 Rm<latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit>

A 2 Rn⇥m<latexit sha1_base64="tU26yT4y0zROPaCEGhakhgjzO58=">AAACBnicbVDLSsNAFJ34rPUVdSnCYBFclUQEXVbduKxiH9DEMplO2qGTSZi5EUrIyo2/4saFIm79Bnf+jZO2C209MHDmnHu5954gEVyD43xbC4tLyyurpbXy+sbm1ra9s9vUcaooa9BYxKodEM0El6wBHARrJ4qRKBCsFQyvCr/1wJTmsbyDUcL8iPQlDzklYKSufXCBPS6xFxEYBEF2m99n5gc8YhpHedeuOFVnDDxP3CmpoCnqXfvL68U0jZgEKojWHddJwM+IAk4Fy8teqllC6JD0WcdQScwcPxufkeMjo/RwGCvzJOCx+rsjI5HWoygwlcW6etYrxP+8TgrhuZ9xmaTAJJ0MClOBIcZFJrjHFaMgRoYQqrjZFdMBUYSCSa5sQnBnT54nzZOq61Tdm9NK7XIaRwnto0N0jFx0hmroGtVRA1H0iJ7RK3qznqwX6936mJQuWNOePfQH1ucPg9+YhA==</latexit><latexit sha1_base64="tU26yT4y0zROPaCEGhakhgjzO58=">AAACBnicbVDLSsNAFJ34rPUVdSnCYBFclUQEXVbduKxiH9DEMplO2qGTSZi5EUrIyo2/4saFIm79Bnf+jZO2C209MHDmnHu5954gEVyD43xbC4tLyyurpbXy+sbm1ra9s9vUcaooa9BYxKodEM0El6wBHARrJ4qRKBCsFQyvCr/1wJTmsbyDUcL8iPQlDzklYKSufXCBPS6xFxEYBEF2m99n5gc8YhpHedeuOFVnDDxP3CmpoCnqXfvL68U0jZgEKojWHddJwM+IAk4Fy8teqllC6JD0WcdQScwcPxufkeMjo/RwGCvzJOCx+rsjI5HWoygwlcW6etYrxP+8TgrhuZ9xmaTAJJ0MClOBIcZFJrjHFaMgRoYQqrjZFdMBUYSCSa5sQnBnT54nzZOq61Tdm9NK7XIaRwnto0N0jFx0hmroGtVRA1H0iJ7RK3qznqwX6936mJQuWNOePfQH1ucPg9+YhA==</latexit><latexit sha1_base64="tU26yT4y0zROPaCEGhakhgjzO58=">AAACBnicbVDLSsNAFJ34rPUVdSnCYBFclUQEXVbduKxiH9DEMplO2qGTSZi5EUrIyo2/4saFIm79Bnf+jZO2C209MHDmnHu5954gEVyD43xbC4tLyyurpbXy+sbm1ra9s9vUcaooa9BYxKodEM0El6wBHARrJ4qRKBCsFQyvCr/1wJTmsbyDUcL8iPQlDzklYKSufXCBPS6xFxEYBEF2m99n5gc8YhpHedeuOFVnDDxP3CmpoCnqXfvL68U0jZgEKojWHddJwM+IAk4Fy8teqllC6JD0WcdQScwcPxufkeMjo/RwGCvzJOCx+rsjI5HWoygwlcW6etYrxP+8TgrhuZ9xmaTAJJ0MClOBIcZFJrjHFaMgRoYQqrjZFdMBUYSCSa5sQnBnT54nzZOq61Tdm9NK7XIaRwnto0N0jFx0hmroGtVRA1H0iJ7RK3qznqwX6936mJQuWNOePfQH1ucPg9+YhA==</latexit><latexit sha1_base64="tU26yT4y0zROPaCEGhakhgjzO58=">AAACBnicbVDLSsNAFJ34rPUVdSnCYBFclUQEXVbduKxiH9DEMplO2qGTSZi5EUrIyo2/4saFIm79Bnf+jZO2C209MHDmnHu5954gEVyD43xbC4tLyyurpbXy+sbm1ra9s9vUcaooa9BYxKodEM0El6wBHARrJ4qRKBCsFQyvCr/1wJTmsbyDUcL8iPQlDzklYKSufXCBPS6xFxEYBEF2m99n5gc8YhpHedeuOFVnDDxP3CmpoCnqXfvL68U0jZgEKojWHddJwM+IAk4Fy8teqllC6JD0WcdQScwcPxufkeMjo/RwGCvzJOCx+rsjI5HWoygwlcW6etYrxP+8TgrhuZ9xmaTAJJ0MClOBIcZFJrjHFaMgRoYQqrjZFdMBUYSCSa5sQnBnT54nzZOq61Tdm9NK7XIaRwnto0N0jFx0hmroGtVRA1H0iJ7RK3qznqwX6936mJQuWNOePfQH1ucPg9+YhA==</latexit>

AT 1 = 0<latexit sha1_base64="cyr5ZsGXW9pO0KmpWskPSAmVz34=">AAAB8HicbVBNSwMxEJ2tX7V+VT16CRbBU9kVQS9C1YvHCv2Sdi3ZNNuGJtklyQpl6a/w4kERr/4cb/4b0+0etPXBwOO9GWbmBTFn2rjut1NYWV1b3yhulra2d3b3yvsHLR0litAmiXikOgHWlDNJm4YZTjuxolgEnLaD8e3Mbz9RpVkkG2YSU1/goWQhI9hY6eH6sYE8dIXcfrniVt0MaJl4OalAjnq//NUbRCQRVBrCsdZdz42Nn2JlGOF0WuolmsaYjPGQdi2VWFDtp9nBU3RilQEKI2VLGpSpvydSLLSeiMB2CmxGetGbif953cSEl37KZJwYKsl8UZhwZCI0+x4NmKLE8IklmChmb0VkhBUmxmZUsiF4iy8vk9ZZ1XOr3v15pXaTx1GEIziGU/DgAmpwB3VoAgEBz/AKb45yXpx352PeWnDymUP4A+fzB1IhjsU=</latexit><latexit sha1_base64="cyr5ZsGXW9pO0KmpWskPSAmVz34=">AAAB8HicbVBNSwMxEJ2tX7V+VT16CRbBU9kVQS9C1YvHCv2Sdi3ZNNuGJtklyQpl6a/w4kERr/4cb/4b0+0etPXBwOO9GWbmBTFn2rjut1NYWV1b3yhulra2d3b3yvsHLR0litAmiXikOgHWlDNJm4YZTjuxolgEnLaD8e3Mbz9RpVkkG2YSU1/goWQhI9hY6eH6sYE8dIXcfrniVt0MaJl4OalAjnq//NUbRCQRVBrCsdZdz42Nn2JlGOF0WuolmsaYjPGQdi2VWFDtp9nBU3RilQEKI2VLGpSpvydSLLSeiMB2CmxGetGbif953cSEl37KZJwYKsl8UZhwZCI0+x4NmKLE8IklmChmb0VkhBUmxmZUsiF4iy8vk9ZZ1XOr3v15pXaTx1GEIziGU/DgAmpwB3VoAgEBz/AKb45yXpx352PeWnDymUP4A+fzB1IhjsU=</latexit><latexit sha1_base64="cyr5ZsGXW9pO0KmpWskPSAmVz34=">AAAB8HicbVBNSwMxEJ2tX7V+VT16CRbBU9kVQS9C1YvHCv2Sdi3ZNNuGJtklyQpl6a/w4kERr/4cb/4b0+0etPXBwOO9GWbmBTFn2rjut1NYWV1b3yhulra2d3b3yvsHLR0litAmiXikOgHWlDNJm4YZTjuxolgEnLaD8e3Mbz9RpVkkG2YSU1/goWQhI9hY6eH6sYE8dIXcfrniVt0MaJl4OalAjnq//NUbRCQRVBrCsdZdz42Nn2JlGOF0WuolmsaYjPGQdi2VWFDtp9nBU3RilQEKI2VLGpSpvydSLLSeiMB2CmxGetGbif953cSEl37KZJwYKsl8UZhwZCI0+x4NmKLE8IklmChmb0VkhBUmxmZUsiF4iy8vk9ZZ1XOr3v15pXaTx1GEIziGU/DgAmpwB3VoAgEBz/AKb45yXpx352PeWnDymUP4A+fzB1IhjsU=</latexit><latexit sha1_base64="cyr5ZsGXW9pO0KmpWskPSAmVz34=">AAAB8HicbVBNSwMxEJ2tX7V+VT16CRbBU9kVQS9C1YvHCv2Sdi3ZNNuGJtklyQpl6a/w4kERr/4cb/4b0+0etPXBwOO9GWbmBTFn2rjut1NYWV1b3yhulra2d3b3yvsHLR0litAmiXikOgHWlDNJm4YZTjuxolgEnLaD8e3Mbz9RpVkkG2YSU1/goWQhI9hY6eH6sYE8dIXcfrniVt0MaJl4OalAjnq//NUbRCQRVBrCsdZdz42Nn2JlGOF0WuolmsaYjPGQdi2VWFDtp9nBU3RilQEKI2VLGpSpvydSLLSeiMB2CmxGetGbif953cSEl37KZJwYKsl8UZhwZCI0+x4NmKLE8IklmChmb0VkhBUmxmZUsiF4iy8vk9ZZ1XOr3v15pXaTx1GEIziGU/DgAmpwB3VoAgEBz/AKb45yXpx352PeWnDymUP4A+fzB1IhjsU=</latexit>

,X

i

ai = 0<latexit sha1_base64="+PL7JPH+bqHynUJosSdRIxohH5A=">AAACB3icbVBNS8NAEN34WetX1KMgi0XwVBIR9CIUvXjwUMF+QBPCZrtpl242YXeilNCbF/+KFw+KePUvePPfuG1z0NYHA4/3ZpiZF6aCa3Ccb2thcWl5ZbW0Vl7f2Nzatnd2mzrJFGUNmohEtUOimeCSNYCDYO1UMRKHgrXCwdXYb90zpXki72CYMj8mPckjTgkYKbAPvBsWgeK9PhClkgfs6SwOOCamLrAT2BWn6kyA54lbkAoqUA/sL6+b0CxmEqggWndcJwU/Jwo4FWxU9jLNUkIHpMc6hkoSM+3nkz9G+MgoXRwlypQEPFF/T+Qk1noYh6YzJtDXs95Y/M/rZBCd+zmXaQZM0umiKBMYEjwOBXe5YhTE0BBCFTe3YtonilAw0ZVNCO7sy/OkeVJ1nap7e1qpXRZxlNA+OkTHyEVnqIauUR01EEWP6Bm9ojfryXqx3q2PaeuCVczsoT+wPn8Az0OYlg==</latexit><latexit sha1_base64="+PL7JPH+bqHynUJosSdRIxohH5A=">AAACB3icbVBNS8NAEN34WetX1KMgi0XwVBIR9CIUvXjwUMF+QBPCZrtpl242YXeilNCbF/+KFw+KePUvePPfuG1z0NYHA4/3ZpiZF6aCa3Ccb2thcWl5ZbW0Vl7f2Nzatnd2mzrJFGUNmohEtUOimeCSNYCDYO1UMRKHgrXCwdXYb90zpXki72CYMj8mPckjTgkYKbAPvBsWgeK9PhClkgfs6SwOOCamLrAT2BWn6kyA54lbkAoqUA/sL6+b0CxmEqggWndcJwU/Jwo4FWxU9jLNUkIHpMc6hkoSM+3nkz9G+MgoXRwlypQEPFF/T+Qk1noYh6YzJtDXs95Y/M/rZBCd+zmXaQZM0umiKBMYEjwOBXe5YhTE0BBCFTe3YtonilAw0ZVNCO7sy/OkeVJ1nap7e1qpXRZxlNA+OkTHyEVnqIauUR01EEWP6Bm9ojfryXqx3q2PaeuCVczsoT+wPn8Az0OYlg==</latexit><latexit sha1_base64="+PL7JPH+bqHynUJosSdRIxohH5A=">AAACB3icbVBNS8NAEN34WetX1KMgi0XwVBIR9CIUvXjwUMF+QBPCZrtpl242YXeilNCbF/+KFw+KePUvePPfuG1z0NYHA4/3ZpiZF6aCa3Ccb2thcWl5ZbW0Vl7f2Nzatnd2mzrJFGUNmohEtUOimeCSNYCDYO1UMRKHgrXCwdXYb90zpXki72CYMj8mPckjTgkYKbAPvBsWgeK9PhClkgfs6SwOOCamLrAT2BWn6kyA54lbkAoqUA/sL6+b0CxmEqggWndcJwU/Jwo4FWxU9jLNUkIHpMc6hkoSM+3nkz9G+MgoXRwlypQEPFF/T+Qk1noYh6YzJtDXs95Y/M/rZBCd+zmXaQZM0umiKBMYEjwOBXe5YhTE0BBCFTe3YtonilAw0ZVNCO7sy/OkeVJ1nap7e1qpXRZxlNA+OkTHyEVnqIauUR01EEWP6Bm9ojfryXqx3q2PaeuCVczsoT+wPn8Az0OYlg==</latexit><latexit sha1_base64="+PL7JPH+bqHynUJosSdRIxohH5A=">AAACB3icbVBNS8NAEN34WetX1KMgi0XwVBIR9CIUvXjwUMF+QBPCZrtpl242YXeilNCbF/+KFw+KePUvePPfuG1z0NYHA4/3ZpiZF6aCa3Ccb2thcWl5ZbW0Vl7f2Nzatnd2mzrJFGUNmohEtUOimeCSNYCDYO1UMRKHgrXCwdXYb90zpXki72CYMj8mPckjTgkYKbAPvBsWgeK9PhClkgfs6SwOOCamLrAT2BWn6kyA54lbkAoqUA/sL6+b0CxmEqggWndcJwU/Jwo4FWxU9jLNUkIHpMc6hkoSM+3nkz9G+MgoXRwlypQEPFF/T+Qk1noYh6YzJtDXs95Y/M/rZBCd+zmXaQZM0umiKBMYEjwOBXe5YhTE0BBCFTe3YtonilAw0ZVNCO7sy/OkeVJ1nap7e1qpXRZxlNA+OkTHyEVnqIauUR01EEWP6Bm9ojfryXqx3q2PaeuCVczsoT+wPn8Az0OYlg==</latexit>

C =1

n

X

i

aTi ai<latexit sha1_base64="1sPqTb8QurV3li5czvZ+EP5N9GY=">AAACBnicbVDLSsNAFL2pr1pfUZciDBbBVUlE0I1Q7MZlhb6giWEynbRDJ5MwMxFK6MqNv+LGhSJu/QZ3/o3Tx0JbD9zL4Zx7mbknTDlT2nG+rcLK6tr6RnGztLW9s7tn7x+0VJJJQpsk4YnshFhRzgRtaqY57aSS4jjktB0OaxO//UClYolo6FFK/Rj3BYsYwdpIgX1cQ9fIiyQmyEUCeSqLA4ZwwO4bpgV22ak4U6Bl4s5JGeaoB/aX10tIFlOhCcdKdV0n1X6OpWaE03HJyxRNMRniPu0aKnBMlZ9PzxijU6P0UJRIU0Kjqfp7I8exUqM4NJMx1gO16E3E/7xupqMrP2cizTQVZPZQlHGkEzTJBPWYpETzkSGYSGb+isgAm0y0Sa5kQnAXT14mrfOK61Tcu4ty9WYeRxGO4ATOwIVLqMIt1KEJBB7hGV7hzXqyXqx362M2WrDmO4fwB9bnD1IElyE=</latexit><latexit sha1_base64="1sPqTb8QurV3li5czvZ+EP5N9GY=">AAACBnicbVDLSsNAFL2pr1pfUZciDBbBVUlE0I1Q7MZlhb6giWEynbRDJ5MwMxFK6MqNv+LGhSJu/QZ3/o3Tx0JbD9zL4Zx7mbknTDlT2nG+rcLK6tr6RnGztLW9s7tn7x+0VJJJQpsk4YnshFhRzgRtaqY57aSS4jjktB0OaxO//UClYolo6FFK/Rj3BYsYwdpIgX1cQ9fIiyQmyEUCeSqLA4ZwwO4bpgV22ak4U6Bl4s5JGeaoB/aX10tIFlOhCcdKdV0n1X6OpWaE03HJyxRNMRniPu0aKnBMlZ9PzxijU6P0UJRIU0Kjqfp7I8exUqM4NJMx1gO16E3E/7xupqMrP2cizTQVZPZQlHGkEzTJBPWYpETzkSGYSGb+isgAm0y0Sa5kQnAXT14mrfOK61Tcu4ty9WYeRxGO4ATOwIVLqMIt1KEJBB7hGV7hzXqyXqx362M2WrDmO4fwB9bnD1IElyE=</latexit><latexit sha1_base64="1sPqTb8QurV3li5czvZ+EP5N9GY=">AAACBnicbVDLSsNAFL2pr1pfUZciDBbBVUlE0I1Q7MZlhb6giWEynbRDJ5MwMxFK6MqNv+LGhSJu/QZ3/o3Tx0JbD9zL4Zx7mbknTDlT2nG+rcLK6tr6RnGztLW9s7tn7x+0VJJJQpsk4YnshFhRzgRtaqY57aSS4jjktB0OaxO//UClYolo6FFK/Rj3BYsYwdpIgX1cQ9fIiyQmyEUCeSqLA4ZwwO4bpgV22ak4U6Bl4s5JGeaoB/aX10tIFlOhCcdKdV0n1X6OpWaE03HJyxRNMRniPu0aKnBMlZ9PzxijU6P0UJRIU0Kjqfp7I8exUqM4NJMx1gO16E3E/7xupqMrP2cizTQVZPZQlHGkEzTJBPWYpETzkSGYSGb+isgAm0y0Sa5kQnAXT14mrfOK61Tcu4ty9WYeRxGO4ATOwIVLqMIt1KEJBB7hGV7hzXqyXqx362M2WrDmO4fwB9bnD1IElyE=</latexit><latexit sha1_base64="1sPqTb8QurV3li5czvZ+EP5N9GY=">AAACBnicbVDLSsNAFL2pr1pfUZciDBbBVUlE0I1Q7MZlhb6giWEynbRDJ5MwMxFK6MqNv+LGhSJu/QZ3/o3Tx0JbD9zL4Zx7mbknTDlT2nG+rcLK6tr6RnGztLW9s7tn7x+0VJJJQpsk4YnshFhRzgRtaqY57aSS4jjktB0OaxO//UClYolo6FFK/Rj3BYsYwdpIgX1cQ9fIiyQmyEUCeSqLA4ZwwO4bpgV22ak4U6Bl4s5JGeaoB/aX10tIFlOhCcdKdV0n1X6OpWaE03HJyxRNMRniPu0aKnBMlZ9PzxijU6P0UJRIU0Kjqfp7I8exUqM4NJMx1gO16E3E/7xupqMrP2cizTQVZPZQlHGkEzTJBPWYpETzkSGYSGb+isgAm0y0Sa5kQnAXT14mrfOK61Tcu4ty9WYeRxGO4ATOwIVLqMIt1KEJBB7hGV7hzXqyXqx362M2WrDmO4fwB9bnD1IElyE=</latexit>

Notation:

Centered data:

Find principal components by diagonalizing

Cv = �v<latexit sha1_base64="6pWje61hjjUi8/u5tMDSz6y2nrc=">AAAB9XicbVDLSgMxFL2pr1pfVZdugkVwVWZE0I1Q7MZlBfuAdiyZTKYNzWSGJFMpQ//DjQtF3Pov7vwb03YW2nogcDjnXO7N8RPBtXGcb1RYW9/Y3Cpul3Z29/YPyodHLR2nirImjUWsOj7RTHDJmoYbwTqJYiTyBWv7o/rMb4+Z0jyWD2aSMC8iA8lDTomx0mN9jG9wT9h8QPC4X644VWcOvErcnFQgR6Nf/uoFMU0jJg0VROuu6yTGy4gynAo2LfVSzRJCR2TAupZKEjHtZfOrp/jMKgEOY2WfNHiu/p7ISKT1JPJtMiJmqJe9mfif101NeO1lXCapYZIuFoWpwCbGswpwwBWjRkwsIVRxeyumQ6IINbaoki3BXf7yKmldVF2n6t5fVmq3eR1FOIFTOAcXrqAGd9CAJlBQ8Ayv8Iae0At6Rx+LaAHlM8fwB+jzBwxWkY8=</latexit><latexit sha1_base64="6pWje61hjjUi8/u5tMDSz6y2nrc=">AAAB9XicbVDLSgMxFL2pr1pfVZdugkVwVWZE0I1Q7MZlBfuAdiyZTKYNzWSGJFMpQ//DjQtF3Pov7vwb03YW2nogcDjnXO7N8RPBtXGcb1RYW9/Y3Cpul3Z29/YPyodHLR2nirImjUWsOj7RTHDJmoYbwTqJYiTyBWv7o/rMb4+Z0jyWD2aSMC8iA8lDTomx0mN9jG9wT9h8QPC4X644VWcOvErcnFQgR6Nf/uoFMU0jJg0VROuu6yTGy4gynAo2LfVSzRJCR2TAupZKEjHtZfOrp/jMKgEOY2WfNHiu/p7ISKT1JPJtMiJmqJe9mfif101NeO1lXCapYZIuFoWpwCbGswpwwBWjRkwsIVRxeyumQ6IINbaoki3BXf7yKmldVF2n6t5fVmq3eR1FOIFTOAcXrqAGd9CAJlBQ8Ayv8Iae0At6Rx+LaAHlM8fwB+jzBwxWkY8=</latexit><latexit sha1_base64="6pWje61hjjUi8/u5tMDSz6y2nrc=">AAAB9XicbVDLSgMxFL2pr1pfVZdugkVwVWZE0I1Q7MZlBfuAdiyZTKYNzWSGJFMpQ//DjQtF3Pov7vwb03YW2nogcDjnXO7N8RPBtXGcb1RYW9/Y3Cpul3Z29/YPyodHLR2nirImjUWsOj7RTHDJmoYbwTqJYiTyBWv7o/rMb4+Z0jyWD2aSMC8iA8lDTomx0mN9jG9wT9h8QPC4X644VWcOvErcnFQgR6Nf/uoFMU0jJg0VROuu6yTGy4gynAo2LfVSzRJCR2TAupZKEjHtZfOrp/jMKgEOY2WfNHiu/p7ISKT1JPJtMiJmqJe9mfif101NeO1lXCapYZIuFoWpwCbGswpwwBWjRkwsIVRxeyumQ6IINbaoki3BXf7yKmldVF2n6t5fVmq3eR1FOIFTOAcXrqAGd9CAJlBQ8Ayv8Iae0At6Rx+LaAHlM8fwB+jzBwxWkY8=</latexit><latexit sha1_base64="6pWje61hjjUi8/u5tMDSz6y2nrc=">AAAB9XicbVDLSgMxFL2pr1pfVZdugkVwVWZE0I1Q7MZlBfuAdiyZTKYNzWSGJFMpQ//DjQtF3Pov7vwb03YW2nogcDjnXO7N8RPBtXGcb1RYW9/Y3Cpul3Z29/YPyodHLR2nirImjUWsOj7RTHDJmoYbwTqJYiTyBWv7o/rMb4+Z0jyWD2aSMC8iA8lDTomx0mN9jG9wT9h8QPC4X644VWcOvErcnFQgR6Nf/uoFMU0jJg0VROuu6yTGy4gynAo2LfVSzRJCR2TAupZKEjHtZfOrp/jMKgEOY2WfNHiu/p7ISKT1JPJtMiJmqJe9mfif101NeO1lXCapYZIuFoWpwCbGswpwwBWjRkwsIVRxeyumQ6IINbaoki3BXf7yKmldVF2n6t5fVmq3eR1FOIFTOAcXrqAGd9CAJlBQ8Ayv8Iae0At6Rx+LaAHlM8fwB+jzBwxWkY8=</latexit>

Page 18: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Using PCA

1. Find principal components, by computing eigenvectors sorted in the order of decreasing eigenvalue and establish a cutoff;

2. Project test points onto principal components;

3. Use coefficients for visualization, classification, regression, etc.

Page 19: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

PCA in dot products

Principal directions lie in the span of data points

v =X

i

↵iaTi

<latexit sha1_base64="mXTgWYhfQEyQqaBem9iUFe7iQ3E=">AAACAnicbZDLSsNAFIZP6q3WW9SVuBksgquSiKAboejGZYXeoIlhMp20QyeTMDMplFDc+CpuXCji1qdw59s4vSy09YcDH/85h5nzhylnSjvOt1VYWV1b3yhulra2d3b37P2DpkoySWiDJDyR7RArypmgDc00p+1UUhyHnLbCwe2k3xpSqVgi6nqUUj/GPcEiRrA2VmAfDdE18lQWBwx5mKd9bMDUQz2wy07FmQotgzuHMsxVC+wvr5uQLKZCE46V6rhOqv0cS80Ip+OSlymaYjLAPdoxKHBMlZ9PTxijU+N0UZRIU0Kjqft7I8exUqM4NJMx1n212JuY//U6mY6u/JyJNNNUkNlDUcaRTtAkD9RlkhLNRwYwkcz8FZE+lphok1rJhOAunrwMzfOK61Tc+4ty9WYeRxGO4QTOwIVLqMId1KABBB7hGV7hzXqyXqx362M2WrDmO4fwR9bnD7NQllY=</latexit><latexit sha1_base64="mXTgWYhfQEyQqaBem9iUFe7iQ3E=">AAACAnicbZDLSsNAFIZP6q3WW9SVuBksgquSiKAboejGZYXeoIlhMp20QyeTMDMplFDc+CpuXCji1qdw59s4vSy09YcDH/85h5nzhylnSjvOt1VYWV1b3yhulra2d3b37P2DpkoySWiDJDyR7RArypmgDc00p+1UUhyHnLbCwe2k3xpSqVgi6nqUUj/GPcEiRrA2VmAfDdE18lQWBwx5mKd9bMDUQz2wy07FmQotgzuHMsxVC+wvr5uQLKZCE46V6rhOqv0cS80Ip+OSlymaYjLAPdoxKHBMlZ9PTxijU+N0UZRIU0Kjqft7I8exUqM4NJMx1n212JuY//U6mY6u/JyJNNNUkNlDUcaRTtAkD9RlkhLNRwYwkcz8FZE+lphok1rJhOAunrwMzfOK61Tc+4ty9WYeRxGO4QTOwIVLqMId1KABBB7hGV7hzXqyXqx362M2WrDmO4fwR9bnD7NQllY=</latexit><latexit sha1_base64="mXTgWYhfQEyQqaBem9iUFe7iQ3E=">AAACAnicbZDLSsNAFIZP6q3WW9SVuBksgquSiKAboejGZYXeoIlhMp20QyeTMDMplFDc+CpuXCji1qdw59s4vSy09YcDH/85h5nzhylnSjvOt1VYWV1b3yhulra2d3b37P2DpkoySWiDJDyR7RArypmgDc00p+1UUhyHnLbCwe2k3xpSqVgi6nqUUj/GPcEiRrA2VmAfDdE18lQWBwx5mKd9bMDUQz2wy07FmQotgzuHMsxVC+wvr5uQLKZCE46V6rhOqv0cS80Ip+OSlymaYjLAPdoxKHBMlZ9PTxijU+N0UZRIU0Kjqft7I8exUqM4NJMx1n212JuY//U6mY6u/JyJNNNUkNlDUcaRTtAkD9RlkhLNRwYwkcz8FZE+lphok1rJhOAunrwMzfOK61Tc+4ty9WYeRxGO4QTOwIVLqMId1KABBB7hGV7hzXqyXqx362M2WrDmO4fwR9bnD7NQllY=</latexit><latexit sha1_base64="mXTgWYhfQEyQqaBem9iUFe7iQ3E=">AAACAnicbZDLSsNAFIZP6q3WW9SVuBksgquSiKAboejGZYXeoIlhMp20QyeTMDMplFDc+CpuXCji1qdw59s4vSy09YcDH/85h5nzhylnSjvOt1VYWV1b3yhulra2d3b37P2DpkoySWiDJDyR7RArypmgDc00p+1UUhyHnLbCwe2k3xpSqVgi6nqUUj/GPcEiRrA2VmAfDdE18lQWBwx5mKd9bMDUQz2wy07FmQotgzuHMsxVC+wvr5uQLKZCE46V6rhOqv0cS80Ip+OSlymaYjLAPdoxKHBMlZ9PTxijU+N0UZRIU0Kjqft7I8exUqM4NJMx1n212JuY//U6mY6u/JyJNNNUkNlDUcaRTtAkD9RlkhLNRwYwkcz8FZE+lphok1rJhOAunrwMzfOK61Tc+4ty9WYeRxGO4QTOwIVLqMId1KABBB7hGV7hzXqyXqx362M2WrDmO4fwR9bnD7NQllY=</latexit>

Page 20: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Kernel PCAIf we send data to a higher dimensional space:

x 7! �(x)<latexit sha1_base64="Eeje23qlPuGa/IH7WkSjQo4uKns=">AAAB+nicbVBNS8NAEJ3Ur1q/Uj16WSxCvZREBD0WvXisYD+gCWWz3bRLN5uwu9GW2J/ixYMiXv0l3vw3btsctPXBwOO9GWbmBQlnSjvOt1VYW9/Y3Cpul3Z29/YP7PJhS8WpJLRJYh7LToAV5UzQpmaa004iKY4CTtvB6Gbmtx+oVCwW93qSUD/CA8FCRrA2Us8uj5EX4UTpGHmNIauOz3p2xak5c6BV4uakAjkaPfvL68ckjajQhGOluq6TaD/DUjPC6bTkpYommIzwgHYNFTiiys/mp0/RqVH6KIylKaHRXP09keFIqUkUmM4I66Fa9mbif1431eGVnzGRpJoKslgUphyZR2c5oD6TlGg+MQQTycytiAyxxESbtEomBHf55VXSOq+5Ts29u6jUr/M4inAMJ1AFFy6hDrfQgCYQeIRneIU368l6sd6tj0VrwcpnjuAPrM8fJnqTQw==</latexit><latexit sha1_base64="Eeje23qlPuGa/IH7WkSjQo4uKns=">AAAB+nicbVBNS8NAEJ3Ur1q/Uj16WSxCvZREBD0WvXisYD+gCWWz3bRLN5uwu9GW2J/ixYMiXv0l3vw3btsctPXBwOO9GWbmBQlnSjvOt1VYW9/Y3Cpul3Z29/YP7PJhS8WpJLRJYh7LToAV5UzQpmaa004iKY4CTtvB6Gbmtx+oVCwW93qSUD/CA8FCRrA2Us8uj5EX4UTpGHmNIauOz3p2xak5c6BV4uakAjkaPfvL68ckjajQhGOluq6TaD/DUjPC6bTkpYommIzwgHYNFTiiys/mp0/RqVH6KIylKaHRXP09keFIqUkUmM4I66Fa9mbif1431eGVnzGRpJoKslgUphyZR2c5oD6TlGg+MQQTycytiAyxxESbtEomBHf55VXSOq+5Ts29u6jUr/M4inAMJ1AFFy6hDrfQgCYQeIRneIU368l6sd6tj0VrwcpnjuAPrM8fJnqTQw==</latexit><latexit sha1_base64="Eeje23qlPuGa/IH7WkSjQo4uKns=">AAAB+nicbVBNS8NAEJ3Ur1q/Uj16WSxCvZREBD0WvXisYD+gCWWz3bRLN5uwu9GW2J/ixYMiXv0l3vw3btsctPXBwOO9GWbmBQlnSjvOt1VYW9/Y3Cpul3Z29/YP7PJhS8WpJLRJYh7LToAV5UzQpmaa004iKY4CTtvB6Gbmtx+oVCwW93qSUD/CA8FCRrA2Us8uj5EX4UTpGHmNIauOz3p2xak5c6BV4uakAjkaPfvL68ckjajQhGOluq6TaD/DUjPC6bTkpYommIzwgHYNFTiiys/mp0/RqVH6KIylKaHRXP09keFIqUkUmM4I66Fa9mbif1431eGVnzGRpJoKslgUphyZR2c5oD6TlGg+MQQTycytiAyxxESbtEomBHf55VXSOq+5Ts29u6jUr/M4inAMJ1AFFy6hDrfQgCYQeIRneIU368l6sd6tj0VrwcpnjuAPrM8fJnqTQw==</latexit><latexit sha1_base64="Eeje23qlPuGa/IH7WkSjQo4uKns=">AAAB+nicbVBNS8NAEJ3Ur1q/Uj16WSxCvZREBD0WvXisYD+gCWWz3bRLN5uwu9GW2J/ixYMiXv0l3vw3btsctPXBwOO9GWbmBQlnSjvOt1VYW9/Y3Cpul3Z29/YP7PJhS8WpJLRJYh7LToAV5UzQpmaa004iKY4CTtvB6Gbmtx+oVCwW93qSUD/CA8FCRrA2Us8uj5EX4UTpGHmNIauOz3p2xak5c6BV4uakAjkaPfvL68ckjajQhGOluq6TaD/DUjPC6bTkpYommIzwgHYNFTiiys/mp0/RqVH6KIylKaHRXP09keFIqUkUmM4I66Fa9mbif1431eGVnzGRpJoKslgUphyZR2c5oD6TlGg+MQQTycytiAyxxESbtEomBHf55VXSOq+5Ts29u6jUr/M4inAMJ1AFFy6hDrfQgCYQeIRneIU368l6sd6tj0VrwcpnjuAPrM8fJnqTQw==</latexit>

and have it centered:X

i

�(ai) = 0<latexit sha1_base64="p8XGEfbkGbCRQIhg0q23qcH5ghM=">AAAB/nicbVBNS8NAEJ3Ur1q/ouLJy2IR6qUkIuhFKHrxWMF+QBPCZrtpl+4mYXcjlFDwr3jxoIhXf4c3/43bNgdtfTDweG+GmXlhypnSjvNtlVZW19Y3ypuVre2d3T17/6CtkkwS2iIJT2Q3xIpyFtOWZprTbiopFiGnnXB0O/U7j1QqlsQPepxSX+BBzCJGsDZSYB95KhMBQ15zyGo4YGfoGjkosKtO3ZkBLRO3IFUo0AzsL6+fkEzQWBOOleq5Tqr9HEvNCKeTipcpmmIywgPaMzTGgio/n50/QadG6aMokaZijWbq74kcC6XGIjSdAuuhWvSm4n9eL9PRlZ+zOM00jcl8UZRxpBM0zQL1maRE87EhmEhmbkVkiCUm2iRWMSG4iy8vk/Z53XXq7v1FtXFTxFGGYziBGrhwCQ24gya0gEAOz/AKb9aT9WK9Wx/z1pJVzBzCH1ifP3vDk9o=</latexit><latexit sha1_base64="p8XGEfbkGbCRQIhg0q23qcH5ghM=">AAAB/nicbVBNS8NAEJ3Ur1q/ouLJy2IR6qUkIuhFKHrxWMF+QBPCZrtpl+4mYXcjlFDwr3jxoIhXf4c3/43bNgdtfTDweG+GmXlhypnSjvNtlVZW19Y3ypuVre2d3T17/6CtkkwS2iIJT2Q3xIpyFtOWZprTbiopFiGnnXB0O/U7j1QqlsQPepxSX+BBzCJGsDZSYB95KhMBQ15zyGo4YGfoGjkosKtO3ZkBLRO3IFUo0AzsL6+fkEzQWBOOleq5Tqr9HEvNCKeTipcpmmIywgPaMzTGgio/n50/QadG6aMokaZijWbq74kcC6XGIjSdAuuhWvSm4n9eL9PRlZ+zOM00jcl8UZRxpBM0zQL1maRE87EhmEhmbkVkiCUm2iRWMSG4iy8vk/Z53XXq7v1FtXFTxFGGYziBGrhwCQ24gya0gEAOz/AKb9aT9WK9Wx/z1pJVzBzCH1ifP3vDk9o=</latexit><latexit sha1_base64="p8XGEfbkGbCRQIhg0q23qcH5ghM=">AAAB/nicbVBNS8NAEJ3Ur1q/ouLJy2IR6qUkIuhFKHrxWMF+QBPCZrtpl+4mYXcjlFDwr3jxoIhXf4c3/43bNgdtfTDweG+GmXlhypnSjvNtlVZW19Y3ypuVre2d3T17/6CtkkwS2iIJT2Q3xIpyFtOWZprTbiopFiGnnXB0O/U7j1QqlsQPepxSX+BBzCJGsDZSYB95KhMBQ15zyGo4YGfoGjkosKtO3ZkBLRO3IFUo0AzsL6+fkEzQWBOOleq5Tqr9HEvNCKeTipcpmmIywgPaMzTGgio/n50/QadG6aMokaZijWbq74kcC6XGIjSdAuuhWvSm4n9eL9PRlZ+zOM00jcl8UZRxpBM0zQL1maRE87EhmEhmbkVkiCUm2iRWMSG4iy8vk/Z53XXq7v1FtXFTxFGGYziBGrhwCQ24gya0gEAOz/AKb9aT9WK9Wx/z1pJVzBzCH1ifP3vDk9o=</latexit><latexit sha1_base64="p8XGEfbkGbCRQIhg0q23qcH5ghM=">AAAB/nicbVBNS8NAEJ3Ur1q/ouLJy2IR6qUkIuhFKHrxWMF+QBPCZrtpl+4mYXcjlFDwr3jxoIhXf4c3/43bNgdtfTDweG+GmXlhypnSjvNtlVZW19Y3ypuVre2d3T17/6CtkkwS2iIJT2Q3xIpyFtOWZprTbiopFiGnnXB0O/U7j1QqlsQPepxSX+BBzCJGsDZSYB95KhMBQ15zyGo4YGfoGjkosKtO3ZkBLRO3IFUo0AzsL6+fkEzQWBOOleq5Tqr9HEvNCKeTipcpmmIywgPaMzTGgio/n50/QadG6aMokaZijWbq74kcC6XGIjSdAuuhWvSm4n9eL9PRlZ+zOM00jcl8UZRxpBM0zQL1maRE87EhmEhmbkVkiCUm2iRWMSG4iy8vk/Z53XXq7v1FtXFTxFGGYziBGrhwCQ24gya0gEAOz/AKb9aT9WK9Wx/z1pJVzBzCH1ifP3vDk9o=</latexit>

C =1

n

X

i

�(ai)T�(ai)

<latexit sha1_base64="YVe3OeEN0bsyiREaSKervEhlo2E=">AAACEnicbZDLSgMxFIYz9VbrbdSlm2AR2k2ZEUE3QrEblxV6ETrjkEkzbWiSGZKMUIY+gxtfxY0LRdy6cufbmLaDaOsPgY//nMPJ+cOEUaUd58sqrKyurW8UN0tb2zu7e/b+QUfFqcSkjWMWy9sQKcKoIG1NNSO3iSSIh4x0w1FjWu/eE6loLFp6nBCfo4GgEcVIGyuwqw14Cb1IIgxdKKCnUh5Q6DWHtIICWr1r/WBgl52aMxNcBjeHMsjVDOxPrx/jlBOhMUNK9Vwn0X6GpKaYkUnJSxVJEB6hAekZFIgT5WezkybwxDh9GMXSPKHhzP09kSGu1JiHppMjPVSLtan5X62X6ujCz6hIUk0Eni+KUgZ1DKf5wD6VBGs2NoCwpOavEA+RyUebFEsmBHfx5GXonNZcp+benJXrV3kcRXAEjkEFuOAc1ME1aII2wOABPIEX8Go9Ws/Wm/U+by1Y+cwh+CPr4xsLnJs1</latexit><latexit sha1_base64="YVe3OeEN0bsyiREaSKervEhlo2E=">AAACEnicbZDLSgMxFIYz9VbrbdSlm2AR2k2ZEUE3QrEblxV6ETrjkEkzbWiSGZKMUIY+gxtfxY0LRdy6cufbmLaDaOsPgY//nMPJ+cOEUaUd58sqrKyurW8UN0tb2zu7e/b+QUfFqcSkjWMWy9sQKcKoIG1NNSO3iSSIh4x0w1FjWu/eE6loLFp6nBCfo4GgEcVIGyuwqw14Cb1IIgxdKKCnUh5Q6DWHtIICWr1r/WBgl52aMxNcBjeHMsjVDOxPrx/jlBOhMUNK9Vwn0X6GpKaYkUnJSxVJEB6hAekZFIgT5WezkybwxDh9GMXSPKHhzP09kSGu1JiHppMjPVSLtan5X62X6ujCz6hIUk0Eni+KUgZ1DKf5wD6VBGs2NoCwpOavEA+RyUebFEsmBHfx5GXonNZcp+benJXrV3kcRXAEjkEFuOAc1ME1aII2wOABPIEX8Go9Ws/Wm/U+by1Y+cwh+CPr4xsLnJs1</latexit><latexit sha1_base64="YVe3OeEN0bsyiREaSKervEhlo2E=">AAACEnicbZDLSgMxFIYz9VbrbdSlm2AR2k2ZEUE3QrEblxV6ETrjkEkzbWiSGZKMUIY+gxtfxY0LRdy6cufbmLaDaOsPgY//nMPJ+cOEUaUd58sqrKyurW8UN0tb2zu7e/b+QUfFqcSkjWMWy9sQKcKoIG1NNSO3iSSIh4x0w1FjWu/eE6loLFp6nBCfo4GgEcVIGyuwqw14Cb1IIgxdKKCnUh5Q6DWHtIICWr1r/WBgl52aMxNcBjeHMsjVDOxPrx/jlBOhMUNK9Vwn0X6GpKaYkUnJSxVJEB6hAekZFIgT5WezkybwxDh9GMXSPKHhzP09kSGu1JiHppMjPVSLtan5X62X6ujCz6hIUk0Eni+KUgZ1DKf5wD6VBGs2NoCwpOavEA+RyUebFEsmBHfx5GXonNZcp+benJXrV3kcRXAEjkEFuOAc1ME1aII2wOABPIEX8Go9Ws/Wm/U+by1Y+cwh+CPr4xsLnJs1</latexit><latexit sha1_base64="YVe3OeEN0bsyiREaSKervEhlo2E=">AAACEnicbZDLSgMxFIYz9VbrbdSlm2AR2k2ZEUE3QrEblxV6ETrjkEkzbWiSGZKMUIY+gxtfxY0LRdy6cufbmLaDaOsPgY//nMPJ+cOEUaUd58sqrKyurW8UN0tb2zu7e/b+QUfFqcSkjWMWy9sQKcKoIG1NNSO3iSSIh4x0w1FjWu/eE6loLFp6nBCfo4GgEcVIGyuwqw14Cb1IIgxdKKCnUh5Q6DWHtIICWr1r/WBgl52aMxNcBjeHMsjVDOxPrx/jlBOhMUNK9Vwn0X6GpKaYkUnJSxVJEB6hAekZFIgT5WezkybwxDh9GMXSPKHhzP09kSGu1JiHppMjPVSLtan5X62X6ujCz6hIUk0Eni+KUgZ1DKf5wD6VBGs2NoCwpOavEA+RyUebFEsmBHfx5GXonNZcp+benJXrV3kcRXAEjkEFuOAc1ME1aII2wOABPIEX8Go9Ws/Wm/U+by1Y+cwh+CPr4xsLnJs1</latexit>

Covariance is:

Page 21: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Kernel PCAC =

1

n

X

i

�(ai)T�(ai)

<latexit sha1_base64="YVe3OeEN0bsyiREaSKervEhlo2E=">AAACEnicbZDLSgMxFIYz9VbrbdSlm2AR2k2ZEUE3QrEblxV6ETrjkEkzbWiSGZKMUIY+gxtfxY0LRdy6cufbmLaDaOsPgY//nMPJ+cOEUaUd58sqrKyurW8UN0tb2zu7e/b+QUfFqcSkjWMWy9sQKcKoIG1NNSO3iSSIh4x0w1FjWu/eE6loLFp6nBCfo4GgEcVIGyuwqw14Cb1IIgxdKKCnUh5Q6DWHtIICWr1r/WBgl52aMxNcBjeHMsjVDOxPrx/jlBOhMUNK9Vwn0X6GpKaYkUnJSxVJEB6hAekZFIgT5WezkybwxDh9GMXSPKHhzP09kSGu1JiHppMjPVSLtan5X62X6ujCz6hIUk0Eni+KUgZ1DKf5wD6VBGs2NoCwpOavEA+RyUebFEsmBHfx5GXonNZcp+benJXrV3kcRXAEjkEFuOAc1ME1aII2wOABPIEX8Go9Ws/Wm/U+by1Y+cwh+CPr4xsLnJs1</latexit><latexit sha1_base64="YVe3OeEN0bsyiREaSKervEhlo2E=">AAACEnicbZDLSgMxFIYz9VbrbdSlm2AR2k2ZEUE3QrEblxV6ETrjkEkzbWiSGZKMUIY+gxtfxY0LRdy6cufbmLaDaOsPgY//nMPJ+cOEUaUd58sqrKyurW8UN0tb2zu7e/b+QUfFqcSkjWMWy9sQKcKoIG1NNSO3iSSIh4x0w1FjWu/eE6loLFp6nBCfo4GgEcVIGyuwqw14Cb1IIgxdKKCnUh5Q6DWHtIICWr1r/WBgl52aMxNcBjeHMsjVDOxPrx/jlBOhMUNK9Vwn0X6GpKaYkUnJSxVJEB6hAekZFIgT5WezkybwxDh9GMXSPKHhzP09kSGu1JiHppMjPVSLtan5X62X6ujCz6hIUk0Eni+KUgZ1DKf5wD6VBGs2NoCwpOavEA+RyUebFEsmBHfx5GXonNZcp+benJXrV3kcRXAEjkEFuOAc1ME1aII2wOABPIEX8Go9Ws/Wm/U+by1Y+cwh+CPr4xsLnJs1</latexit><latexit sha1_base64="YVe3OeEN0bsyiREaSKervEhlo2E=">AAACEnicbZDLSgMxFIYz9VbrbdSlm2AR2k2ZEUE3QrEblxV6ETrjkEkzbWiSGZKMUIY+gxtfxY0LRdy6cufbmLaDaOsPgY//nMPJ+cOEUaUd58sqrKyurW8UN0tb2zu7e/b+QUfFqcSkjWMWy9sQKcKoIG1NNSO3iSSIh4x0w1FjWu/eE6loLFp6nBCfo4GgEcVIGyuwqw14Cb1IIgxdKKCnUh5Q6DWHtIICWr1r/WBgl52aMxNcBjeHMsjVDOxPrx/jlBOhMUNK9Vwn0X6GpKaYkUnJSxVJEB6hAekZFIgT5WezkybwxDh9GMXSPKHhzP09kSGu1JiHppMjPVSLtan5X62X6ujCz6hIUk0Eni+KUgZ1DKf5wD6VBGs2NoCwpOavEA+RyUebFEsmBHfx5GXonNZcp+benJXrV3kcRXAEjkEFuOAc1ME1aII2wOABPIEX8Go9Ws/Wm/U+by1Y+cwh+CPr4xsLnJs1</latexit><latexit sha1_base64="YVe3OeEN0bsyiREaSKervEhlo2E=">AAACEnicbZDLSgMxFIYz9VbrbdSlm2AR2k2ZEUE3QrEblxV6ETrjkEkzbWiSGZKMUIY+gxtfxY0LRdy6cufbmLaDaOsPgY//nMPJ+cOEUaUd58sqrKyurW8UN0tb2zu7e/b+QUfFqcSkjWMWy9sQKcKoIG1NNSO3iSSIh4x0w1FjWu/eE6loLFp6nBCfo4GgEcVIGyuwqw14Cb1IIgxdKKCnUh5Q6DWHtIICWr1r/WBgl52aMxNcBjeHMsjVDOxPrx/jlBOhMUNK9Vwn0X6GpKaYkUnJSxVJEB6hAekZFIgT5WezkybwxDh9GMXSPKHhzP09kSGu1JiHppMjPVSLtan5X62X6ujCz6hIUk0Eni+KUgZ1DKf5wD6VBGs2NoCwpOavEA+RyUebFEsmBHfx5GXonNZcp+benJXrV3kcRXAEjkEFuOAc1ME1aII2wOABPIEX8Go9Ws/Wm/U+by1Y+cwh+CPr4xsLnJs1</latexit>

diagonalizable by Cv = �v<latexit sha1_base64="6pWje61hjjUi8/u5tMDSz6y2nrc=">AAAB9XicbVDLSgMxFL2pr1pfVZdugkVwVWZE0I1Q7MZlBfuAdiyZTKYNzWSGJFMpQ//DjQtF3Pov7vwb03YW2nogcDjnXO7N8RPBtXGcb1RYW9/Y3Cpul3Z29/YPyodHLR2nirImjUWsOj7RTHDJmoYbwTqJYiTyBWv7o/rMb4+Z0jyWD2aSMC8iA8lDTomx0mN9jG9wT9h8QPC4X644VWcOvErcnFQgR6Nf/uoFMU0jJg0VROuu6yTGy4gynAo2LfVSzRJCR2TAupZKEjHtZfOrp/jMKgEOY2WfNHiu/p7ISKT1JPJtMiJmqJe9mfif101NeO1lXCapYZIuFoWpwCbGswpwwBWjRkwsIVRxeyumQ6IINbaoki3BXf7yKmldVF2n6t5fVmq3eR1FOIFTOAcXrqAGd9CAJlBQ8Ayv8Iae0At6Rx+LaAHlM8fwB+jzBwxWkY8=</latexit><latexit sha1_base64="6pWje61hjjUi8/u5tMDSz6y2nrc=">AAAB9XicbVDLSgMxFL2pr1pfVZdugkVwVWZE0I1Q7MZlBfuAdiyZTKYNzWSGJFMpQ//DjQtF3Pov7vwb03YW2nogcDjnXO7N8RPBtXGcb1RYW9/Y3Cpul3Z29/YPyodHLR2nirImjUWsOj7RTHDJmoYbwTqJYiTyBWv7o/rMb4+Z0jyWD2aSMC8iA8lDTomx0mN9jG9wT9h8QPC4X644VWcOvErcnFQgR6Nf/uoFMU0jJg0VROuu6yTGy4gynAo2LfVSzRJCR2TAupZKEjHtZfOrp/jMKgEOY2WfNHiu/p7ISKT1JPJtMiJmqJe9mfif101NeO1lXCapYZIuFoWpwCbGswpwwBWjRkwsIVRxeyumQ6IINbaoki3BXf7yKmldVF2n6t5fVmq3eR1FOIFTOAcXrqAGd9CAJlBQ8Ayv8Iae0At6Rx+LaAHlM8fwB+jzBwxWkY8=</latexit><latexit sha1_base64="6pWje61hjjUi8/u5tMDSz6y2nrc=">AAAB9XicbVDLSgMxFL2pr1pfVZdugkVwVWZE0I1Q7MZlBfuAdiyZTKYNzWSGJFMpQ//DjQtF3Pov7vwb03YW2nogcDjnXO7N8RPBtXGcb1RYW9/Y3Cpul3Z29/YPyodHLR2nirImjUWsOj7RTHDJmoYbwTqJYiTyBWv7o/rMb4+Z0jyWD2aSMC8iA8lDTomx0mN9jG9wT9h8QPC4X644VWcOvErcnFQgR6Nf/uoFMU0jJg0VROuu6yTGy4gynAo2LfVSzRJCR2TAupZKEjHtZfOrp/jMKgEOY2WfNHiu/p7ISKT1JPJtMiJmqJe9mfif101NeO1lXCapYZIuFoWpwCbGswpwwBWjRkwsIVRxeyumQ6IINbaoki3BXf7yKmldVF2n6t5fVmq3eR1FOIFTOAcXrqAGd9CAJlBQ8Ayv8Iae0At6Rx+LaAHlM8fwB+jzBwxWkY8=</latexit>

Finding the principal directions amounts to finding coefficients of

v =X

i

↵i�(ai)T

<latexit sha1_base64="RAxIQSRgtub82mnbjTA1YL0GozE=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqJuSiKAboejGZYXeoIlhMp00Q2eSMDMplNClG1/FjQtF3PoI7nwbJ20W2vrDwMd/zuHM+f2EUaks69sorayurW+UNytb2zu7e+b+QUfGqcCkjWMWi56PJGE0Im1FFSO9RBDEfUa6/ug2r3fHREgaRy01SYjL0TCiAcVIacszj8fwGjoy5R6FDmJJiHJohrSm4eyh5ZlVq27NBJfBLqAKCjU988sZxDjlJFKYISn7tpUoN0NCUczItOKkkiQIj9CQ9DVGiBPpZrNDpvBUOwMYxEK/SMGZ+3siQ1zKCfd1J0cqlIu13Pyv1k9VcOVmNEpSRSI8XxSkDKoY5qnAARUEKzbRgLCg+q8Qh0ggrHR2FR2CvXjyMnTO67ZVt+8vqo2bIo4yOAInoAZscAka4A40QRtg8AiewSt4M56MF+Pd+Ji3loxi5hD8kfH5A4cKmGA=</latexit><latexit sha1_base64="RAxIQSRgtub82mnbjTA1YL0GozE=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqJuSiKAboejGZYXeoIlhMp00Q2eSMDMplNClG1/FjQtF3PoI7nwbJ20W2vrDwMd/zuHM+f2EUaks69sorayurW+UNytb2zu7e+b+QUfGqcCkjWMWi56PJGE0Im1FFSO9RBDEfUa6/ug2r3fHREgaRy01SYjL0TCiAcVIacszj8fwGjoy5R6FDmJJiHJohrSm4eyh5ZlVq27NBJfBLqAKCjU988sZxDjlJFKYISn7tpUoN0NCUczItOKkkiQIj9CQ9DVGiBPpZrNDpvBUOwMYxEK/SMGZ+3siQ1zKCfd1J0cqlIu13Pyv1k9VcOVmNEpSRSI8XxSkDKoY5qnAARUEKzbRgLCg+q8Qh0ggrHR2FR2CvXjyMnTO67ZVt+8vqo2bIo4yOAInoAZscAka4A40QRtg8AiewSt4M56MF+Pd+Ji3loxi5hD8kfH5A4cKmGA=</latexit><latexit sha1_base64="RAxIQSRgtub82mnbjTA1YL0GozE=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqJuSiKAboejGZYXeoIlhMp00Q2eSMDMplNClG1/FjQtF3PoI7nwbJ20W2vrDwMd/zuHM+f2EUaks69sorayurW+UNytb2zu7e+b+QUfGqcCkjWMWi56PJGE0Im1FFSO9RBDEfUa6/ug2r3fHREgaRy01SYjL0TCiAcVIacszj8fwGjoy5R6FDmJJiHJohrSm4eyh5ZlVq27NBJfBLqAKCjU988sZxDjlJFKYISn7tpUoN0NCUczItOKkkiQIj9CQ9DVGiBPpZrNDpvBUOwMYxEK/SMGZ+3siQ1zKCfd1J0cqlIu13Pyv1k9VcOVmNEpSRSI8XxSkDKoY5qnAARUEKzbRgLCg+q8Qh0ggrHR2FR2CvXjyMnTO67ZVt+8vqo2bIo4yOAInoAZscAka4A40QRtg8AiewSt4M56MF+Pd+Ji3loxi5hD8kfH5A4cKmGA=</latexit><latexit sha1_base64="RAxIQSRgtub82mnbjTA1YL0GozE=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqJuSiKAboejGZYXeoIlhMp00Q2eSMDMplNClG1/FjQtF3PoI7nwbJ20W2vrDwMd/zuHM+f2EUaks69sorayurW+UNytb2zu7e+b+QUfGqcCkjWMWi56PJGE0Im1FFSO9RBDEfUa6/ug2r3fHREgaRy01SYjL0TCiAcVIacszj8fwGjoy5R6FDmJJiHJohrSm4eyh5ZlVq27NBJfBLqAKCjU988sZxDjlJFKYISn7tpUoN0NCUczItOKkkiQIj9CQ9DVGiBPpZrNDpvBUOwMYxEK/SMGZ+3siQ1zKCfd1J0cqlIu13Pyv1k9VcOVmNEpSRSI8XxSkDKoY5qnAARUEKzbRgLCg+q8Qh0ggrHR2FR2CvXjyMnTO67ZVt+8vqo2bIo4yOAInoAZscAka4A40QRtg8AiewSt4M56MF+Pd+Ji3loxi5hD8kfH5A4cKmGA=</latexit>

We do that by solving an eigenvector problem

K↵ = �̃↵<latexit sha1_base64="+FUdJ45h9JKA4FPo3cnmOBgO5Gc=">AAACCnicbVDLSsNAFJ3UV62vqEs3o0VwVRIRdCMU3QhuKtgHNKHcTCbt0MkkzEyEErp246+4caGIW7/AnX/jtM1CWw8MHM65hzv3BClnSjvOt1VaWl5ZXSuvVzY2t7Z37N29lkoySWiTJDyRnQAU5UzQpmaa004qKcQBp+1geD3x2w9UKpaIez1KqR9DX7CIEdBG6tmHt9gDng4AX2JPMx7S3OMmHsK4MHp21ak5U+BF4hakigo0evaXFyYki6nQhINSXddJtZ+D1IxwOq54maIpkCH0addQATFVfj49ZYyPjRLiKJHmCY2n6u9EDrFSozgwkzHogZr3JuJ/XjfT0YWfM5FmmgoyWxRlHOsET3rBIZOUaD4yBIhk5q+YDEAC0aa9iinBnT95kbROa65Tc+/OqvWroo4yOkBH6AS56BzV0Q1qoCYi6BE9o1f0Zj1ZL9a79TEbLVlFZh/9gfX5Axh3meQ=</latexit><latexit sha1_base64="+FUdJ45h9JKA4FPo3cnmOBgO5Gc=">AAACCnicbVDLSsNAFJ3UV62vqEs3o0VwVRIRdCMU3QhuKtgHNKHcTCbt0MkkzEyEErp246+4caGIW7/AnX/jtM1CWw8MHM65hzv3BClnSjvOt1VaWl5ZXSuvVzY2t7Z37N29lkoySWiTJDyRnQAU5UzQpmaa004qKcQBp+1geD3x2w9UKpaIez1KqR9DX7CIEdBG6tmHt9gDng4AX2JPMx7S3OMmHsK4MHp21ak5U+BF4hakigo0evaXFyYki6nQhINSXddJtZ+D1IxwOq54maIpkCH0addQATFVfj49ZYyPjRLiKJHmCY2n6u9EDrFSozgwkzHogZr3JuJ/XjfT0YWfM5FmmgoyWxRlHOsET3rBIZOUaD4yBIhk5q+YDEAC0aa9iinBnT95kbROa65Tc+/OqvWroo4yOkBH6AS56BzV0Q1qoCYi6BE9o1f0Zj1ZL9a79TEbLVlFZh/9gfX5Axh3meQ=</latexit><latexit sha1_base64="+FUdJ45h9JKA4FPo3cnmOBgO5Gc=">AAACCnicbVDLSsNAFJ3UV62vqEs3o0VwVRIRdCMU3QhuKtgHNKHcTCbt0MkkzEyEErp246+4caGIW7/AnX/jtM1CWw8MHM65hzv3BClnSjvOt1VaWl5ZXSuvVzY2t7Z37N29lkoySWiTJDyRnQAU5UzQpmaa004qKcQBp+1geD3x2w9UKpaIez1KqR9DX7CIEdBG6tmHt9gDng4AX2JPMx7S3OMmHsK4MHp21ak5U+BF4hakigo0evaXFyYki6nQhINSXddJtZ+D1IxwOq54maIpkCH0addQATFVfj49ZYyPjRLiKJHmCY2n6u9EDrFSozgwkzHogZr3JuJ/XjfT0YWfM5FmmgoyWxRlHOsET3rBIZOUaD4yBIhk5q+YDEAC0aa9iinBnT95kbROa65Tc+/OqvWroo4yOkBH6AS56BzV0Q1qoCYi6BE9o1f0Zj1ZL9a79TEbLVlFZh/9gfX5Axh3meQ=</latexit><latexit sha1_base64="+FUdJ45h9JKA4FPo3cnmOBgO5Gc=">AAACCnicbVDLSsNAFJ3UV62vqEs3o0VwVRIRdCMU3QhuKtgHNKHcTCbt0MkkzEyEErp246+4caGIW7/AnX/jtM1CWw8MHM65hzv3BClnSjvOt1VaWl5ZXSuvVzY2t7Z37N29lkoySWiTJDyRnQAU5UzQpmaa004qKcQBp+1geD3x2w9UKpaIez1KqR9DX7CIEdBG6tmHt9gDng4AX2JPMx7S3OMmHsK4MHp21ak5U+BF4hakigo0evaXFyYki6nQhINSXddJtZ+D1IxwOq54maIpkCH0addQATFVfj49ZYyPjRLiKJHmCY2n6u9EDrFSozgwkzHogZr3JuJ/XjfT0YWfM5FmmgoyWxRlHOsET3rBIZOUaD4yBIhk5q+YDEAC0aa9iinBnT95kbROa65Tc+/OqvWroo4yOkBH6AS56BzV0Q1qoCYi6BE9o1f0Zj1ZL9a79TEbLVlFZh/9gfX5Axh3meQ=</latexit>

Page 22: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Kernel PCALast step: find the projection of a new point a onto the principal components

h�(a), vji =X

i

↵jiK(x, xi)<latexit sha1_base64="7rNdxGlqov3B8g2LD8u1bxUP8U4=">AAACIXicbVBNS8NAEN34bf2qevSyWIQWSklEsBdB9CJ4qWA/oAlhst22WzebsLspltC/4sW/4sWDIt7EP+O2zUFbHww83pthZl4Qc6a0bX9ZS8srq2vrG5u5re2d3b38/kFDRYkktE4iHslWAIpyJmhdM81pK5YUwoDTZvBwPfGbQyoVi8S9HsXUC6EnWJcR0Eby81WXg+hxit1anxWhVB76A+zKmXaBXZWEPsMu8LgPfjpg49viY/nRZyU/X7Ar9hR4kTgZKaAMNT//6XYikoRUaMJBqbZjx9pLQWpGOB3n3ETRGMgD9GjbUAEhVV46/XCMT4zSwd1ImhIaT9XfEymESo3CwHSGoPtq3puI/3ntRHerXspEnGgqyGxRN+FYR3gSF+4wSYnmI0OASGZuxaQPEog2oeZMCM78y4ukcVpx7Ipzd1a4vMri2EBH6BgVkYPO0SW6QTVURwQ9oRf0ht6tZ+vV+rA+Z61LVjZziP7A+v4BfpGiew==</latexit><latexit sha1_base64="7rNdxGlqov3B8g2LD8u1bxUP8U4=">AAACIXicbVBNS8NAEN34bf2qevSyWIQWSklEsBdB9CJ4qWA/oAlhst22WzebsLspltC/4sW/4sWDIt7EP+O2zUFbHww83pthZl4Qc6a0bX9ZS8srq2vrG5u5re2d3b38/kFDRYkktE4iHslWAIpyJmhdM81pK5YUwoDTZvBwPfGbQyoVi8S9HsXUC6EnWJcR0Eby81WXg+hxit1anxWhVB76A+zKmXaBXZWEPsMu8LgPfjpg49viY/nRZyU/X7Ar9hR4kTgZKaAMNT//6XYikoRUaMJBqbZjx9pLQWpGOB3n3ETRGMgD9GjbUAEhVV46/XCMT4zSwd1ImhIaT9XfEymESo3CwHSGoPtq3puI/3ntRHerXspEnGgqyGxRN+FYR3gSF+4wSYnmI0OASGZuxaQPEog2oeZMCM78y4ukcVpx7Ipzd1a4vMri2EBH6BgVkYPO0SW6QTVURwQ9oRf0ht6tZ+vV+rA+Z61LVjZziP7A+v4BfpGiew==</latexit><latexit sha1_base64="7rNdxGlqov3B8g2LD8u1bxUP8U4=">AAACIXicbVBNS8NAEN34bf2qevSyWIQWSklEsBdB9CJ4qWA/oAlhst22WzebsLspltC/4sW/4sWDIt7EP+O2zUFbHww83pthZl4Qc6a0bX9ZS8srq2vrG5u5re2d3b38/kFDRYkktE4iHslWAIpyJmhdM81pK5YUwoDTZvBwPfGbQyoVi8S9HsXUC6EnWJcR0Eby81WXg+hxit1anxWhVB76A+zKmXaBXZWEPsMu8LgPfjpg49viY/nRZyU/X7Ar9hR4kTgZKaAMNT//6XYikoRUaMJBqbZjx9pLQWpGOB3n3ETRGMgD9GjbUAEhVV46/XCMT4zSwd1ImhIaT9XfEymESo3CwHSGoPtq3puI/3ntRHerXspEnGgqyGxRN+FYR3gSF+4wSYnmI0OASGZuxaQPEog2oeZMCM78y4ukcVpx7Ipzd1a4vMri2EBH6BgVkYPO0SW6QTVURwQ9oRf0ht6tZ+vV+rA+Z61LVjZziP7A+v4BfpGiew==</latexit><latexit sha1_base64="7rNdxGlqov3B8g2LD8u1bxUP8U4=">AAACIXicbVBNS8NAEN34bf2qevSyWIQWSklEsBdB9CJ4qWA/oAlhst22WzebsLspltC/4sW/4sWDIt7EP+O2zUFbHww83pthZl4Qc6a0bX9ZS8srq2vrG5u5re2d3b38/kFDRYkktE4iHslWAIpyJmhdM81pK5YUwoDTZvBwPfGbQyoVi8S9HsXUC6EnWJcR0Eby81WXg+hxit1anxWhVB76A+zKmXaBXZWEPsMu8LgPfjpg49viY/nRZyU/X7Ar9hR4kTgZKaAMNT//6XYikoRUaMJBqbZjx9pLQWpGOB3n3ETRGMgD9GjbUAEhVV46/XCMT4zSwd1ImhIaT9XfEymESo3CwHSGoPtq3puI/3ntRHerXspEnGgqyGxRN+FYR3gSF+4wSYnmI0OASGZuxaQPEog2oeZMCM78y4ukcVpx7Ipzd1a4vMri2EBH6BgVkYPO0SW6QTVURwQ9oRf0ht6tZ+vV+rA+Z61LVjZziP7A+v4BfpGiew==</latexit>

Page 23: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Normalizing the feature spaceCentering features �̃(a) = �(a)� 1

n

X

i

�(ai)<latexit sha1_base64="EyugDzsEPmro1jzrS2Mv2Gtl2e8=">AAACHnicbZDLSsNAFIYnXmu9RV26GSxCu7AkouhGKLpxWcFeoAlhMp20Q2cmYWYilNAnceOruHGhiOBK38ZJm4W2Hhj4+P9zOHP+MGFUacf5tpaWV1bX1ksb5c2t7Z1de2+/reJUYtLCMYtlN0SKMCpIS1PNSDeRBPGQkU44usn9zgORisbiXo8T4nM0EDSiGGkjBfa5pynrk8xrDumkimrwCuaY0wn0IokwdAX0VMoDWjgBrQV2xak704KL4BZQAUU1A/vT68c45URozJBSPddJtJ8hqSlmZFL2UkUShEdoQHoGBeJE+dn0vAk8NkofRrE0T2g4VX9PZIgrNeah6eRID9W8l4v/eb1UR5d+RkWSaiLwbFGUMqhjmGcF+1QSrNnYAMKSmr9CPEQmE20SLZsQ3PmTF6F9Wnedunt3VmlcF3GUwCE4AlXgggvQALegCVoAg0fwDF7Bm/VkvVjv1sesdckqZg7An7K+fgDhSp/S</latexit><latexit sha1_base64="EyugDzsEPmro1jzrS2Mv2Gtl2e8=">AAACHnicbZDLSsNAFIYnXmu9RV26GSxCu7AkouhGKLpxWcFeoAlhMp20Q2cmYWYilNAnceOruHGhiOBK38ZJm4W2Hhj4+P9zOHP+MGFUacf5tpaWV1bX1ksb5c2t7Z1de2+/reJUYtLCMYtlN0SKMCpIS1PNSDeRBPGQkU44usn9zgORisbiXo8T4nM0EDSiGGkjBfa5pynrk8xrDumkimrwCuaY0wn0IokwdAX0VMoDWjgBrQV2xak704KL4BZQAUU1A/vT68c45URozJBSPddJtJ8hqSlmZFL2UkUShEdoQHoGBeJE+dn0vAk8NkofRrE0T2g4VX9PZIgrNeah6eRID9W8l4v/eb1UR5d+RkWSaiLwbFGUMqhjmGcF+1QSrNnYAMKSmr9CPEQmE20SLZsQ3PmTF6F9Wnedunt3VmlcF3GUwCE4AlXgggvQALegCVoAg0fwDF7Bm/VkvVjv1sesdckqZg7An7K+fgDhSp/S</latexit><latexit sha1_base64="EyugDzsEPmro1jzrS2Mv2Gtl2e8=">AAACHnicbZDLSsNAFIYnXmu9RV26GSxCu7AkouhGKLpxWcFeoAlhMp20Q2cmYWYilNAnceOruHGhiOBK38ZJm4W2Hhj4+P9zOHP+MGFUacf5tpaWV1bX1ksb5c2t7Z1de2+/reJUYtLCMYtlN0SKMCpIS1PNSDeRBPGQkU44usn9zgORisbiXo8T4nM0EDSiGGkjBfa5pynrk8xrDumkimrwCuaY0wn0IokwdAX0VMoDWjgBrQV2xak704KL4BZQAUU1A/vT68c45URozJBSPddJtJ8hqSlmZFL2UkUShEdoQHoGBeJE+dn0vAk8NkofRrE0T2g4VX9PZIgrNeah6eRID9W8l4v/eb1UR5d+RkWSaiLwbFGUMqhjmGcF+1QSrNnYAMKSmr9CPEQmE20SLZsQ3PmTF6F9Wnedunt3VmlcF3GUwCE4AlXgggvQALegCVoAg0fwDF7Bm/VkvVjv1sesdckqZg7An7K+fgDhSp/S</latexit><latexit sha1_base64="EyugDzsEPmro1jzrS2Mv2Gtl2e8=">AAACHnicbZDLSsNAFIYnXmu9RV26GSxCu7AkouhGKLpxWcFeoAlhMp20Q2cmYWYilNAnceOruHGhiOBK38ZJm4W2Hhj4+P9zOHP+MGFUacf5tpaWV1bX1ksb5c2t7Z1de2+/reJUYtLCMYtlN0SKMCpIS1PNSDeRBPGQkU44usn9zgORisbiXo8T4nM0EDSiGGkjBfa5pynrk8xrDumkimrwCuaY0wn0IokwdAX0VMoDWjgBrQV2xak704KL4BZQAUU1A/vT68c45URozJBSPddJtJ8hqSlmZFL2UkUShEdoQHoGBeJE+dn0vAk8NkofRrE0T2g4VX9PZIgrNeah6eRID9W8l4v/eb1UR5d+RkWSaiLwbFGUMqhjmGcF+1QSrNnYAMKSmr9CPEQmE20SLZsQ3PmTF6F9Wnedunt3VmlcF3GUwCE4AlXgggvQALegCVoAg0fwDF7Bm/VkvVjv1sesdckqZg7An7K+fgDhSp/S</latexit>

K̃ = JKJ<latexit sha1_base64="p2qv0B+pEAeu67IV9E6NgVkuKP0=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigl6EohdpLxXsB7ShbDabdulmE3Y3Qg39JV48KOLVn+LNf+O2zUFbHww83pthZp6fcKa043xbhbX1jc2t4nZpZ3dvv2wfHLZVnEpCWyTmsez6WFHOBG1ppjntJpLiyOe0449vZ37nkUrFYvGgJwn1IjwULGQEayMN7HJfMx7QrDFF16jeqA/silN15kCrxM1JBXI0B/ZXP4hJGlGhCcdK9Vwn0V6GpWaE02mpnyqaYDLGQ9ozVOCIKi+bHz5Fp0YJUBhLU0Kjufp7IsORUpPIN50R1iO17M3E/7xeqsMrL2MiSTUVZLEoTDnSMZqlgAImKdF8YggmkplbERlhiYk2WZVMCO7yy6ukfV51nap7f1Gp3eRxFOEYTuAMXLiEGtxBE1pAIIVneIU368l6sd6tj0VrwcpnjuAPrM8ffR2STg==</latexit><latexit sha1_base64="p2qv0B+pEAeu67IV9E6NgVkuKP0=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigl6EohdpLxXsB7ShbDabdulmE3Y3Qg39JV48KOLVn+LNf+O2zUFbHww83pthZp6fcKa043xbhbX1jc2t4nZpZ3dvv2wfHLZVnEpCWyTmsez6WFHOBG1ppjntJpLiyOe0449vZ37nkUrFYvGgJwn1IjwULGQEayMN7HJfMx7QrDFF16jeqA/silN15kCrxM1JBXI0B/ZXP4hJGlGhCcdK9Vwn0V6GpWaE02mpnyqaYDLGQ9ozVOCIKi+bHz5Fp0YJUBhLU0Kjufp7IsORUpPIN50R1iO17M3E/7xeqsMrL2MiSTUVZLEoTDnSMZqlgAImKdF8YggmkplbERlhiYk2WZVMCO7yy6ukfV51nap7f1Gp3eRxFOEYTuAMXLiEGtxBE1pAIIVneIU368l6sd6tj0VrwcpnjuAPrM8ffR2STg==</latexit><latexit sha1_base64="p2qv0B+pEAeu67IV9E6NgVkuKP0=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigl6EohdpLxXsB7ShbDabdulmE3Y3Qg39JV48KOLVn+LNf+O2zUFbHww83pthZp6fcKa043xbhbX1jc2t4nZpZ3dvv2wfHLZVnEpCWyTmsez6WFHOBG1ppjntJpLiyOe0449vZ37nkUrFYvGgJwn1IjwULGQEayMN7HJfMx7QrDFF16jeqA/silN15kCrxM1JBXI0B/ZXP4hJGlGhCcdK9Vwn0V6GpWaE02mpnyqaYDLGQ9ozVOCIKi+bHz5Fp0YJUBhLU0Kjufp7IsORUpPIN50R1iO17M3E/7xeqsMrL2MiSTUVZLEoTDnSMZqlgAImKdF8YggmkplbERlhiYk2WZVMCO7yy6ukfV51nap7f1Gp3eRxFOEYTuAMXLiEGtxBE1pAIIVneIU368l6sd6tj0VrwcpnjuAPrM8ffR2STg==</latexit><latexit sha1_base64="p2qv0B+pEAeu67IV9E6NgVkuKP0=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigl6EohdpLxXsB7ShbDabdulmE3Y3Qg39JV48KOLVn+LNf+O2zUFbHww83pthZp6fcKa043xbhbX1jc2t4nZpZ3dvv2wfHLZVnEpCWyTmsez6WFHOBG1ppjntJpLiyOe0449vZ37nkUrFYvGgJwn1IjwULGQEayMN7HJfMx7QrDFF16jeqA/silN15kCrxM1JBXI0B/ZXP4hJGlGhCcdK9Vwn0V6GpWaE02mpnyqaYDLGQ9ozVOCIKi+bHz5Fp0YJUBhLU0Kjufp7IsORUpPIN50R1iO17M3E/7xeqsMrL2MiSTUVZLEoTDnSMZqlgAImKdF8YggmkplbERlhiYk2WZVMCO7yy6ukfV51nap7f1Gp3eRxFOEYTuAMXLiEGtxBE1pAIIVneIU368l6sd6tj0VrwcpnjuAPrM8ffR2STg==</latexit>

J = I � 1

n1n1

Tn

<latexit sha1_base64="lHzK6UW3sjf0YaUCjq47G0aJ0PA=">AAACEnicbZDLSgMxFIYzXmu9jbp0EyyCLiwTEXQjFN2oqwq9QWccMmmmDc1khiQjlKHP4MZXceNCEbeu3Pk2pu0sausPgY//nEPO+YOEM6Ud58daWFxaXlktrBXXNza3tu2d3YaKU0loncQ8lq0AK8qZoHXNNKetRFIcBZw2g/71qN58pFKxWNT0IKFehLuChYxgbSzfPr6Dl/D2xA0lJhAJ6EZY94IwQ0NfTPNDzbdLTtkZC84DyqEEclV9+9vtxCSNqNCEY6XayEm0l2GpGeF0WHRTRRNM+rhL2wYFjqjysvFJQ3honA4MY2me0HDsTk9kOFJqEAWmc7Slmq2NzP9q7VSHF17GRJJqKsjkozDlUMdwlA/sMEmJ5gMDmEhmdoWkh0062qRYNCGg2ZPnoXFaRk4Z3Z+VKld5HAWwDw7AEUDgHFTADaiCOiDgCbyAN/BuPVuv1of1OWldsPKZPfBH1tcvKz2ckw==</latexit><latexit sha1_base64="lHzK6UW3sjf0YaUCjq47G0aJ0PA=">AAACEnicbZDLSgMxFIYzXmu9jbp0EyyCLiwTEXQjFN2oqwq9QWccMmmmDc1khiQjlKHP4MZXceNCEbeu3Pk2pu0sausPgY//nEPO+YOEM6Ud58daWFxaXlktrBXXNza3tu2d3YaKU0loncQ8lq0AK8qZoHXNNKetRFIcBZw2g/71qN58pFKxWNT0IKFehLuChYxgbSzfPr6Dl/D2xA0lJhAJ6EZY94IwQ0NfTPNDzbdLTtkZC84DyqEEclV9+9vtxCSNqNCEY6XayEm0l2GpGeF0WHRTRRNM+rhL2wYFjqjysvFJQ3honA4MY2me0HDsTk9kOFJqEAWmc7Slmq2NzP9q7VSHF17GRJJqKsjkozDlUMdwlA/sMEmJ5gMDmEhmdoWkh0062qRYNCGg2ZPnoXFaRk4Z3Z+VKld5HAWwDw7AEUDgHFTADaiCOiDgCbyAN/BuPVuv1of1OWldsPKZPfBH1tcvKz2ckw==</latexit><latexit sha1_base64="lHzK6UW3sjf0YaUCjq47G0aJ0PA=">AAACEnicbZDLSgMxFIYzXmu9jbp0EyyCLiwTEXQjFN2oqwq9QWccMmmmDc1khiQjlKHP4MZXceNCEbeu3Pk2pu0sausPgY//nEPO+YOEM6Ud58daWFxaXlktrBXXNza3tu2d3YaKU0loncQ8lq0AK8qZoHXNNKetRFIcBZw2g/71qN58pFKxWNT0IKFehLuChYxgbSzfPr6Dl/D2xA0lJhAJ6EZY94IwQ0NfTPNDzbdLTtkZC84DyqEEclV9+9vtxCSNqNCEY6XayEm0l2GpGeF0WHRTRRNM+rhL2wYFjqjysvFJQ3honA4MY2me0HDsTk9kOFJqEAWmc7Slmq2NzP9q7VSHF17GRJJqKsjkozDlUMdwlA/sMEmJ5gMDmEhmdoWkh0062qRYNCGg2ZPnoXFaRk4Z3Z+VKld5HAWwDw7AEUDgHFTADaiCOiDgCbyAN/BuPVuv1of1OWldsPKZPfBH1tcvKz2ckw==</latexit><latexit sha1_base64="lHzK6UW3sjf0YaUCjq47G0aJ0PA=">AAACEnicbZDLSgMxFIYzXmu9jbp0EyyCLiwTEXQjFN2oqwq9QWccMmmmDc1khiQjlKHP4MZXceNCEbeu3Pk2pu0sausPgY//nEPO+YOEM6Ud58daWFxaXlktrBXXNza3tu2d3YaKU0loncQ8lq0AK8qZoHXNNKetRFIcBZw2g/71qN58pFKxWNT0IKFehLuChYxgbSzfPr6Dl/D2xA0lJhAJ6EZY94IwQ0NfTPNDzbdLTtkZC84DyqEEclV9+9vtxCSNqNCEY6XayEm0l2GpGeF0WHRTRRNM+rhL2wYFjqjysvFJQ3honA4MY2me0HDsTk9kOFJqEAWmc7Slmq2NzP9q7VSHF17GRJJqKsjkozDlUMdwlA/sMEmJ5gMDmEhmdoWkh0062qRYNCGg2ZPnoXFaRk4Z3Z+VKld5HAWwDw7AEUDgHFTADaiCOiDgCbyAN/BuPVuv1of1OWldsPKZPfBH1tcvKz2ckw==</latexit>

Page 24: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

kPCA: algorithm

1. Choose your kernel

2. Construct centered kernel matrix

3. Solve the eigenvalue problem

4. For any data point, represent it

K̃ = JKJ<latexit sha1_base64="p2qv0B+pEAeu67IV9E6NgVkuKP0=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigl6EohdpLxXsB7ShbDabdulmE3Y3Qg39JV48KOLVn+LNf+O2zUFbHww83pthZp6fcKa043xbhbX1jc2t4nZpZ3dvv2wfHLZVnEpCWyTmsez6WFHOBG1ppjntJpLiyOe0449vZ37nkUrFYvGgJwn1IjwULGQEayMN7HJfMx7QrDFF16jeqA/silN15kCrxM1JBXI0B/ZXP4hJGlGhCcdK9Vwn0V6GpWaE02mpnyqaYDLGQ9ozVOCIKi+bHz5Fp0YJUBhLU0Kjufp7IsORUpPIN50R1iO17M3E/7xeqsMrL2MiSTUVZLEoTDnSMZqlgAImKdF8YggmkplbERlhiYk2WZVMCO7yy6ukfV51nap7f1Gp3eRxFOEYTuAMXLiEGtxBE1pAIIVneIU368l6sd6tj0VrwcpnjuAPrM8ffR2STg==</latexit><latexit sha1_base64="p2qv0B+pEAeu67IV9E6NgVkuKP0=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigl6EohdpLxXsB7ShbDabdulmE3Y3Qg39JV48KOLVn+LNf+O2zUFbHww83pthZp6fcKa043xbhbX1jc2t4nZpZ3dvv2wfHLZVnEpCWyTmsez6WFHOBG1ppjntJpLiyOe0449vZ37nkUrFYvGgJwn1IjwULGQEayMN7HJfMx7QrDFF16jeqA/silN15kCrxM1JBXI0B/ZXP4hJGlGhCcdK9Vwn0V6GpWaE02mpnyqaYDLGQ9ozVOCIKi+bHz5Fp0YJUBhLU0Kjufp7IsORUpPIN50R1iO17M3E/7xeqsMrL2MiSTUVZLEoTDnSMZqlgAImKdF8YggmkplbERlhiYk2WZVMCO7yy6ukfV51nap7f1Gp3eRxFOEYTuAMXLiEGtxBE1pAIIVneIU368l6sd6tj0VrwcpnjuAPrM8ffR2STg==</latexit><latexit sha1_base64="p2qv0B+pEAeu67IV9E6NgVkuKP0=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigl6EohdpLxXsB7ShbDabdulmE3Y3Qg39JV48KOLVn+LNf+O2zUFbHww83pthZp6fcKa043xbhbX1jc2t4nZpZ3dvv2wfHLZVnEpCWyTmsez6WFHOBG1ppjntJpLiyOe0449vZ37nkUrFYvGgJwn1IjwULGQEayMN7HJfMx7QrDFF16jeqA/silN15kCrxM1JBXI0B/ZXP4hJGlGhCcdK9Vwn0V6GpWaE02mpnyqaYDLGQ9ozVOCIKi+bHz5Fp0YJUBhLU0Kjufp7IsORUpPIN50R1iO17M3E/7xeqsMrL2MiSTUVZLEoTDnSMZqlgAImKdF8YggmkplbERlhiYk2WZVMCO7yy6ukfV51nap7f1Gp3eRxFOEYTuAMXLiEGtxBE1pAIIVneIU368l6sd6tj0VrwcpnjuAPrM8ffR2STg==</latexit>

K↵ = �̃↵<latexit sha1_base64="+FUdJ45h9JKA4FPo3cnmOBgO5Gc=">AAACCnicbVDLSsNAFJ3UV62vqEs3o0VwVRIRdCMU3QhuKtgHNKHcTCbt0MkkzEyEErp246+4caGIW7/AnX/jtM1CWw8MHM65hzv3BClnSjvOt1VaWl5ZXSuvVzY2t7Z37N29lkoySWiTJDyRnQAU5UzQpmaa004qKcQBp+1geD3x2w9UKpaIez1KqR9DX7CIEdBG6tmHt9gDng4AX2JPMx7S3OMmHsK4MHp21ak5U+BF4hakigo0evaXFyYki6nQhINSXddJtZ+D1IxwOq54maIpkCH0addQATFVfj49ZYyPjRLiKJHmCY2n6u9EDrFSozgwkzHogZr3JuJ/XjfT0YWfM5FmmgoyWxRlHOsET3rBIZOUaD4yBIhk5q+YDEAC0aa9iinBnT95kbROa65Tc+/OqvWroo4yOkBH6AS56BzV0Q1qoCYi6BE9o1f0Zj1ZL9a79TEbLVlFZh/9gfX5Axh3meQ=</latexit><latexit sha1_base64="+FUdJ45h9JKA4FPo3cnmOBgO5Gc=">AAACCnicbVDLSsNAFJ3UV62vqEs3o0VwVRIRdCMU3QhuKtgHNKHcTCbt0MkkzEyEErp246+4caGIW7/AnX/jtM1CWw8MHM65hzv3BClnSjvOt1VaWl5ZXSuvVzY2t7Z37N29lkoySWiTJDyRnQAU5UzQpmaa004qKcQBp+1geD3x2w9UKpaIez1KqR9DX7CIEdBG6tmHt9gDng4AX2JPMx7S3OMmHsK4MHp21ak5U+BF4hakigo0evaXFyYki6nQhINSXddJtZ+D1IxwOq54maIpkCH0addQATFVfj49ZYyPjRLiKJHmCY2n6u9EDrFSozgwkzHogZr3JuJ/XjfT0YWfM5FmmgoyWxRlHOsET3rBIZOUaD4yBIhk5q+YDEAC0aa9iinBnT95kbROa65Tc+/OqvWroo4yOkBH6AS56BzV0Q1qoCYi6BE9o1f0Zj1ZL9a79TEbLVlFZh/9gfX5Axh3meQ=</latexit><latexit sha1_base64="+FUdJ45h9JKA4FPo3cnmOBgO5Gc=">AAACCnicbVDLSsNAFJ3UV62vqEs3o0VwVRIRdCMU3QhuKtgHNKHcTCbt0MkkzEyEErp246+4caGIW7/AnX/jtM1CWw8MHM65hzv3BClnSjvOt1VaWl5ZXSuvVzY2t7Z37N29lkoySWiTJDyRnQAU5UzQpmaa004qKcQBp+1geD3x2w9UKpaIez1KqR9DX7CIEdBG6tmHt9gDng4AX2JPMx7S3OMmHsK4MHp21ak5U+BF4hakigo0evaXFyYki6nQhINSXddJtZ+D1IxwOq54maIpkCH0addQATFVfj49ZYyPjRLiKJHmCY2n6u9EDrFSozgwkzHogZr3JuJ/XjfT0YWfM5FmmgoyWxRlHOsET3rBIZOUaD4yBIhk5q+YDEAC0aa9iinBnT95kbROa65Tc+/OqvWroo4yOkBH6AS56BzV0Q1qoCYi6BE9o1f0Zj1ZL9a79TEbLVlFZh/9gfX5Axh3meQ=</latexit>

yj =X

i

↵jiK(x, xi)<latexit sha1_base64="wEHL7a0+2KStbREwn/H0Xr4tdK4=">AAACCXicbVDLSsNAFJ3UV62vqEs3g0WoICURQTdC0Y3gpoJ9QBPCZDppp51JwsxEGkK3bvwVNy4UcesfuPNvnLZZaOuBC4dz7uXee/yYUaks69soLC2vrK4V10sbm1vbO+buXlNGicCkgSMWibaPJGE0JA1FFSPtWBDEfUZa/vB64rceiJA0Cu9VGhOXo15IA4qR0pJnwtQbwEvoyIR7FDqIxX3kZQM6vq2MTkYePfbMslW1poCLxM5JGeSoe+aX041wwkmoMENSdmwrVm6GhKKYkXHJSSSJER6iHuloGiJOpJtNPxnDI610YRAJXaGCU/X3RIa4lCn3dSdHqi/nvYn4n9dJVHDhZjSME0VCPFsUJAyqCE5igV0qCFYs1QRhQfWtEPeRQFjp8Eo6BHv+5UXSPK3aVtW+OyvXrvI4iuAAHIIKsME5qIEbUAcNgMEjeAav4M14Ml6Md+Nj1low8pl98AfG5w8aJplP</latexit><latexit sha1_base64="wEHL7a0+2KStbREwn/H0Xr4tdK4=">AAACCXicbVDLSsNAFJ3UV62vqEs3g0WoICURQTdC0Y3gpoJ9QBPCZDppp51JwsxEGkK3bvwVNy4UcesfuPNvnLZZaOuBC4dz7uXee/yYUaks69soLC2vrK4V10sbm1vbO+buXlNGicCkgSMWibaPJGE0JA1FFSPtWBDEfUZa/vB64rceiJA0Cu9VGhOXo15IA4qR0pJnwtQbwEvoyIR7FDqIxX3kZQM6vq2MTkYePfbMslW1poCLxM5JGeSoe+aX041wwkmoMENSdmwrVm6GhKKYkXHJSSSJER6iHuloGiJOpJtNPxnDI610YRAJXaGCU/X3RIa4lCn3dSdHqi/nvYn4n9dJVHDhZjSME0VCPFsUJAyqCE5igV0qCFYs1QRhQfWtEPeRQFjp8Eo6BHv+5UXSPK3aVtW+OyvXrvI4iuAAHIIKsME5qIEbUAcNgMEjeAav4M14Ml6Md+Nj1low8pl98AfG5w8aJplP</latexit><latexit sha1_base64="wEHL7a0+2KStbREwn/H0Xr4tdK4=">AAACCXicbVDLSsNAFJ3UV62vqEs3g0WoICURQTdC0Y3gpoJ9QBPCZDppp51JwsxEGkK3bvwVNy4UcesfuPNvnLZZaOuBC4dz7uXee/yYUaks69soLC2vrK4V10sbm1vbO+buXlNGicCkgSMWibaPJGE0JA1FFSPtWBDEfUZa/vB64rceiJA0Cu9VGhOXo15IA4qR0pJnwtQbwEvoyIR7FDqIxX3kZQM6vq2MTkYePfbMslW1poCLxM5JGeSoe+aX041wwkmoMENSdmwrVm6GhKKYkXHJSSSJER6iHuloGiJOpJtNPxnDI610YRAJXaGCU/X3RIa4lCn3dSdHqi/nvYn4n9dJVHDhZjSME0VCPFsUJAyqCE5igV0qCFYs1QRhQfWtEPeRQFjp8Eo6BHv+5UXSPK3aVtW+OyvXrvI4iuAAHIIKsME5qIEbUAcNgMEjeAav4M14Ml6Md+Nj1low8pl98AfG5w8aJplP</latexit><latexit sha1_base64="wEHL7a0+2KStbREwn/H0Xr4tdK4=">AAACCXicbVDLSsNAFJ3UV62vqEs3g0WoICURQTdC0Y3gpoJ9QBPCZDppp51JwsxEGkK3bvwVNy4UcesfuPNvnLZZaOuBC4dz7uXee/yYUaks69soLC2vrK4V10sbm1vbO+buXlNGicCkgSMWibaPJGE0JA1FFSPtWBDEfUZa/vB64rceiJA0Cu9VGhOXo15IA4qR0pJnwtQbwEvoyIR7FDqIxX3kZQM6vq2MTkYePfbMslW1poCLxM5JGeSoe+aX041wwkmoMENSdmwrVm6GhKKYkXHJSSSJER6iHuloGiJOpJtNPxnDI610YRAJXaGCU/X3RIa4lCn3dSdHqi/nvYn4n9dJVHDhZjSME0VCPFsUJAyqCE5igV0qCFYs1QRhQfWtEPeRQFjp8Eo6BHv+5UXSPK3aVtW+OyvXrvI4iuAAHIIKsME5qIEbUAcNgMEjeAav4M14Ml6Md+Nj1low8pl98AfG5w8aJplP</latexit>

Page 25: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Kernel function

Real-valued function of two arguments

When symmetric and non-negative can be interpreted as a similarity measure

It is a way to introduce prior knowledge about the data

Page 26: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Gaussian kernel

k(x, x0) = exp(�1/2(x� x0)T⌃�1(x� x0))<latexit sha1_base64="lA57R0BjI2gUIrVodWA6wZuXCrk=">AAACKHicbVBNS8NAEN34bf2qevSyWIQUbE1E0ItY9OKxom2FJi2b7bRdupuE3Y20hP4cL/4VLyKKePWXuK1FtPpg4PHeDDPzgpgzpR3n3ZqZnZtfWFxazqysrq1vZDe3qipKJIUKjXgkbwOigLMQKpppDrexBCICDrWgdzHya3cgFYvCGz2IwRekE7I2o0QbqZk969n9/X7DiyUTkMen2IN+bBfcg0O7X/jWGzfeNesI0kgL7vCnkW9mc07RGQP/Je6E5NAE5Wb22WtFNBEQasqJUnXXibWfEqkZ5TDMeImCmNAe6UDd0JAIUH46fnSI94zSwu1Imgo1Hqs/J1IilBqIwHQKortq2huJ/3n1RLdP/JSFcaIhpF+L2gnHOsKj1HCLSaCaDwwhVDJzK6ZdIgnVJtuMCcGdfvkvqR4WXafoXh3lSueTOJbQDtpFNnLRMSqhS1RGFUTRPXpEL+jVerCerDfr/at1xprMbKNfsD4+ATvspEM=</latexit><latexit sha1_base64="lA57R0BjI2gUIrVodWA6wZuXCrk=">AAACKHicbVBNS8NAEN34bf2qevSyWIQUbE1E0ItY9OKxom2FJi2b7bRdupuE3Y20hP4cL/4VLyKKePWXuK1FtPpg4PHeDDPzgpgzpR3n3ZqZnZtfWFxazqysrq1vZDe3qipKJIUKjXgkbwOigLMQKpppDrexBCICDrWgdzHya3cgFYvCGz2IwRekE7I2o0QbqZk969n9/X7DiyUTkMen2IN+bBfcg0O7X/jWGzfeNesI0kgL7vCnkW9mc07RGQP/Je6E5NAE5Wb22WtFNBEQasqJUnXXibWfEqkZ5TDMeImCmNAe6UDd0JAIUH46fnSI94zSwu1Imgo1Hqs/J1IilBqIwHQKortq2huJ/3n1RLdP/JSFcaIhpF+L2gnHOsKj1HCLSaCaDwwhVDJzK6ZdIgnVJtuMCcGdfvkvqR4WXafoXh3lSueTOJbQDtpFNnLRMSqhS1RGFUTRPXpEL+jVerCerDfr/at1xprMbKNfsD4+ATvspEM=</latexit><latexit sha1_base64="lA57R0BjI2gUIrVodWA6wZuXCrk=">AAACKHicbVBNS8NAEN34bf2qevSyWIQUbE1E0ItY9OKxom2FJi2b7bRdupuE3Y20hP4cL/4VLyKKePWXuK1FtPpg4PHeDDPzgpgzpR3n3ZqZnZtfWFxazqysrq1vZDe3qipKJIUKjXgkbwOigLMQKpppDrexBCICDrWgdzHya3cgFYvCGz2IwRekE7I2o0QbqZk969n9/X7DiyUTkMen2IN+bBfcg0O7X/jWGzfeNesI0kgL7vCnkW9mc07RGQP/Je6E5NAE5Wb22WtFNBEQasqJUnXXibWfEqkZ5TDMeImCmNAe6UDd0JAIUH46fnSI94zSwu1Imgo1Hqs/J1IilBqIwHQKortq2huJ/3n1RLdP/JSFcaIhpF+L2gnHOsKj1HCLSaCaDwwhVDJzK6ZdIgnVJtuMCcGdfvkvqR4WXafoXh3lSueTOJbQDtpFNnLRMSqhS1RGFUTRPXpEL+jVerCerDfr/at1xprMbKNfsD4+ATvspEM=</latexit><latexit sha1_base64="lA57R0BjI2gUIrVodWA6wZuXCrk=">AAACKHicbVBNS8NAEN34bf2qevSyWIQUbE1E0ItY9OKxom2FJi2b7bRdupuE3Y20hP4cL/4VLyKKePWXuK1FtPpg4PHeDDPzgpgzpR3n3ZqZnZtfWFxazqysrq1vZDe3qipKJIUKjXgkbwOigLMQKpppDrexBCICDrWgdzHya3cgFYvCGz2IwRekE7I2o0QbqZk969n9/X7DiyUTkMen2IN+bBfcg0O7X/jWGzfeNesI0kgL7vCnkW9mc07RGQP/Je6E5NAE5Wb22WtFNBEQasqJUnXXibWfEqkZ5TDMeImCmNAe6UDd0JAIUH46fnSI94zSwu1Imgo1Hqs/J1IilBqIwHQKortq2huJ/3n1RLdP/JSFcaIhpF+L2gnHOsKj1HCLSaCaDwwhVDJzK6ZdIgnVJtuMCcGdfvkvqR4WXafoXh3lSueTOJbQDtpFNnLRMSqhS1RGFUTRPXpEL+jVerCerDfr/at1xprMbKNfsD4+ATvspEM=</latexit>

Page 27: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Cosine similarity

k(x, x0) =xTx0

kxkkx0k<latexit sha1_base64="9sXSSeNWieqcDawQRIqsRuxFDAI=">AAACHnicbZDLSgMxFIYz9VbrbdSlm2ARKkiZEUU3QtGNywq9QWdaMmmmDc1cSDIyZTpP4sZXceNCEcGVvo1pO4K2Hkj4+f5zSM7vhIwKaRhfWm5peWV1Lb9e2Njc2t7Rd/caIog4JnUcsIC3HCQIoz6pSyoZaYWcIM9hpOkMbyZ+855wQQO/JkchsT3U96lLMZIKdfXzYSk+iTtWyKlHjuEVtFyOcBJ3aj8wTaxxbI3VlQFrnHb1olE2pgUXhZmJIsiq2tU/rF6AI4/4EjMkRNs0QmkniEuKGUkLViRIiPAQ9UlbSR95RNjJdL0UHinSg27A1fElnNLfEwnyhBh5jur0kByIeW8C//PakXQv7YT6YSSJj2cPuRGDMoCTrGCPcoIlGymBMKfqrxAPkMpHqkQLKgRzfuVF0Tgtm0bZvDsrVq6zOPLgAByCEjDBBaiAW1AFdYDBA3gCL+BVe9SetTftfdaa07KZffCntM9vllejcQ==</latexit><latexit sha1_base64="9sXSSeNWieqcDawQRIqsRuxFDAI=">AAACHnicbZDLSgMxFIYz9VbrbdSlm2ARKkiZEUU3QtGNywq9QWdaMmmmDc1cSDIyZTpP4sZXceNCEcGVvo1pO4K2Hkj4+f5zSM7vhIwKaRhfWm5peWV1Lb9e2Njc2t7Rd/caIog4JnUcsIC3HCQIoz6pSyoZaYWcIM9hpOkMbyZ+855wQQO/JkchsT3U96lLMZIKdfXzYSk+iTtWyKlHjuEVtFyOcBJ3aj8wTaxxbI3VlQFrnHb1olE2pgUXhZmJIsiq2tU/rF6AI4/4EjMkRNs0QmkniEuKGUkLViRIiPAQ9UlbSR95RNjJdL0UHinSg27A1fElnNLfEwnyhBh5jur0kByIeW8C//PakXQv7YT6YSSJj2cPuRGDMoCTrGCPcoIlGymBMKfqrxAPkMpHqkQLKgRzfuVF0Tgtm0bZvDsrVq6zOPLgAByCEjDBBaiAW1AFdYDBA3gCL+BVe9SetTftfdaa07KZffCntM9vllejcQ==</latexit><latexit sha1_base64="9sXSSeNWieqcDawQRIqsRuxFDAI=">AAACHnicbZDLSgMxFIYz9VbrbdSlm2ARKkiZEUU3QtGNywq9QWdaMmmmDc1cSDIyZTpP4sZXceNCEcGVvo1pO4K2Hkj4+f5zSM7vhIwKaRhfWm5peWV1Lb9e2Njc2t7Rd/caIog4JnUcsIC3HCQIoz6pSyoZaYWcIM9hpOkMbyZ+855wQQO/JkchsT3U96lLMZIKdfXzYSk+iTtWyKlHjuEVtFyOcBJ3aj8wTaxxbI3VlQFrnHb1olE2pgUXhZmJIsiq2tU/rF6AI4/4EjMkRNs0QmkniEuKGUkLViRIiPAQ9UlbSR95RNjJdL0UHinSg27A1fElnNLfEwnyhBh5jur0kByIeW8C//PakXQv7YT6YSSJj2cPuRGDMoCTrGCPcoIlGymBMKfqrxAPkMpHqkQLKgRzfuVF0Tgtm0bZvDsrVq6zOPLgAByCEjDBBaiAW1AFdYDBA3gCL+BVe9SetTftfdaa07KZffCntM9vllejcQ==</latexit><latexit sha1_base64="9sXSSeNWieqcDawQRIqsRuxFDAI=">AAACHnicbZDLSgMxFIYz9VbrbdSlm2ARKkiZEUU3QtGNywq9QWdaMmmmDc1cSDIyZTpP4sZXceNCEcGVvo1pO4K2Hkj4+f5zSM7vhIwKaRhfWm5peWV1Lb9e2Njc2t7Rd/caIog4JnUcsIC3HCQIoz6pSyoZaYWcIM9hpOkMbyZ+855wQQO/JkchsT3U96lLMZIKdfXzYSk+iTtWyKlHjuEVtFyOcBJ3aj8wTaxxbI3VlQFrnHb1olE2pgUXhZmJIsiq2tU/rF6AI4/4EjMkRNs0QmkniEuKGUkLViRIiPAQ9UlbSR95RNjJdL0UHinSg27A1fElnNLfEwnyhBh5jur0kByIeW8C//PakXQv7YT6YSSJj2cPuRGDMoCTrGCPcoIlGymBMKfqrxAPkMpHqkQLKgRzfuVF0Tgtm0bZvDsrVq6zOPLgAByCEjDBBaiAW1AFdYDBA3gCL+BVe9SetTftfdaa07KZffCntM9vllejcQ==</latexit>

Page 28: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Mercer kernelsNot all functions allow the kernel trick, only Mercer kernels

Page 29: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Mercer kernelsNot all functions allow the kernel trick, only Mercer kernels

A Mercer kernel is a positive definite kernel

Page 30: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Mercer kernelsNot all functions allow the kernel trick, only Mercer kernels

A Mercer kernel is a positive definite kernel

Mercer theorem: if K is positive definite, we can compute an EVD

Page 31: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Mercer kernelsNot all functions allow the kernel trick, only Mercer kernels

A Mercer kernel is a positive definite kernel

Mercer theorem: if K is positive definite, we can compute an EVD

K = UT⇤U<latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit>

Page 32: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Mercer kernelsNot all functions allow the kernel trick, only Mercer kernels

A Mercer kernel is a positive definite kernel

Mercer theorem: if K is positive definite, we can compute an EVD

K = UT⇤U<latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit>

kij = (⇤12U:,i)

T (⇤12U:,j)

<latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit><latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit><latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit><latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit>

Page 33: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Mercer kernelsNot all functions allow the kernel trick, only Mercer kernels

A Mercer kernel is a positive definite kernel

Mercer theorem: if K is positive definite, we can compute an EVD

K = UT⇤U<latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit>

kij = (⇤12U:,i)

T (⇤12U:,j)

<latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit><latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit><latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit><latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit>

define �(ai) = ⇤12U:,i

<latexit sha1_base64="bxx5p6RG9R99Sx4/RvjTTrBLwgY=">AAACDXicbVDLSsNAFJ3UV62vqEs3g1WoICUpgiIIRTcuXFSwD2himEwm7dDJJMxMhBL6A278FTcuFHHr3p1/47TNQlsPDBzOuYc79/gJo1JZ1rdRWFhcWl4prpbW1jc2t8ztnZaMU4FJE8csFh0fScIoJ01FFSOdRBAU+Yy0/cHV2G8/ECFpzO/UMCFuhHqchhQjpSXPPHAafVpBHj2CF9C50cEA3TuhQBjaNdj0svNjOvLMslW1JoDzxM5JGeRoeOaXE8Q4jQhXmCEpu7aVKDdDQlHMyKjkpJIkCA9Qj3Q15Sgi0s0m14zgoVYCGMZCP67gRP2dyFAk5TDy9WSEVF/OemPxP6+bqvDMzShPUkU4ni4KUwZVDMfVwIAKghUbaoKwoPqvEPeRrkLpAku6BHv25HnSqlVtq2rfnpTrl3kdRbAH9kEF2OAU1ME1aIAmwOARPINX8GY8GS/Gu/ExHS0YeWYX/IHx+QP6IZmY</latexit><latexit sha1_base64="bxx5p6RG9R99Sx4/RvjTTrBLwgY=">AAACDXicbVDLSsNAFJ3UV62vqEs3g1WoICUpgiIIRTcuXFSwD2himEwm7dDJJMxMhBL6A278FTcuFHHr3p1/47TNQlsPDBzOuYc79/gJo1JZ1rdRWFhcWl4prpbW1jc2t8ztnZaMU4FJE8csFh0fScIoJ01FFSOdRBAU+Yy0/cHV2G8/ECFpzO/UMCFuhHqchhQjpSXPPHAafVpBHj2CF9C50cEA3TuhQBjaNdj0svNjOvLMslW1JoDzxM5JGeRoeOaXE8Q4jQhXmCEpu7aVKDdDQlHMyKjkpJIkCA9Qj3Q15Sgi0s0m14zgoVYCGMZCP67gRP2dyFAk5TDy9WSEVF/OemPxP6+bqvDMzShPUkU4ni4KUwZVDMfVwIAKghUbaoKwoPqvEPeRrkLpAku6BHv25HnSqlVtq2rfnpTrl3kdRbAH9kEF2OAU1ME1aIAmwOARPINX8GY8GS/Gu/ExHS0YeWYX/IHx+QP6IZmY</latexit><latexit sha1_base64="bxx5p6RG9R99Sx4/RvjTTrBLwgY=">AAACDXicbVDLSsNAFJ3UV62vqEs3g1WoICUpgiIIRTcuXFSwD2himEwm7dDJJMxMhBL6A278FTcuFHHr3p1/47TNQlsPDBzOuYc79/gJo1JZ1rdRWFhcWl4prpbW1jc2t8ztnZaMU4FJE8csFh0fScIoJ01FFSOdRBAU+Yy0/cHV2G8/ECFpzO/UMCFuhHqchhQjpSXPPHAafVpBHj2CF9C50cEA3TuhQBjaNdj0svNjOvLMslW1JoDzxM5JGeRoeOaXE8Q4jQhXmCEpu7aVKDdDQlHMyKjkpJIkCA9Qj3Q15Sgi0s0m14zgoVYCGMZCP67gRP2dyFAk5TDy9WSEVF/OemPxP6+bqvDMzShPUkU4ni4KUwZVDMfVwIAKghUbaoKwoPqvEPeRrkLpAku6BHv25HnSqlVtq2rfnpTrl3kdRbAH9kEF2OAU1ME1aIAmwOARPINX8GY8GS/Gu/ExHS0YeWYX/IHx+QP6IZmY</latexit>

Page 34: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Mercer kernelsNot all functions allow the kernel trick, only Mercer kernels

A Mercer kernel is a positive definite kernel

Mercer theorem: if K is positive definite, we can compute an EVD

K = UT⇤U<latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit><latexit sha1_base64="EX8JptGrttSmOMcleqTR1T/9rDQ=">AAAB+XicbVBNS8NAFHypX7V+RT16WSyCp5KIoBeh6EXQQ4WmLbSxbDabdulmE3Y3hRL6T7x4UMSr/8Sb/8Ztm4NWBxaGmTe8txOknCntOF9WaWV1bX2jvFnZ2t7Z3bP3D1oqySShHkl4IjsBVpQzQT3NNKedVFIcB5y2g9HNzG+PqVQsEU09Sakf44FgESNYG6lv23foCnmPzd69yYQYeX276tScOdBf4hakCgUaffuzFyYki6nQhGOluq6Taj/HUjPC6bTSyxRNMRnhAe0aKnBMlZ/PL5+iE6OEKEqkeUKjufozkeNYqUkcmMkY66Fa9mbif14309GlnzORZpoKslgUZRzpBM1qQCGTlGg+MQQTycytiAyxxESbsiqmBHf5y39J66zmOjX34bxavy7qKMMRHMMpuHABdbiFBnhAYAxP8AKvVm49W2/W+2K0ZBWZQ/gF6+MbWeuSLA==</latexit>

kij = (⇤12U:,i)

T (⇤12U:,j)

<latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit><latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit><latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit><latexit sha1_base64="7PSVHk128MLp8+3WPM34PhlN8JU=">AAACKHicdVDLSgMxFM3UV62vqks3wSK0IGWmCIogFt24cFGhL+hMh0yaadNmHiQZoQzzOW78FTciinTrl5i2s9BWDwQO59zDzT1OyKiQuj7RMiura+sb2c3c1vbO7l5+/6Apgohj0sABC3jbQYIw6pOGpJKRdsgJ8hxGWs7oduq3HgkXNPDrchwSy0N9n7oUI6kkO389smM6TOAVLJr3KtZDXdPlCEOjAht2fHlKk1K3/o83TEp2vqCX9RngMjFSUgApanb+zewFOPKILzFDQnQMPZRWjLikmJEkZ0aChAiPUJ90FPWRR4QVzw5N4IlSetANuHq+hDP1ZyJGnhBjz1GTHpIDsehNxb+8TiTdCyumfhhJ4uP5IjdiUAZw2hrsUU6wZGNFEOZU/RXiAVJdSNVtTpVgLJ68TJqVsqGXjYezQvUmrSMLjsAxKAIDnIMquAM10AAYPIEX8A4+tGftVfvUJvPRjJZmDsEvaF/fV4ujqg==</latexit>

define �(ai) = ⇤12U:,i

<latexit sha1_base64="bxx5p6RG9R99Sx4/RvjTTrBLwgY=">AAACDXicbVDLSsNAFJ3UV62vqEs3g1WoICUpgiIIRTcuXFSwD2himEwm7dDJJMxMhBL6A278FTcuFHHr3p1/47TNQlsPDBzOuYc79/gJo1JZ1rdRWFhcWl4prpbW1jc2t8ztnZaMU4FJE8csFh0fScIoJ01FFSOdRBAU+Yy0/cHV2G8/ECFpzO/UMCFuhHqchhQjpSXPPHAafVpBHj2CF9C50cEA3TuhQBjaNdj0svNjOvLMslW1JoDzxM5JGeRoeOaXE8Q4jQhXmCEpu7aVKDdDQlHMyKjkpJIkCA9Qj3Q15Sgi0s0m14zgoVYCGMZCP67gRP2dyFAk5TDy9WSEVF/OemPxP6+bqvDMzShPUkU4ni4KUwZVDMfVwIAKghUbaoKwoPqvEPeRrkLpAku6BHv25HnSqlVtq2rfnpTrl3kdRbAH9kEF2OAU1ME1aIAmwOARPINX8GY8GS/Gu/ExHS0YeWYX/IHx+QP6IZmY</latexit><latexit sha1_base64="bxx5p6RG9R99Sx4/RvjTTrBLwgY=">AAACDXicbVDLSsNAFJ3UV62vqEs3g1WoICUpgiIIRTcuXFSwD2himEwm7dDJJMxMhBL6A278FTcuFHHr3p1/47TNQlsPDBzOuYc79/gJo1JZ1rdRWFhcWl4prpbW1jc2t8ztnZaMU4FJE8csFh0fScIoJ01FFSOdRBAU+Yy0/cHV2G8/ECFpzO/UMCFuhHqchhQjpSXPPHAafVpBHj2CF9C50cEA3TuhQBjaNdj0svNjOvLMslW1JoDzxM5JGeRoeOaXE8Q4jQhXmCEpu7aVKDdDQlHMyKjkpJIkCA9Qj3Q15Sgi0s0m14zgoVYCGMZCP67gRP2dyFAk5TDy9WSEVF/OemPxP6+bqvDMzShPUkU4ni4KUwZVDMfVwIAKghUbaoKwoPqvEPeRrkLpAku6BHv25HnSqlVtq2rfnpTrl3kdRbAH9kEF2OAU1ME1aIAmwOARPINX8GY8GS/Gu/ExHS0YeWYX/IHx+QP6IZmY</latexit><latexit sha1_base64="bxx5p6RG9R99Sx4/RvjTTrBLwgY=">AAACDXicbVDLSsNAFJ3UV62vqEs3g1WoICUpgiIIRTcuXFSwD2himEwm7dDJJMxMhBL6A278FTcuFHHr3p1/47TNQlsPDBzOuYc79/gJo1JZ1rdRWFhcWl4prpbW1jc2t8ztnZaMU4FJE8csFh0fScIoJ01FFSOdRBAU+Yy0/cHV2G8/ECFpzO/UMCFuhHqchhQjpSXPPHAafVpBHj2CF9C50cEA3TuhQBjaNdj0svNjOvLMslW1JoDzxM5JGeRoeOaXE8Q4jQhXmCEpu7aVKDdDQlHMyKjkpJIkCA9Qj3Q15Sgi0s0m14zgoVYCGMZCP67gRP2dyFAk5TDy9WSEVF/OemPxP6+bqvDMzShPUkU4ni4KUwZVDMfVwIAKghUbaoKwoPqvEPeRrkLpAku6BHv25HnSqlVtq2rfnpTrl3kdRbAH9kEF2OAU1ME1aIAmwOARPINX8GY8GS/Gu/ExHS0YeWYX/IHx+QP6IZmY</latexit>

K can be computed by inner product of feature vectors implicitly defined by U

Page 35: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Kernel PCA: properties

Kernel matrix is n⇥ n<latexit sha1_base64="n0+SQDjdsr7eyqurHHzy1g6Wn80=">AAAB8XicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8cK9gPbUDbbTbt0swm7E6GE/gsvHhTx6r/x5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecJLyIKZDJSLBKFrpUZEeipgbovpu1at5c5BV4hekCgUafferN0hYFnOFTFJjur6XYpBTjYJJPq30MsNTysZ0yLuWKmrXBPn84ik5s8qARIm2pZDM1d8TOY2NmcSh7YwpjsyyNxP/87oZRtdBLlSaIVdssSjKJMGEzN4nA6E5QzmxhDIt7K2EjaimDG1IFRuCv/zyKmld1Hyv5t9fVus3RRxlOIFTOAcfrqAOd9CAJjBQ8Ayv8OYY58V5dz4WrSWnmDmGP3A+fwAChpB4</latexit><latexit sha1_base64="n0+SQDjdsr7eyqurHHzy1g6Wn80=">AAAB8XicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8cK9gPbUDbbTbt0swm7E6GE/gsvHhTx6r/x5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecJLyIKZDJSLBKFrpUZEeipgbovpu1at5c5BV4hekCgUafferN0hYFnOFTFJjur6XYpBTjYJJPq30MsNTysZ0yLuWKmrXBPn84ik5s8qARIm2pZDM1d8TOY2NmcSh7YwpjsyyNxP/87oZRtdBLlSaIVdssSjKJMGEzN4nA6E5QzmxhDIt7K2EjaimDG1IFRuCv/zyKmld1Hyv5t9fVus3RRxlOIFTOAcfrqAOd9CAJjBQ8Ayv8OYY58V5dz4WrSWnmDmGP3A+fwAChpB4</latexit><latexit sha1_base64="n0+SQDjdsr7eyqurHHzy1g6Wn80=">AAAB8XicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8cK9gPbUDbbTbt0swm7E6GE/gsvHhTx6r/x5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecJLyIKZDJSLBKFrpUZEeipgbovpu1at5c5BV4hekCgUafferN0hYFnOFTFJjur6XYpBTjYJJPq30MsNTysZ0yLuWKmrXBPn84ik5s8qARIm2pZDM1d8TOY2NmcSh7YwpjsyyNxP/87oZRtdBLlSaIVdssSjKJMGEzN4nA6E5QzmxhDIt7K2EjaimDG1IFRuCv/zyKmld1Hyv5t9fVus3RRxlOIFTOAcfrqAOd9CAJjBQ8Ayv8OYY58V5dz4WrSWnmDmGP3A+fwAChpB4</latexit><latexit sha1_base64="n0+SQDjdsr7eyqurHHzy1g6Wn80=">AAAB8XicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8cK9gPbUDbbTbt0swm7E6GE/gsvHhTx6r/x5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecJLyIKZDJSLBKFrpUZEeipgbovpu1at5c5BV4hekCgUafferN0hYFnOFTFJjur6XYpBTjYJJPq30MsNTysZ0yLuWKmrXBPn84ik5s8qARIm2pZDM1d8TOY2NmcSh7YwpjsyyNxP/87oZRtdBLlSaIVdssSjKJMGEzN4nA6E5QzmxhDIt7K2EjaimDG1IFRuCv/zyKmld1Hyv5t9fVus3RRxlOIFTOAcfrqAOd9CAJjBQ8Ayv8OYY58V5dz4WrSWnmDmGP3A+fwAChpB4</latexit>

Can give a good reencoding of the data when it lies along a non-linear manifold

Can have n positive eigenvalues (not necessarily dimensionality reduction)

Unlike PCA, some directions in feature space might not have preimages in input space.

Page 36: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Note on Mercer kernels

Page 37: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Note on Mercer kernels“Sigmoid kernel” k(x, x0) = tanh(�xTx0 + r)

<latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit><latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit><latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit><latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit>

Page 38: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Note on Mercer kernels“Sigmoid kernel” k(x, x0) = tanh(�xTx0 + r)

<latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit><latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit><latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit><latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit>

Not a sigmoid, nor a Mercer kernel

Page 39: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Note on Mercer kernels“Sigmoid kernel” k(x, x0) = tanh(�xTx0 + r)

<latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit><latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit><latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit><latexit sha1_base64="ry9qXQ+iFPtdaqnG04r+yqOynR0=">AAACFnicbVDLSgNBEJyNrxhfqx69DAYhooZdEfQiiF48RsgLsjH0TibJkJnZZWZWEoJf4cVf8eJBEa/izb9x8hA0saChqOqmuyuMOdPG876c1Nz8wuJSejmzsrq2vuFubpV1lChCSyTikaqGoClnkpYMM5xWY0VBhJxWwu7V0K/cUaVZJIumH9O6gLZkLUbAWKnhHnVzvcPebRArJug+PseBAdnJBW0QAnDvtvjj4QOs9htu1st7I+BZ4k9IFk1QaLifQTMiiaDSEA5a13wvNvUBKMMIp/eZINE0BtKFNq1ZKkFQXR+M3rrHe1Zp4lakbEmDR+rviQEIrfsitJ0CTEdPe0PxP6+WmNZZfcBknBgqyXhRK+HYRHiYEW4yRYnhfUuAKGZvxaQDCoixSWZsCP70y7OkfJz3vbx/c5K9uJzEkUY7aBflkI9O0QW6RgVUQgQ9oCf0gl6dR+fZeXPex60pZzKzjf7A+fgGpgudxg==</latexit>

Not a sigmoid, nor a Mercer kernel

Inspired by the multi-layer perceptron, but there is no good reason to use it…

Page 40: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Note on Mercer kernels

k(x, x0) =<latexit sha1_base64="NAMTGXD8lfBYRZtV2DhoDMROtkU=">AAAB+XicbVBNSwMxEJ31s9avVY9egkWoIGVXBL0IRS8eK9gPaNeSTdM2NMkuSba0LP0nXjwo4tV/4s1/Y9ruQVsfDDzem2FmXhhzpo3nfTsrq2vrG5u5rfz2zu7evntwWNNRogitkohHqhFiTTmTtGqY4bQRK4pFyGk9HNxN/fqQKs0i+WjGMQ0E7knWZQQbK7Vdd1AcnY+eWrFigp6hG9R2C17JmwEtEz8jBchQabtfrU5EEkGlIRxr3fS92AQpVoYRTif5VqJpjMkA92jTUokF1UE6u3yCTq3SQd1I2ZIGzdTfEykWWo9FaDsFNn296E3F/7xmYrrXQcpknBgqyXxRN+HIRGgaA+owRYnhY0swUczeikgfK0yMDStvQ/AXX14mtYuS75X8h8tC+TaLIwfHcAJF8OEKynAPFagCgSE8wyu8Oanz4rw7H/PWFSebOYI/cD5/ANQPknc=</latexit><latexit sha1_base64="NAMTGXD8lfBYRZtV2DhoDMROtkU=">AAAB+XicbVBNSwMxEJ31s9avVY9egkWoIGVXBL0IRS8eK9gPaNeSTdM2NMkuSba0LP0nXjwo4tV/4s1/Y9ruQVsfDDzem2FmXhhzpo3nfTsrq2vrG5u5rfz2zu7evntwWNNRogitkohHqhFiTTmTtGqY4bQRK4pFyGk9HNxN/fqQKs0i+WjGMQ0E7knWZQQbK7Vdd1AcnY+eWrFigp6hG9R2C17JmwEtEz8jBchQabtfrU5EEkGlIRxr3fS92AQpVoYRTif5VqJpjMkA92jTUokF1UE6u3yCTq3SQd1I2ZIGzdTfEykWWo9FaDsFNn296E3F/7xmYrrXQcpknBgqyXxRN+HIRGgaA+owRYnhY0swUczeikgfK0yMDStvQ/AXX14mtYuS75X8h8tC+TaLIwfHcAJF8OEKynAPFagCgSE8wyu8Oanz4rw7H/PWFSebOYI/cD5/ANQPknc=</latexit><latexit sha1_base64="NAMTGXD8lfBYRZtV2DhoDMROtkU=">AAAB+XicbVBNSwMxEJ31s9avVY9egkWoIGVXBL0IRS8eK9gPaNeSTdM2NMkuSba0LP0nXjwo4tV/4s1/Y9ruQVsfDDzem2FmXhhzpo3nfTsrq2vrG5u5rfz2zu7evntwWNNRogitkohHqhFiTTmTtGqY4bQRK4pFyGk9HNxN/fqQKs0i+WjGMQ0E7knWZQQbK7Vdd1AcnY+eWrFigp6hG9R2C17JmwEtEz8jBchQabtfrU5EEkGlIRxr3fS92AQpVoYRTif5VqJpjMkA92jTUokF1UE6u3yCTq3SQd1I2ZIGzdTfEykWWo9FaDsFNn296E3F/7xmYrrXQcpknBgqyXxRN+HIRGgaA+owRYnhY0swUczeikgfK0yMDStvQ/AXX14mtYuS75X8h8tC+TaLIwfHcAJF8OEKynAPFagCgSE8wyu8Oanz4rw7H/PWFSebOYI/cD5/ANQPknc=</latexit><latexit sha1_base64="NAMTGXD8lfBYRZtV2DhoDMROtkU=">AAAB+XicbVBNSwMxEJ31s9avVY9egkWoIGVXBL0IRS8eK9gPaNeSTdM2NMkuSba0LP0nXjwo4tV/4s1/Y9ruQVsfDDzem2FmXhhzpo3nfTsrq2vrG5u5rfz2zu7evntwWNNRogitkohHqhFiTTmTtGqY4bQRK4pFyGk9HNxN/fqQKs0i+WjGMQ0E7knWZQQbK7Vdd1AcnY+eWrFigp6hG9R2C17JmwEtEz8jBchQabtfrU5EEkGlIRxr3fS92AQpVoYRTif5VqJpjMkA92jTUokF1UE6u3yCTq3SQd1I2ZIGzdTfEykWWo9FaDsFNn296E3F/7xmYrrXQcpknBgqyXxRN+HIRGgaA+owRYnhY0swUczeikgfK0yMDStvQ/AXX14mtYuS75X8h8tC+TaLIwfHcAJF8OEKynAPFagCgSE8wyu8Oanz4rw7H/PWFSebOYI/cD5/ANQPknc=</latexit>

True NN kernel:

x̃ =

1x

<latexit sha1_base64="oqLl+mwc6yxARsov7Uj6SeeZtXs=">AAACKnicbVDLSsNAFJ34rPFVdelmsAiuSiKCboSqG5cV7AOaUiaTm3boZBJmJtIS+j1u/BU3XSjFrR/ipC0+Wg8MHM659869x084U9pxJtbK6tr6xmZhy97e2d3bLx4c1lWcSgo1GvNYNn2igDMBNc00h2YigUQ+h4bfv8v9xhNIxWLxqIcJtCPSFSxklGgjdYo3nmY8gMyLiO75YTYYjfA19nzoMpH5RpRsMLJdz7N/KmwPRPBtdoolp+xMgZeJOyclNEe1Uxx7QUzTCISmnCjVcp1EtzMiNaMczPRUQUJon3ShZaggEah2Nj11hE+NEuAwluYJjafq746MREoNI99U5vuqRS8X//NaqQ6v2hkTSapB0NlHYcqxjnGeGw6YBKr50BBCJTO7YtojklBt0rVNCO7iycukfl52nbL7cFGq3M7jKKBjdILOkIsuUQXdoyqqIYqe0St6Q+/WizW2JtbHrHTFmvccoT+wPr8AV9+obQ==</latexit><latexit sha1_base64="oqLl+mwc6yxARsov7Uj6SeeZtXs=">AAACKnicbVDLSsNAFJ34rPFVdelmsAiuSiKCboSqG5cV7AOaUiaTm3boZBJmJtIS+j1u/BU3XSjFrR/ipC0+Wg8MHM659869x084U9pxJtbK6tr6xmZhy97e2d3bLx4c1lWcSgo1GvNYNn2igDMBNc00h2YigUQ+h4bfv8v9xhNIxWLxqIcJtCPSFSxklGgjdYo3nmY8gMyLiO75YTYYjfA19nzoMpH5RpRsMLJdz7N/KmwPRPBtdoolp+xMgZeJOyclNEe1Uxx7QUzTCISmnCjVcp1EtzMiNaMczPRUQUJon3ShZaggEah2Nj11hE+NEuAwluYJjafq746MREoNI99U5vuqRS8X//NaqQ6v2hkTSapB0NlHYcqxjnGeGw6YBKr50BBCJTO7YtojklBt0rVNCO7iycukfl52nbL7cFGq3M7jKKBjdILOkIsuUQXdoyqqIYqe0St6Q+/WizW2JtbHrHTFmvccoT+wPr8AV9+obQ==</latexit><latexit sha1_base64="oqLl+mwc6yxARsov7Uj6SeeZtXs=">AAACKnicbVDLSsNAFJ34rPFVdelmsAiuSiKCboSqG5cV7AOaUiaTm3boZBJmJtIS+j1u/BU3XSjFrR/ipC0+Wg8MHM659869x084U9pxJtbK6tr6xmZhy97e2d3bLx4c1lWcSgo1GvNYNn2igDMBNc00h2YigUQ+h4bfv8v9xhNIxWLxqIcJtCPSFSxklGgjdYo3nmY8gMyLiO75YTYYjfA19nzoMpH5RpRsMLJdz7N/KmwPRPBtdoolp+xMgZeJOyclNEe1Uxx7QUzTCISmnCjVcp1EtzMiNaMczPRUQUJon3ShZaggEah2Nj11hE+NEuAwluYJjafq746MREoNI99U5vuqRS8X//NaqQ6v2hkTSapB0NlHYcqxjnGeGw6YBKr50BBCJTO7YtojklBt0rVNCO7iycukfl52nbL7cFGq3M7jKKBjdILOkIsuUQXdoyqqIYqe0St6Q+/WizW2JtbHrHTFmvccoT+wPr8AV9+obQ==</latexit><latexit sha1_base64="oqLl+mwc6yxARsov7Uj6SeeZtXs=">AAACKnicbVDLSsNAFJ34rPFVdelmsAiuSiKCboSqG5cV7AOaUiaTm3boZBJmJtIS+j1u/BU3XSjFrR/ipC0+Wg8MHM659869x084U9pxJtbK6tr6xmZhy97e2d3bLx4c1lWcSgo1GvNYNn2igDMBNc00h2YigUQ+h4bfv8v9xhNIxWLxqIcJtCPSFSxklGgjdYo3nmY8gMyLiO75YTYYjfA19nzoMpH5RpRsMLJdz7N/KmwPRPBtdoolp+xMgZeJOyclNEe1Uxx7QUzTCISmnCjVcp1EtzMiNaMczPRUQUJon3ShZaggEah2Nj11hE+NEuAwluYJjafq746MREoNI99U5vuqRS8X//NaqQ6v2hkTSapB0NlHYcqxjnGeGw6YBKr50BBCJTO7YtojklBt0rVNCO7iycukfl52nbL7cFGq3M7jKKBjdILOkIsuUQXdoyqqIYqe0St6Q+/WizW2JtbHrHTFmvccoT+wPr8AV9+obQ==</latexit>

⌃<latexit sha1_base64="xl6gt5zesTVLaQIKxfcD+hK+ntk=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKEI9BLx4jmgckS5idzCZj5rHMzAphyT948aCIV//Hm3/jJNmDJhY0FFXddHdFCWfG+v63V1hb39jcKm6Xdnb39g/Kh0cto1JNaJMornQnwoZyJmnTMstpJ9EUi4jTdjS+mfntJ6oNU/LBThIaCjyULGYEWye1evdsKHC/XPGr/hxolQQ5qUCORr/81RsokgoqLeHYmG7gJzbMsLaMcDot9VJDE0zGeEi7jkosqAmz+bVTdOaUAYqVdiUtmqu/JzIsjJmIyHUKbEdm2ZuJ/3nd1MZXYcZkkloqyWJRnHJkFZq9jgZMU2L5xBFMNHO3IjLCGhPrAiq5EILll1dJ66Ia+NXg7rJSv87jKMIJnMI5BFCDOtxCA5pA4BGe4RXePOW9eO/ex6K14OUzx/AH3ucPa1+PAw==</latexit><latexit sha1_base64="xl6gt5zesTVLaQIKxfcD+hK+ntk=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKEI9BLx4jmgckS5idzCZj5rHMzAphyT948aCIV//Hm3/jJNmDJhY0FFXddHdFCWfG+v63V1hb39jcKm6Xdnb39g/Kh0cto1JNaJMornQnwoZyJmnTMstpJ9EUi4jTdjS+mfntJ6oNU/LBThIaCjyULGYEWye1evdsKHC/XPGr/hxolQQ5qUCORr/81RsokgoqLeHYmG7gJzbMsLaMcDot9VJDE0zGeEi7jkosqAmz+bVTdOaUAYqVdiUtmqu/JzIsjJmIyHUKbEdm2ZuJ/3nd1MZXYcZkkloqyWJRnHJkFZq9jgZMU2L5xBFMNHO3IjLCGhPrAiq5EILll1dJ66Ia+NXg7rJSv87jKMIJnMI5BFCDOtxCA5pA4BGe4RXePOW9eO/ex6K14OUzx/AH3ucPa1+PAw==</latexit><latexit sha1_base64="xl6gt5zesTVLaQIKxfcD+hK+ntk=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKEI9BLx4jmgckS5idzCZj5rHMzAphyT948aCIV//Hm3/jJNmDJhY0FFXddHdFCWfG+v63V1hb39jcKm6Xdnb39g/Kh0cto1JNaJMornQnwoZyJmnTMstpJ9EUi4jTdjS+mfntJ6oNU/LBThIaCjyULGYEWye1evdsKHC/XPGr/hxolQQ5qUCORr/81RsokgoqLeHYmG7gJzbMsLaMcDot9VJDE0zGeEi7jkosqAmz+bVTdOaUAYqVdiUtmqu/JzIsjJmIyHUKbEdm2ZuJ/3nd1MZXYcZkkloqyWJRnHJkFZq9jgZMU2L5xBFMNHO3IjLCGhPrAiq5EILll1dJ66Ia+NXg7rJSv87jKMIJnMI5BFCDOtxCA5pA4BGe4RXePOW9eO/ex6K14OUzx/AH3ucPa1+PAw==</latexit><latexit sha1_base64="xl6gt5zesTVLaQIKxfcD+hK+ntk=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKEI9BLx4jmgckS5idzCZj5rHMzAphyT948aCIV//Hm3/jJNmDJhY0FFXddHdFCWfG+v63V1hb39jcKm6Xdnb39g/Kh0cto1JNaJMornQnwoZyJmnTMstpJ9EUi4jTdjS+mfntJ6oNU/LBThIaCjyULGYEWye1evdsKHC/XPGr/hxolQQ5qUCORr/81RsokgoqLeHYmG7gJzbMsLaMcDot9VJDE0zGeEi7jkosqAmz+bVTdOaUAYqVdiUtmqu/JzIsjJmIyHUKbEdm2ZuJ/3nd1MZXYcZkkloqyWJRnHJkFZq9jgZMU2L5xBFMNHO3IjLCGhPrAiq5EILll1dJ66Ia+NXg7rJSv87jKMIJnMI5BFCDOtxCA5pA4BGe4RXePOW9eO/ex6K14OUzx/AH3ucPa1+PAw==</latexit>

Covariance matrix of a Gaussian process

Murphy, 2012� = 10<latexit sha1_base64="7DcvypsyjNY2TwKStaH9k4JTaHY=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoBch6MVjBPOAzRJmJ7PJkHksM7NCWPIZXjwo4tWv8ebfOEn2oIkFDUVVN91dccqZsb7/7ZXW1jc2t8rblZ3dvf2D6uFR26hME9oiiivdjbGhnEnassxy2k01xSLmtBOP72Z+54lqw5R8tJOURgIPJUsYwdZJYc+wocDoBgV+v1rz6/4caJUEBalBgWa/+tUbKJIJKi3h2Jgw8FMb5VhbRjidVnqZoSkmYzykoaMSC2qifH7yFJ05ZYASpV1Ji+bq74kcC2MmInadAtuRWfZm4n9emNnkOsqZTDNLJVksSjKOrEKz/9GAaUosnziCiWbuVkRGWGNiXUoVF0Kw/PIqaV/UA78ePFzWGrdFHGU4gVM4hwCuoAH30IQWEFDwDK/w5lnvxXv3PhatJa+YOYY/8D5/ALN4kDM=</latexit><latexit sha1_base64="7DcvypsyjNY2TwKStaH9k4JTaHY=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoBch6MVjBPOAzRJmJ7PJkHksM7NCWPIZXjwo4tWv8ebfOEn2oIkFDUVVN91dccqZsb7/7ZXW1jc2t8rblZ3dvf2D6uFR26hME9oiiivdjbGhnEnassxy2k01xSLmtBOP72Z+54lqw5R8tJOURgIPJUsYwdZJYc+wocDoBgV+v1rz6/4caJUEBalBgWa/+tUbKJIJKi3h2Jgw8FMb5VhbRjidVnqZoSkmYzykoaMSC2qifH7yFJ05ZYASpV1Ji+bq74kcC2MmInadAtuRWfZm4n9emNnkOsqZTDNLJVksSjKOrEKz/9GAaUosnziCiWbuVkRGWGNiXUoVF0Kw/PIqaV/UA78ePFzWGrdFHGU4gVM4hwCuoAH30IQWEFDwDK/w5lnvxXv3PhatJa+YOYY/8D5/ALN4kDM=</latexit><latexit sha1_base64="7DcvypsyjNY2TwKStaH9k4JTaHY=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoBch6MVjBPOAzRJmJ7PJkHksM7NCWPIZXjwo4tWv8ebfOEn2oIkFDUVVN91dccqZsb7/7ZXW1jc2t8rblZ3dvf2D6uFR26hME9oiiivdjbGhnEnassxy2k01xSLmtBOP72Z+54lqw5R8tJOURgIPJUsYwdZJYc+wocDoBgV+v1rz6/4caJUEBalBgWa/+tUbKJIJKi3h2Jgw8FMb5VhbRjidVnqZoSkmYzykoaMSC2qifH7yFJ05ZYASpV1Ji+bq74kcC2MmInadAtuRWfZm4n9emNnkOsqZTDNLJVksSjKOrEKz/9GAaUosnziCiWbuVkRGWGNiXUoVF0Kw/PIqaV/UA78ePFzWGrdFHGU4gVM4hwCuoAH30IQWEFDwDK/w5lnvxXv3PhatJa+YOYY/8D5/ALN4kDM=</latexit><latexit sha1_base64="7DcvypsyjNY2TwKStaH9k4JTaHY=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoBch6MVjBPOAzRJmJ7PJkHksM7NCWPIZXjwo4tWv8ebfOEn2oIkFDUVVN91dccqZsb7/7ZXW1jc2t8rblZ3dvf2D6uFR26hME9oiiivdjbGhnEnassxy2k01xSLmtBOP72Z+54lqw5R8tJOURgIPJUsYwdZJYc+wocDoBgV+v1rz6/4caJUEBalBgWa/+tUbKJIJKi3h2Jgw8FMb5VhbRjidVnqZoSkmYzykoaMSC2qifH7yFJ05ZYASpV1Ji+bq74kcC2MmInadAtuRWfZm4n9emNnkOsqZTDNLJVksSjKOrEKz/9GAaUosnziCiWbuVkRGWGNiXUoVF0Kw/PIqaV/UA78ePFzWGrdFHGU4gVM4hwCuoAH30IQWEFDwDK/w5lnvxXv3PhatJa+YOYY/8D5/ALN4kDM=</latexit>

Page 41: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCA

Ghashami, Perry, Phillips, Streaming kernel PCA, AISTATS, 2016

Builds on frequent directions

Page 42: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Vanilla kPCA

Page 43: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Vanilla kPCA1. Provides set of basis vectors capturing nonlinear structures within

large datasets;2. Potentially a great tool for data analysis and learning…

Page 44: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Vanilla kPCA1. Provides set of basis vectors capturing nonlinear structures within

large datasets;2. Potentially a great tool for data analysis and learning…

But

Page 45: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Vanilla kPCA1. Provides set of basis vectors capturing nonlinear structures within

large datasets;2. Potentially a great tool for data analysis and learning…

But

To allow for nonlinear relations, typically a full nxn kernel matrix is constructed over n data points

Page 46: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Vanilla kPCA1. Provides set of basis vectors capturing nonlinear structures within

large datasets;2. Potentially a great tool for data analysis and learning…

But

To allow for nonlinear relations, typically a full nxn kernel matrix is constructed over n data points

Requires too much space and time for large values of n

Page 47: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Vanilla kPCA: how much is too much?

Storing matrix K: O(n2)<latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit><latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit><latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit><latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit>

Page 48: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Vanilla kPCA: how much is too much?

Storing matrix K: O(n2)<latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit><latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit><latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit><latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit>

Computing the decomposition of K: O(n3)<latexit sha1_base64="pUzYwhqmk7Jc05OtuANfZpfZ6hg=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquFfRY9OLNCvYD2rVk02wbm02WJCuUpf/BiwdFvPp/vPlvTNs9aOuDgcd7M8zMC2LOtHHdbye3srq2vpHfLGxt7+zuFfcPmlomitAGkVyqdoA15UzQhmGG03asKI4CTlvB6Hrqt56o0kyKezOOqR/hgWAhI9hYqXlbFg/V016x5FbcGdAy8TJSggz1XvGr25ckiagwhGOtO54bGz/FyjDC6aTQTTSNMRnhAe1YKnBEtZ/Orp2gE6v0USiVLWHQTP09keJI63EU2M4Im6Fe9Kbif14nMeGlnzIRJ4YKMl8UJhwZiaavoz5TlBg+tgQTxeytiAyxwsTYgAo2BG/x5WXSPKt4bsW7Oy/VrrI48nAEx1AGDy6gBjdQhwYQeIRneIU3RzovzrvzMW/NOdnMIfyB8/kDYm6OVQ==</latexit><latexit sha1_base64="pUzYwhqmk7Jc05OtuANfZpfZ6hg=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquFfRY9OLNCvYD2rVk02wbm02WJCuUpf/BiwdFvPp/vPlvTNs9aOuDgcd7M8zMC2LOtHHdbye3srq2vpHfLGxt7+zuFfcPmlomitAGkVyqdoA15UzQhmGG03asKI4CTlvB6Hrqt56o0kyKezOOqR/hgWAhI9hYqXlbFg/V016x5FbcGdAy8TJSggz1XvGr25ckiagwhGOtO54bGz/FyjDC6aTQTTSNMRnhAe1YKnBEtZ/Orp2gE6v0USiVLWHQTP09keJI63EU2M4Im6Fe9Kbif14nMeGlnzIRJ4YKMl8UJhwZiaavoz5TlBg+tgQTxeytiAyxwsTYgAo2BG/x5WXSPKt4bsW7Oy/VrrI48nAEx1AGDy6gBjdQhwYQeIRneIU3RzovzrvzMW/NOdnMIfyB8/kDYm6OVQ==</latexit><latexit sha1_base64="pUzYwhqmk7Jc05OtuANfZpfZ6hg=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquFfRY9OLNCvYD2rVk02wbm02WJCuUpf/BiwdFvPp/vPlvTNs9aOuDgcd7M8zMC2LOtHHdbye3srq2vpHfLGxt7+zuFfcPmlomitAGkVyqdoA15UzQhmGG03asKI4CTlvB6Hrqt56o0kyKezOOqR/hgWAhI9hYqXlbFg/V016x5FbcGdAy8TJSggz1XvGr25ckiagwhGOtO54bGz/FyjDC6aTQTTSNMRnhAe1YKnBEtZ/Orp2gE6v0USiVLWHQTP09keJI63EU2M4Im6Fe9Kbif14nMeGlnzIRJ4YKMl8UJhwZiaavoz5TlBg+tgQTxeytiAyxwsTYgAo2BG/x5WXSPKt4bsW7Oy/VrrI48nAEx1AGDy6gBjdQhwYQeIRneIU3RzovzrvzMW/NOdnMIfyB8/kDYm6OVQ==</latexit><latexit sha1_base64="pUzYwhqmk7Jc05OtuANfZpfZ6hg=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquFfRY9OLNCvYD2rVk02wbm02WJCuUpf/BiwdFvPp/vPlvTNs9aOuDgcd7M8zMC2LOtHHdbye3srq2vpHfLGxt7+zuFfcPmlomitAGkVyqdoA15UzQhmGG03asKI4CTlvB6Hrqt56o0kyKezOOqR/hgWAhI9hYqXlbFg/V016x5FbcGdAy8TJSggz1XvGr25ckiagwhGOtO54bGz/FyjDC6aTQTTSNMRnhAe1YKnBEtZ/Orp2gE6v0USiVLWHQTP09keJI63EU2M4Im6Fe9Kbif14nMeGlnzIRJ4YKMl8UJhwZiaavoz5TlBg+tgQTxeytiAyxwsTYgAo2BG/x5WXSPKt4bsW7Oy/VrrI48nAEx1AGDy6gBjdQhwYQeIRneIU3RzovzrvzMW/NOdnMIfyB8/kDYm6OVQ==</latexit>

Page 49: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Vanilla kPCA: how much is too much?

Storing matrix K: O(n2)<latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit><latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit><latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit><latexit sha1_base64="YWVqE8nlPUjFoY6Z77g5giNAzcE=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXspuKeix6MWbFewHtGvJptk2NpssSVYoS/+DFw+KePX/ePPfmLZ70NYHA4/3ZpiZF8ScaeO6305ubX1jcyu/XdjZ3ds/KB4etbRMFKFNIrlUnQBrypmgTcMMp51YURwFnLaD8fXMbz9RpZkU92YSUz/CQ8FCRrCxUuu2LB6q5/1iya24c6BV4mWkBBka/eJXbyBJElFhCMdadz03Nn6KlWGE02mhl2gaYzLGQ9q1VOCIaj+dXztFZ1YZoFAqW8Kgufp7IsWR1pMosJ0RNiO97M3E/7xuYsJLP2UiTgwVZLEoTDgyEs1eRwOmKDF8YgkmitlbERlhhYmxARVsCN7yy6ukVa14bsW7q5XqV1kceTiBUyiDBxdQhxtoQBMIPMIzvMKbI50X5935WLTmnGzmGP7A+fwBYOmOVA==</latexit>

Time to evaluate the kernel function for any arbitrary test vector with respect to all training examples: O(nm)

<latexit sha1_base64="OMLplQzZdF/InFhsmFeSXzDUATY=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBahXkoigh6LXrxZwbSFNpTNdtMu3d2E3Y0QQn+DFw+KePUHefPfuG1z0NYHA4/3ZpiZFyacaeO6305pbX1jc6u8XdnZ3ds/qB4etXWcKkJ9EvNYdUOsKWeS+oYZTruJoliEnHbCye3M7zxRpVksH02W0EDgkWQRI9hYyb+vS3E+qNbchjsHWiVeQWpQoDWofvWHMUkFlYZwrHXPcxMT5FgZRjidVvqppgkmEzyiPUslFlQH+fzYKTqzyhBFsbIlDZqrvydyLLTORGg7BTZjvezNxP+8Xmqi6yBnMkkNlWSxKEo5MjGafY6GTFFieGYJJorZWxEZY4WJsflUbAje8surpH3R8NyG93BZa94UcZThBE6hDh5cQRPuoAU+EGDwDK/w5kjnxXl3PhatJaeYOYY/cD5/AAWLjic=</latexit><latexit sha1_base64="OMLplQzZdF/InFhsmFeSXzDUATY=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBahXkoigh6LXrxZwbSFNpTNdtMu3d2E3Y0QQn+DFw+KePUHefPfuG1z0NYHA4/3ZpiZFyacaeO6305pbX1jc6u8XdnZ3ds/qB4etXWcKkJ9EvNYdUOsKWeS+oYZTruJoliEnHbCye3M7zxRpVksH02W0EDgkWQRI9hYyb+vS3E+qNbchjsHWiVeQWpQoDWofvWHMUkFlYZwrHXPcxMT5FgZRjidVvqppgkmEzyiPUslFlQH+fzYKTqzyhBFsbIlDZqrvydyLLTORGg7BTZjvezNxP+8Xmqi6yBnMkkNlWSxKEo5MjGafY6GTFFieGYJJorZWxEZY4WJsflUbAje8surpH3R8NyG93BZa94UcZThBE6hDh5cQRPuoAU+EGDwDK/w5kjnxXl3PhatJaeYOYY/cD5/AAWLjic=</latexit><latexit sha1_base64="OMLplQzZdF/InFhsmFeSXzDUATY=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBahXkoigh6LXrxZwbSFNpTNdtMu3d2E3Y0QQn+DFw+KePUHefPfuG1z0NYHA4/3ZpiZFyacaeO6305pbX1jc6u8XdnZ3ds/qB4etXWcKkJ9EvNYdUOsKWeS+oYZTruJoliEnHbCye3M7zxRpVksH02W0EDgkWQRI9hYyb+vS3E+qNbchjsHWiVeQWpQoDWofvWHMUkFlYZwrHXPcxMT5FgZRjidVvqppgkmEzyiPUslFlQH+fzYKTqzyhBFsbIlDZqrvydyLLTORGg7BTZjvezNxP+8Xmqi6yBnMkkNlWSxKEo5MjGafY6GTFFieGYJJorZWxEZY4WJsflUbAje8surpH3R8NyG93BZa94UcZThBE6hDh5cQRPuoAU+EGDwDK/w5kjnxXl3PhatJaeYOYY/cD5/AAWLjic=</latexit><latexit sha1_base64="OMLplQzZdF/InFhsmFeSXzDUATY=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBahXkoigh6LXrxZwbSFNpTNdtMu3d2E3Y0QQn+DFw+KePUHefPfuG1z0NYHA4/3ZpiZFyacaeO6305pbX1jc6u8XdnZ3ds/qB4etXWcKkJ9EvNYdUOsKWeS+oYZTruJoliEnHbCye3M7zxRpVksH02W0EDgkWQRI9hYyb+vS3E+qNbchjsHWiVeQWpQoDWofvWHMUkFlYZwrHXPcxMT5FgZRjidVvqppgkmEzyiPUslFlQH+fzYKTqzyhBFsbIlDZqrvydyLLTORGg7BTZjvezNxP+8Xmqi6yBnMkkNlWSxKEo5MjGafY6GTFFieGYJJorZWxEZY4WJsflUbAje8surpH3R8NyG93BZa94UcZThBE6hDh5cQRPuoAU+EGDwDK/w5kjnxXl3PhatJaeYOYY/cD5/AAWLjic=</latexit>

Computing the decomposition of K: O(n3)<latexit sha1_base64="pUzYwhqmk7Jc05OtuANfZpfZ6hg=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquFfRY9OLNCvYD2rVk02wbm02WJCuUpf/BiwdFvPp/vPlvTNs9aOuDgcd7M8zMC2LOtHHdbye3srq2vpHfLGxt7+zuFfcPmlomitAGkVyqdoA15UzQhmGG03asKI4CTlvB6Hrqt56o0kyKezOOqR/hgWAhI9hYqXlbFg/V016x5FbcGdAy8TJSggz1XvGr25ckiagwhGOtO54bGz/FyjDC6aTQTTSNMRnhAe1YKnBEtZ/Orp2gE6v0USiVLWHQTP09keJI63EU2M4Im6Fe9Kbif14nMeGlnzIRJ4YKMl8UJhwZiaavoz5TlBg+tgQTxeytiAyxwsTYgAo2BG/x5WXSPKt4bsW7Oy/VrrI48nAEx1AGDy6gBjdQhwYQeIRneIU3RzovzrvzMW/NOdnMIfyB8/kDYm6OVQ==</latexit><latexit sha1_base64="pUzYwhqmk7Jc05OtuANfZpfZ6hg=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquFfRY9OLNCvYD2rVk02wbm02WJCuUpf/BiwdFvPp/vPlvTNs9aOuDgcd7M8zMC2LOtHHdbye3srq2vpHfLGxt7+zuFfcPmlomitAGkVyqdoA15UzQhmGG03asKI4CTlvB6Hrqt56o0kyKezOOqR/hgWAhI9hYqXlbFg/V016x5FbcGdAy8TJSggz1XvGr25ckiagwhGOtO54bGz/FyjDC6aTQTTSNMRnhAe1YKnBEtZ/Orp2gE6v0USiVLWHQTP09keJI63EU2M4Im6Fe9Kbif14nMeGlnzIRJ4YKMl8UJhwZiaavoz5TlBg+tgQTxeytiAyxwsTYgAo2BG/x5WXSPKt4bsW7Oy/VrrI48nAEx1AGDy6gBjdQhwYQeIRneIU3RzovzrvzMW/NOdnMIfyB8/kDYm6OVQ==</latexit><latexit sha1_base64="pUzYwhqmk7Jc05OtuANfZpfZ6hg=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquFfRY9OLNCvYD2rVk02wbm02WJCuUpf/BiwdFvPp/vPlvTNs9aOuDgcd7M8zMC2LOtHHdbye3srq2vpHfLGxt7+zuFfcPmlomitAGkVyqdoA15UzQhmGG03asKI4CTlvB6Hrqt56o0kyKezOOqR/hgWAhI9hYqXlbFg/V016x5FbcGdAy8TJSggz1XvGr25ckiagwhGOtO54bGz/FyjDC6aTQTTSNMRnhAe1YKnBEtZ/Orp2gE6v0USiVLWHQTP09keJI63EU2M4Im6Fe9Kbif14nMeGlnzIRJ4YKMl8UJhwZiaavoz5TlBg+tgQTxeytiAyxwsTYgAo2BG/x5WXSPKt4bsW7Oy/VrrI48nAEx1AGDy6gBjdQhwYQeIRneIU3RzovzrvzMW/NOdnMIfyB8/kDYm6OVQ==</latexit><latexit sha1_base64="pUzYwhqmk7Jc05OtuANfZpfZ6hg=">AAAB7XicbVBNSwMxEJ2tX7V+VT16CRahXsquFfRY9OLNCvYD2rVk02wbm02WJCuUpf/BiwdFvPp/vPlvTNs9aOuDgcd7M8zMC2LOtHHdbye3srq2vpHfLGxt7+zuFfcPmlomitAGkVyqdoA15UzQhmGG03asKI4CTlvB6Hrqt56o0kyKezOOqR/hgWAhI9hYqXlbFg/V016x5FbcGdAy8TJSggz1XvGr25ckiagwhGOtO54bGz/FyjDC6aTQTTSNMRnhAe1YKnBEtZ/Orp2gE6v0USiVLWHQTP09keJI63EU2M4Im6Fe9Kbif14nMeGlnzIRJ4YKMl8UJhwZiaavoz5TlBg+tgQTxeytiAyxwsTYgAo2BG/x5WXSPKt4bsW7Oy/VrrI48nAEx1AGDy6gBjdQhwYQeIRneIU3RzovzrvzMW/NOdnMIfyB8/kDYm6OVQ==</latexit>

Page 50: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

How to alleviate this?

Page 51: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

How to alleviate this?Sample points, to obtain a smaller K

Page 52: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

How to alleviate this?Sample points, to obtain a smaller KWilliams and Seeger. "Using the Nyström method to speed up kernel machines." NIPS. 2001.

Page 53: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

How to alleviate this?Sample points, to obtain a smaller KWilliams and Seeger. "Using the Nyström method to speed up kernel machines." NIPS. 2001.

Approximate but explicit embedding of the lifting into Euclidean space, using a random mapping

Page 54: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

How to alleviate this?Sample points, to obtain a smaller KWilliams and Seeger. "Using the Nyström method to speed up kernel machines." NIPS. 2001.

Approximate but explicit embedding of the lifting into Euclidean space, using a random mapping

Rahimi, Ali, and Benjamin Recht. "Random features for large-scale kernel machines." NIPS. 2008.

Page 55: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Randomized feature maps

Z : Rm ! Rd<latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit>

Page 56: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Randomized feature maps

Z : Rm ! Rd<latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit>

For any shift invariant kernel k(x, y) = k(x� y)<latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit>

Page 57: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Randomized feature maps

Z : Rm ! Rd<latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit>

For any shift invariant kernel k(x, y) = k(x� y)<latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit>

E[hZ(x), Z(y)i] = k(x, y)<latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit><latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit><latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit><latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit>

Page 58: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Randomized feature maps

Z : Rm ! Rd<latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit>

For any shift invariant kernel k(x, y) = k(x� y)<latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit>

E[hZ(x), Z(y)i] = k(x, y)<latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit><latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit><latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit><latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit>

if d = O((d/✏2) log(n/�))<latexit sha1_base64="9mKjKharHTvMK3+B8+kPEi+Bmj8=">AAACDHicbVDLSgMxFM3UV62vqks3wSK0m3amCLoRim7cWcE+oFNLJnPbhmaSIckIpfQD3Pgrblwo4tYPcOffmD4W2nogcDjnXG7uCWLOtHHdbye1srq2vpHezGxt7+zuZfcP6lomikKNSi5VMyAaOBNQM8xwaMYKSBRwaASDq4nfeAClmRR3ZhhDOyI9wbqMEmOlTjYX4gt8k8+HJR9izbgU9+WCz2UvL0p+CNyQQsGm3KI7BV4m3pzk0BzVTvbLDyVNIhCGcqJ1y3Nj0x4RZRjlMM74iYaY0AHpQctSQSLQ7dH0mDE+sUqIu1LZJwyeqr8nRiTSehgFNhkR09eL3kT8z2slpnveHjERJwYEnS3qJhwbiSfN4JApoIYPLSFUMftXTPtEEWpsfxlbgrd48jKpl4ueW/RuT3OVy3kdaXSEjlEeeegMVdA1qqIaougRPaNX9OY8OS/Ou/Mxi6ac+cwh+gPn8wdWXZlG</latexit><latexit sha1_base64="9mKjKharHTvMK3+B8+kPEi+Bmj8=">AAACDHicbVDLSgMxFM3UV62vqks3wSK0m3amCLoRim7cWcE+oFNLJnPbhmaSIckIpfQD3Pgrblwo4tYPcOffmD4W2nogcDjnXG7uCWLOtHHdbye1srq2vpHezGxt7+zuZfcP6lomikKNSi5VMyAaOBNQM8xwaMYKSBRwaASDq4nfeAClmRR3ZhhDOyI9wbqMEmOlTjYX4gt8k8+HJR9izbgU9+WCz2UvL0p+CNyQQsGm3KI7BV4m3pzk0BzVTvbLDyVNIhCGcqJ1y3Nj0x4RZRjlMM74iYaY0AHpQctSQSLQ7dH0mDE+sUqIu1LZJwyeqr8nRiTSehgFNhkR09eL3kT8z2slpnveHjERJwYEnS3qJhwbiSfN4JApoIYPLSFUMftXTPtEEWpsfxlbgrd48jKpl4ueW/RuT3OVy3kdaXSEjlEeeegMVdA1qqIaougRPaNX9OY8OS/Ou/Mxi6ac+cwh+gPn8wdWXZlG</latexit><latexit sha1_base64="9mKjKharHTvMK3+B8+kPEi+Bmj8=">AAACDHicbVDLSgMxFM3UV62vqks3wSK0m3amCLoRim7cWcE+oFNLJnPbhmaSIckIpfQD3Pgrblwo4tYPcOffmD4W2nogcDjnXG7uCWLOtHHdbye1srq2vpHezGxt7+zuZfcP6lomikKNSi5VMyAaOBNQM8xwaMYKSBRwaASDq4nfeAClmRR3ZhhDOyI9wbqMEmOlTjYX4gt8k8+HJR9izbgU9+WCz2UvL0p+CNyQQsGm3KI7BV4m3pzk0BzVTvbLDyVNIhCGcqJ1y3Nj0x4RZRjlMM74iYaY0AHpQctSQSLQ7dH0mDE+sUqIu1LZJwyeqr8nRiTSehgFNhkR09eL3kT8z2slpnveHjERJwYEnS3qJhwbiSfN4JApoIYPLSFUMftXTPtEEWpsfxlbgrd48jKpl4ueW/RuT3OVy3kdaXSEjlEeeegMVdA1qqIaougRPaNX9OY8OS/Ou/Mxi6ac+cwh+gPn8wdWXZlG</latexit><latexit sha1_base64="9mKjKharHTvMK3+B8+kPEi+Bmj8=">AAACDHicbVDLSgMxFM3UV62vqks3wSK0m3amCLoRim7cWcE+oFNLJnPbhmaSIckIpfQD3Pgrblwo4tYPcOffmD4W2nogcDjnXG7uCWLOtHHdbye1srq2vpHezGxt7+zuZfcP6lomikKNSi5VMyAaOBNQM8xwaMYKSBRwaASDq4nfeAClmRR3ZhhDOyI9wbqMEmOlTjYX4gt8k8+HJR9izbgU9+WCz2UvL0p+CNyQQsGm3KI7BV4m3pzk0BzVTvbLDyVNIhCGcqJ1y3Nj0x4RZRjlMM74iYaY0AHpQctSQSLQ7dH0mDE+sUqIu1LZJwyeqr8nRiTSehgFNhkR09eL3kT8z2slpnveHjERJwYEnS3qJhwbiSfN4JApoIYPLSFUMftXTPtEEWpsfxlbgrd48jKpl4ueW/RuT3OVy3kdaXSEjlEeeegMVdA1qqIaougRPaNX9OY8OS/Ou/Mxi6ac+cwh+gPn8wdWXZlG</latexit>

P(|hZ(x), Z(y)i � k(x, y)| ✏) � 1� �<latexit sha1_base64="u9x+KqsvsXPk9f3YvTzuGFhZZM4=">AAACNHicbVDLSgMxFM3Ud32NunQTLMIU2jIjgi5FN4KbClZLO6Vk0ts2NJMZk4w4jP0oN36IGxFcKOLWbzB9LHwdCJyccy/JOUHMmdKu+2zlZmbn5hcWl/LLK6tr6/bG5qWKEkmhRiMeyXpAFHAmoKaZ5lCPJZAw4HAVDE5G/tUNSMUicaHTGFoh6QnWZZRoI7XtMz8kuh8EWXXo3PmciB4H3HBui6WGkxaxLydKeeDcltLiHfY5XGMfYsV4JIzfM1ev7HeAa9K2C27FHQP/Jd6UFNAU1bb96HcimoQgNOVEqabnxrqVEakZ5TDM+4mCmNAB6UHTUEFCUK1sHHqId43Swd1ImiM0HqvfNzISKpWGgZkcRVS/vZH4n9dMdPewlTERJxoEnTzUTTjWER41iDtMAtU8NYRQycxfMe0TSag2PedNCd7vyH/J5V7Fcyve+X7h6HhaxyLaRjvIQR46QEfoFFVRDVF0j57QK3qzHqwX6936mIzmrOnOFvoB6/MLGZSpQg==</latexit><latexit sha1_base64="u9x+KqsvsXPk9f3YvTzuGFhZZM4=">AAACNHicbVDLSgMxFM3Ud32NunQTLMIU2jIjgi5FN4KbClZLO6Vk0ts2NJMZk4w4jP0oN36IGxFcKOLWbzB9LHwdCJyccy/JOUHMmdKu+2zlZmbn5hcWl/LLK6tr6/bG5qWKEkmhRiMeyXpAFHAmoKaZ5lCPJZAw4HAVDE5G/tUNSMUicaHTGFoh6QnWZZRoI7XtMz8kuh8EWXXo3PmciB4H3HBui6WGkxaxLydKeeDcltLiHfY5XGMfYsV4JIzfM1ev7HeAa9K2C27FHQP/Jd6UFNAU1bb96HcimoQgNOVEqabnxrqVEakZ5TDM+4mCmNAB6UHTUEFCUK1sHHqId43Swd1ImiM0HqvfNzISKpWGgZkcRVS/vZH4n9dMdPewlTERJxoEnTzUTTjWER41iDtMAtU8NYRQycxfMe0TSag2PedNCd7vyH/J5V7Fcyve+X7h6HhaxyLaRjvIQR46QEfoFFVRDVF0j57QK3qzHqwX6936mIzmrOnOFvoB6/MLGZSpQg==</latexit><latexit sha1_base64="u9x+KqsvsXPk9f3YvTzuGFhZZM4=">AAACNHicbVDLSgMxFM3Ud32NunQTLMIU2jIjgi5FN4KbClZLO6Vk0ts2NJMZk4w4jP0oN36IGxFcKOLWbzB9LHwdCJyccy/JOUHMmdKu+2zlZmbn5hcWl/LLK6tr6/bG5qWKEkmhRiMeyXpAFHAmoKaZ5lCPJZAw4HAVDE5G/tUNSMUicaHTGFoh6QnWZZRoI7XtMz8kuh8EWXXo3PmciB4H3HBui6WGkxaxLydKeeDcltLiHfY5XGMfYsV4JIzfM1ev7HeAa9K2C27FHQP/Jd6UFNAU1bb96HcimoQgNOVEqabnxrqVEakZ5TDM+4mCmNAB6UHTUEFCUK1sHHqId43Swd1ImiM0HqvfNzISKpWGgZkcRVS/vZH4n9dMdPewlTERJxoEnTzUTTjWER41iDtMAtU8NYRQycxfMe0TSag2PedNCd7vyH/J5V7Fcyve+X7h6HhaxyLaRjvIQR46QEfoFFVRDVF0j57QK3qzHqwX6936mIzmrOnOFvoB6/MLGZSpQg==</latexit><latexit sha1_base64="u9x+KqsvsXPk9f3YvTzuGFhZZM4=">AAACNHicbVDLSgMxFM3Ud32NunQTLMIU2jIjgi5FN4KbClZLO6Vk0ts2NJMZk4w4jP0oN36IGxFcKOLWbzB9LHwdCJyccy/JOUHMmdKu+2zlZmbn5hcWl/LLK6tr6/bG5qWKEkmhRiMeyXpAFHAmoKaZ5lCPJZAw4HAVDE5G/tUNSMUicaHTGFoh6QnWZZRoI7XtMz8kuh8EWXXo3PmciB4H3HBui6WGkxaxLydKeeDcltLiHfY5XGMfYsV4JIzfM1ev7HeAa9K2C27FHQP/Jd6UFNAU1bb96HcimoQgNOVEqabnxrqVEakZ5TDM+4mCmNAB6UHTUEFCUK1sHHqId43Swd1ImiM0HqvfNzISKpWGgZkcRVS/vZH4n9dMdPewlTERJxoEnTzUTTjWER41iDtMAtU8NYRQycxfMe0TSag2PedNCd7vyH/J5V7Fcyve+X7h6HhaxyLaRjvIQR46QEfoFFVRDVF0j57QK3qzHqwX6936mIzmrOnOFvoB6/MLGZSpQg==</latexit>

Page 59: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Randomized feature maps

Z : Rm ! Rd<latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit><latexit sha1_base64="qi6XQ9NrPxOGEVnwnOUGEMhmI9M=">AAACCnicbVC7TsMwFHV4lvIKMLIYKiSmKkFIIKYKFsaC6EM0oXIcp7VqO5HtIFVRZxZ+hYUBhFj5Ajb+BqfNUFqOZOnonHvle06QMKq04/xYC4tLyyurpbXy+sbm1ra9s9tUcSoxaeCYxbIdIEUYFaShqWaknUiCeMBIKxhc5X7rkUhFY3GnhwnxOeoJGlGMtJG69sE9vIAeR7ofBNnt6IFDT8fTQti1K07VGQPOE7cgFVCg3rW/vTDGKSdCY4aU6rhOov0MSU0xI6OylyqSIDxAPdIxVCBOlJ+No4zgkVFCGMXSPKHhWJ3eyBBXasgDM5nfqGa9XPzP66Q6OvczKpJUE4EnH0UpgyZt3gsMqSRYs6EhCEtqboW4jyTC2rRXNiW4s5HnSfOk6jpV9+a0Urss6iiBfXAIjoELzkANXIM6aAAMnsALeAPv1rP1an1Yn5PRBavY2QN/YH39Aj5smf0=</latexit>

For any shift invariant kernel k(x, y) = k(x� y)<latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit>

E[hZ(x), Z(y)i] = k(x, y)<latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit><latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit><latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit><latexit sha1_base64="FrbiT7nwDUttDjNNtjHsr+mO8pQ=">AAACDnicbVBNS8NAEN3Ur1q/oh69LJZCCqUkIuhFKIrgsYJtpW0om+22XbrZhN2NNIT+Ai/+FS8eFPHq2Zv/xm2ag7Y+GHi8N8PMPC9kVCrb/jZyK6tr6xv5zcLW9s7unrl/0JRBJDBp4IAF4t5DkjDKSUNRxch9KAjyPUZa3vhq5rceiJA04HcqDonroyGnA4qR0lLPLF13ugzxISOwbU3KlbYVl2FXpIoLL+DYmlTics8s2lU7BVwmTkaKIEO9Z351+wGOfMIVZkjKjmOHyk2QUBQzMi10I0lChMdoSDqacuQT6SbpO1NY0kofDgKhiyuYqr8nEuRLGfue7vSRGslFbyb+53UiNTh3E8rDSBGO54sGEYMqgLNsYJ8KghWLNUFYUH0rxCMkEFY6wYIOwVl8eZk0T6qOXXVuT4u1yyyOPDgCx8ACDjgDNXAD6qABMHgEz+AVvBlPxovxbnzMW3NGNnMI/sD4/AFn15nH</latexit>

instead of implicitly lifting data points to the high-dimensional feature space by the kernel trick, the maps explicitly embed the data to a low-dimensional Euclidean inner product subspace

Page 60: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCA

Page 61: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCAPhase 1: before the data arrives

Page 62: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCAPhase 1: before the data arrives

(f1, . . . , fd)<latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit>

Page 63: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCAPhase 1: before the data arrives

Generate random feature functions (f1, . . . , fd)<latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit>

Page 64: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCAPhase 1: before the data arrives

Generate random feature functions (f1, . . . , fd)<latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit>

When data arrives

Page 65: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCAPhase 1: before the data arrives

Generate random feature functions (f1, . . . , fd)<latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit>

use random feature functions to map each data point to an approximate feature vector

ai 2 Rm<latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit>

zi 2 Rd<latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit>

When data arrives

Page 66: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCAPhase 1: before the data arrives

Generate random feature functions (f1, . . . , fd)<latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit>

use random feature functions to map each data point to an approximate feature vector

ai 2 Rm<latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit>

zi 2 Rd<latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit>

When data arrives

Phase 2: feed feature vector to frequent directions algorithm

Page 67: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCAPhase 1: before the data arrives

Generate random feature functions (f1, . . . , fd)<latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit>

use random feature functions to map each data point to an approximate feature vector

ai 2 Rm<latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit>

zi 2 Rd<latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit>

When data arrives

Phase 2: feed feature vector to frequent directions algorithm

computing an approximate set of singular vectors W 2 Rd⇥`<latexit sha1_base64="0oWmhk7YZkcenrUm6gQs0/P8epA=">AAACCXicbVBNS8NAEN34WetX1KOXxSJ4KokIeix68VjFfkATy2YzaZduNmF3I5TQqxf/ihcPinj1H3jz37hpc9DWBwOP92aYmReknCntON/W0vLK6tp6ZaO6ubW9s2vv7bdVkkkKLZrwRHYDooAzAS3NNIduKoHEAYdOMLoq/M4DSMUScafHKfgxGQgWMUq0kfo27mCPCezFRA+DIL+d3Och9jSLQWEPOJ/07ZpTd6bAi8QtSQ2VaPbtLy9MaBaD0JQTpXquk2o/J1IzymFS9TIFKaEjMoCeoYKYVX4+/WSCj40S4iiRpoTGU/X3RE5ipcZxYDqLi9W8V4j/eb1MRxd+zkSaaRB0tijKONYJLmLBIZNANR8bQqhk5lZMh0QSqk14VROCO//yImmf1l2n7t6c1RqXZRwVdIiO0Aly0TlqoGvURC1E0SN6Rq/ozXqyXqx362PWumSVMwfoD6zPH++Zmdo=</latexit><latexit sha1_base64="0oWmhk7YZkcenrUm6gQs0/P8epA=">AAACCXicbVBNS8NAEN34WetX1KOXxSJ4KokIeix68VjFfkATy2YzaZduNmF3I5TQqxf/ihcPinj1H3jz37hpc9DWBwOP92aYmReknCntON/W0vLK6tp6ZaO6ubW9s2vv7bdVkkkKLZrwRHYDooAzAS3NNIduKoHEAYdOMLoq/M4DSMUScafHKfgxGQgWMUq0kfo27mCPCezFRA+DIL+d3Och9jSLQWEPOJ/07ZpTd6bAi8QtSQ2VaPbtLy9MaBaD0JQTpXquk2o/J1IzymFS9TIFKaEjMoCeoYKYVX4+/WSCj40S4iiRpoTGU/X3RE5ipcZxYDqLi9W8V4j/eb1MRxd+zkSaaRB0tijKONYJLmLBIZNANR8bQqhk5lZMh0QSqk14VROCO//yImmf1l2n7t6c1RqXZRwVdIiO0Aly0TlqoGvURC1E0SN6Rq/ozXqyXqx362PWumSVMwfoD6zPH++Zmdo=</latexit><latexit sha1_base64="0oWmhk7YZkcenrUm6gQs0/P8epA=">AAACCXicbVBNS8NAEN34WetX1KOXxSJ4KokIeix68VjFfkATy2YzaZduNmF3I5TQqxf/ihcPinj1H3jz37hpc9DWBwOP92aYmReknCntON/W0vLK6tp6ZaO6ubW9s2vv7bdVkkkKLZrwRHYDooAzAS3NNIduKoHEAYdOMLoq/M4DSMUScafHKfgxGQgWMUq0kfo27mCPCezFRA+DIL+d3Och9jSLQWEPOJ/07ZpTd6bAi8QtSQ2VaPbtLy9MaBaD0JQTpXquk2o/J1IzymFS9TIFKaEjMoCeoYKYVX4+/WSCj40S4iiRpoTGU/X3RE5ipcZxYDqLi9W8V4j/eb1MRxd+zkSaaRB0tijKONYJLmLBIZNANR8bQqhk5lZMh0QSqk14VROCO//yImmf1l2n7t6c1RqXZRwVdIiO0Aly0TlqoGvURC1E0SN6Rq/ozXqyXqx362PWumSVMwfoD6zPH++Zmdo=</latexit><latexit sha1_base64="0oWmhk7YZkcenrUm6gQs0/P8epA=">AAACCXicbVBNS8NAEN34WetX1KOXxSJ4KokIeix68VjFfkATy2YzaZduNmF3I5TQqxf/ihcPinj1H3jz37hpc9DWBwOP92aYmReknCntON/W0vLK6tp6ZaO6ubW9s2vv7bdVkkkKLZrwRHYDooAzAS3NNIduKoHEAYdOMLoq/M4DSMUScafHKfgxGQgWMUq0kfo27mCPCezFRA+DIL+d3Och9jSLQWEPOJ/07ZpTd6bAi8QtSQ2VaPbtLy9MaBaD0JQTpXquk2o/J1IzymFS9TIFKaEjMoCeoYKYVX4+/WSCj40S4iiRpoTGU/X3RE5ipcZxYDqLi9W8V4j/eb1MRxd+zkSaaRB0tijKONYJLmLBIZNANR8bQqhk5lZMh0QSqk14VROCO//yImmf1l2n7t6c1RqXZRwVdIiO0Aly0TlqoGvURC1E0SN6Rq/ozXqyXqx362PWumSVMwfoD6zPH++Zmdo=</latexit>

Page 68: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCAPhase 1: before the data arrives

Generate random feature functions (f1, . . . , fd)<latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit><latexit sha1_base64="pe05lwAVD7COl5M3hx1R3IyHc8M=">AAAB+HicbVBNS8NAFHypX7V+NOrRy2IRKpSSiKDHohePFWwttCFsNpt26WYTdjdCDf0lXjwo4tWf4s1/47bNQVsHFoaZN7y3E6ScKe0431ZpbX1jc6u8XdnZ3duv2geHXZVkktAOSXgiewFWlDNBO5ppTnuppDgOOH0Ixjcz/+GRSsUSca8nKfViPBQsYgRrI/l2tR75bmMQJlo1Ij888+2a03TmQKvELUgNCrR9+8uESRZToQnHSvVdJ9VejqVmhNNpZZApmmIyxkPaN1TgmCovnx8+RadGCVGUSPOERnP1dyLHsVKTODCTMdYjtezNxP+8fqajKy9nIs00FWSxKMo40gmatYBCJinRfGIIJpKZWxEZYYmJNl1VTAnu8pdXSfe86TpN9+6i1rou6ijDMZxAHVy4hBbcQhs6QCCDZ3iFN+vJerHerY/FaMkqMkfwB9bnDy9Pkh8=</latexit>

use random feature functions to map each data point to an approximate feature vector

ai 2 Rm<latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit><latexit sha1_base64="ELkqXXJkDg/hbur6F3/rKeEyaNs=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+wAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAB0ZTu</latexit>

zi 2 Rd<latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit><latexit sha1_base64="XbJM0smuD2SjMttetBBTYXRaAEA=">AAAB/XicbVDLSsNAFL3xWeurPnZuBovgqiQi6LLoxmUV+4Amhslk0g6dTMLMRGhD8VfcuFDErf/hzr9x2mahrQcGDufcyz1zgpQzpW3721paXlldWy9tlDe3tnd2K3v7LZVkktAmSXgiOwFWlDNBm5ppTjuppDgOOG0Hg+uJ336kUrFE3OthSr0Y9wSLGMHaSH7lcOQz5DKB3BjrfhDkd+OH0K9U7Zo9BVokTkGqUKDhV77cMCFZTIUmHCvVdexUezmWmhFOx2U3UzTFZIB7tGuowDFVXj5NP0YnRglRlEjzhEZT9fdGjmOlhnFgJicZ1bw3Ef/zupmOLr2ciTTTVJDZoSjjSCdoUgUKmaRE86EhmEhmsiLSxxITbQormxKc+S8vktZZzbFrzu15tX5V1FGCIziGU3DgAupwAw1oAoERPMMrvFlP1ov1bn3MRpesYucA/sD6/AEb7JT+</latexit>

When data arrives

Phase 2: feed feature vector to frequent directions algorithm

computing an approximate set of singular vectors W 2 Rd⇥`<latexit sha1_base64="0oWmhk7YZkcenrUm6gQs0/P8epA=">AAACCXicbVBNS8NAEN34WetX1KOXxSJ4KokIeix68VjFfkATy2YzaZduNmF3I5TQqxf/ihcPinj1H3jz37hpc9DWBwOP92aYmReknCntON/W0vLK6tp6ZaO6ubW9s2vv7bdVkkkKLZrwRHYDooAzAS3NNIduKoHEAYdOMLoq/M4DSMUScafHKfgxGQgWMUq0kfo27mCPCezFRA+DIL+d3Och9jSLQWEPOJ/07ZpTd6bAi8QtSQ2VaPbtLy9MaBaD0JQTpXquk2o/J1IzymFS9TIFKaEjMoCeoYKYVX4+/WSCj40S4iiRpoTGU/X3RE5ipcZxYDqLi9W8V4j/eb1MRxd+zkSaaRB0tijKONYJLmLBIZNANR8bQqhk5lZMh0QSqk14VROCO//yImmf1l2n7t6c1RqXZRwVdIiO0Aly0TlqoGvURC1E0SN6Rq/ozXqyXqx362PWumSVMwfoD6zPH++Zmdo=</latexit><latexit sha1_base64="0oWmhk7YZkcenrUm6gQs0/P8epA=">AAACCXicbVBNS8NAEN34WetX1KOXxSJ4KokIeix68VjFfkATy2YzaZduNmF3I5TQqxf/ihcPinj1H3jz37hpc9DWBwOP92aYmReknCntON/W0vLK6tp6ZaO6ubW9s2vv7bdVkkkKLZrwRHYDooAzAS3NNIduKoHEAYdOMLoq/M4DSMUScafHKfgxGQgWMUq0kfo27mCPCezFRA+DIL+d3Och9jSLQWEPOJ/07ZpTd6bAi8QtSQ2VaPbtLy9MaBaD0JQTpXquk2o/J1IzymFS9TIFKaEjMoCeoYKYVX4+/WSCj40S4iiRpoTGU/X3RE5ipcZxYDqLi9W8V4j/eb1MRxd+zkSaaRB0tijKONYJLmLBIZNANR8bQqhk5lZMh0QSqk14VROCO//yImmf1l2n7t6c1RqXZRwVdIiO0Aly0TlqoGvURC1E0SN6Rq/ozXqyXqx362PWumSVMwfoD6zPH++Zmdo=</latexit><latexit sha1_base64="0oWmhk7YZkcenrUm6gQs0/P8epA=">AAACCXicbVBNS8NAEN34WetX1KOXxSJ4KokIeix68VjFfkATy2YzaZduNmF3I5TQqxf/ihcPinj1H3jz37hpc9DWBwOP92aYmReknCntON/W0vLK6tp6ZaO6ubW9s2vv7bdVkkkKLZrwRHYDooAzAS3NNIduKoHEAYdOMLoq/M4DSMUScafHKfgxGQgWMUq0kfo27mCPCezFRA+DIL+d3Och9jSLQWEPOJ/07ZpTd6bAi8QtSQ2VaPbtLy9MaBaD0JQTpXquk2o/J1IzymFS9TIFKaEjMoCeoYKYVX4+/WSCj40S4iiRpoTGU/X3RE5ipcZxYDqLi9W8V4j/eb1MRxd+zkSaaRB0tijKONYJLmLBIZNANR8bQqhk5lZMh0QSqk14VROCO//yImmf1l2n7t6c1RqXZRwVdIiO0Aly0TlqoGvURC1E0SN6Rq/ozXqyXqx362PWumSVMwfoD6zPH++Zmdo=</latexit><latexit sha1_base64="0oWmhk7YZkcenrUm6gQs0/P8epA=">AAACCXicbVBNS8NAEN34WetX1KOXxSJ4KokIeix68VjFfkATy2YzaZduNmF3I5TQqxf/ihcPinj1H3jz37hpc9DWBwOP92aYmReknCntON/W0vLK6tp6ZaO6ubW9s2vv7bdVkkkKLZrwRHYDooAzAS3NNIduKoHEAYdOMLoq/M4DSMUScafHKfgxGQgWMUq0kfo27mCPCezFRA+DIL+d3Och9jSLQWEPOJ/07ZpTd6bAi8QtSQ2VaPbtLy9MaBaD0JQTpXquk2o/J1IzymFS9TIFKaEjMoCeoYKYVX4+/WSCj40S4iiRpoTGU/X3RE5ipcZxYDqLi9W8V4j/eb1MRxd+zkSaaRB0tijKONYJLmLBIZNANR8bQqhk5lZMh0QSqk14VROCO//yImmf1l2n7t6c1RqXZRwVdIiO0Aly0TlqoGvURC1E0SN6Rq/ozXqyXqx362PWumSVMwfoD6zPH++Zmdo=</latexit>

Ghashami, Perry, Phillips, Streaming kernel PCA, AISTATS, 2016

Page 69: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Streaming kPCA

<̀latexit sha1_base64="NiCNFQINCFG5T4iOY+JPxVaf5XA=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJFmyu3fs7gnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelAhurO9/e6W19Y3NrfJ2ZWd3b/+genjUNnGqGbZYLGLdiahBwRW2LLcCO4lGKiOBj9HkNvcfn1AbHqsHO00wlHSk+JAzanOph0L0qzW/7s9BVklQkBoUaParX71BzFKJyjJBjekGfmLDjGrLmcBZpZcaTCib0BF2HVVUogmz+a0zcuaUARnG2pWyZK7+nsioNGYqI9cpqR2bZS8X//O6qR1ehxlXSWpRscWiYSqIjUn+OBlwjcyKqSOUae5uJWxMNWXWxVNxIQTLL6+S9kU98OvB/WWtcVPEUYYTOIVzCOAKGnAHTWgBgzE8wyu8edJ78d69j0VryStmjuEPvM8fDWaOOw==</latexit><latexit sha1_base64="NiCNFQINCFG5T4iOY+JPxVaf5XA=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJFmyu3fs7gnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelAhurO9/e6W19Y3NrfJ2ZWd3b/+genjUNnGqGbZYLGLdiahBwRW2LLcCO4lGKiOBj9HkNvcfn1AbHqsHO00wlHSk+JAzanOph0L0qzW/7s9BVklQkBoUaParX71BzFKJyjJBjekGfmLDjGrLmcBZpZcaTCib0BF2HVVUogmz+a0zcuaUARnG2pWyZK7+nsioNGYqI9cpqR2bZS8X//O6qR1ehxlXSWpRscWiYSqIjUn+OBlwjcyKqSOUae5uJWxMNWXWxVNxIQTLL6+S9kU98OvB/WWtcVPEUYYTOIVzCOAKGnAHTWgBgzE8wyu8edJ78d69j0VryStmjuEPvM8fDWaOOw==</latexit><latexit sha1_base64="NiCNFQINCFG5T4iOY+JPxVaf5XA=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJFmyu3fs7gnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelAhurO9/e6W19Y3NrfJ2ZWd3b/+genjUNnGqGbZYLGLdiahBwRW2LLcCO4lGKiOBj9HkNvcfn1AbHqsHO00wlHSk+JAzanOph0L0qzW/7s9BVklQkBoUaParX71BzFKJyjJBjekGfmLDjGrLmcBZpZcaTCib0BF2HVVUogmz+a0zcuaUARnG2pWyZK7+nsioNGYqI9cpqR2bZS8X//O6qR1ehxlXSWpRscWiYSqIjUn+OBlwjcyKqSOUae5uJWxMNWXWxVNxIQTLL6+S9kU98OvB/WWtcVPEUYYTOIVzCOAKGnAHTWgBgzE8wyu8edJ78d69j0VryStmjuEPvM8fDWaOOw==</latexit><latexit sha1_base64="NiCNFQINCFG5T4iOY+JPxVaf5XA=">AAAB63icbVA9SwNBEJ2LXzF+RS1tFoNgFe5E0DJoYxnBxEByhL3NJFmyu3fs7gnhyF+wsVDE1j9k579xL7lCEx8MPN6bYWZelAhurO9/e6W19Y3NrfJ2ZWd3b/+genjUNnGqGbZYLGLdiahBwRW2LLcCO4lGKiOBj9HkNvcfn1AbHqsHO00wlHSk+JAzanOph0L0qzW/7s9BVklQkBoUaParX71BzFKJyjJBjekGfmLDjGrLmcBZpZcaTCib0BF2HVVUogmz+a0zcuaUARnG2pWyZK7+nsioNGYqI9cpqR2bZS8X//O6qR1ehxlXSWpRscWiYSqIjUn+OBlwjcyKqSOUae5uJWxMNWXWxVNxIQTLL6+S9kU98OvB/WWtcVPEUYYTOIVzCOAKGnAHTWgBgzE8wyu8edJ78d69j0VryStmjuEPvM8fDWaOOw==</latexit>

-dimensional row space of W encodes a lower dimensional subspace

` ⌧ d<latexit sha1_base64="yhlXyNXtBQG5T9/rHfAOj+a9hw4=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gObUDabSbt0swm7G6GU/gsvHhTx6r/x5r9x2+agrQ8GHu/NMDMvzATXxnW/ndLa+sbmVnm7srO7t39QPTxq6zRXDFssFanqhlSj4BJbhhuB3UwhTUKBnXB0O/M7T6g0T+WDGWcYJHQgecwZNVZ69FEI4tuK+tWaW3fnIKvEK0gNCjT71S8/SlmeoDRMUK17npuZYEKV4UzgtOLnGjPKRnSAPUslTVAHk/nFU3JmlYjEqbIlDZmrvycmNNF6nIS2M6FmqJe9mfif18tNfB1MuMxyg5ItFsW5ICYls/dJxBUyI8aWUKa4vZWwIVWUGRtSxYbgLb+8StoXdc+te/eXtcZNEUcZTuAUzsGDK2jAHTShBQwkPMMrvDnaeXHenY9Fa8kpZo7hD5zPH8RvkE8=</latexit><latexit sha1_base64="yhlXyNXtBQG5T9/rHfAOj+a9hw4=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gObUDabSbt0swm7G6GU/gsvHhTx6r/x5r9x2+agrQ8GHu/NMDMvzATXxnW/ndLa+sbmVnm7srO7t39QPTxq6zRXDFssFanqhlSj4BJbhhuB3UwhTUKBnXB0O/M7T6g0T+WDGWcYJHQgecwZNVZ69FEI4tuK+tWaW3fnIKvEK0gNCjT71S8/SlmeoDRMUK17npuZYEKV4UzgtOLnGjPKRnSAPUslTVAHk/nFU3JmlYjEqbIlDZmrvycmNNF6nIS2M6FmqJe9mfif18tNfB1MuMxyg5ItFsW5ICYls/dJxBUyI8aWUKa4vZWwIVWUGRtSxYbgLb+8StoXdc+te/eXtcZNEUcZTuAUzsGDK2jAHTShBQwkPMMrvDnaeXHenY9Fa8kpZo7hD5zPH8RvkE8=</latexit><latexit sha1_base64="yhlXyNXtBQG5T9/rHfAOj+a9hw4=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gObUDabSbt0swm7G6GU/gsvHhTx6r/x5r9x2+agrQ8GHu/NMDMvzATXxnW/ndLa+sbmVnm7srO7t39QPTxq6zRXDFssFanqhlSj4BJbhhuB3UwhTUKBnXB0O/M7T6g0T+WDGWcYJHQgecwZNVZ69FEI4tuK+tWaW3fnIKvEK0gNCjT71S8/SlmeoDRMUK17npuZYEKV4UzgtOLnGjPKRnSAPUslTVAHk/nFU3JmlYjEqbIlDZmrvycmNNF6nIS2M6FmqJe9mfif18tNfB1MuMxyg5ItFsW5ICYls/dJxBUyI8aWUKa4vZWwIVWUGRtSxYbgLb+8StoXdc+te/eXtcZNEUcZTuAUzsGDK2jAHTShBQwkPMMrvDnaeXHenY9Fa8kpZo7hD5zPH8RvkE8=</latexit><latexit sha1_base64="yhlXyNXtBQG5T9/rHfAOj+a9hw4=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gObUDabSbt0swm7G6GU/gsvHhTx6r/x5r9x2+agrQ8GHu/NMDMvzATXxnW/ndLa+sbmVnm7srO7t39QPTxq6zRXDFssFanqhlSj4BJbhhuB3UwhTUKBnXB0O/M7T6g0T+WDGWcYJHQgecwZNVZ69FEI4tuK+tWaW3fnIKvEK0gNCjT71S8/SlmeoDRMUK17npuZYEKV4UzgtOLnGjPKRnSAPUslTVAHk/nFU3JmlYjEqbIlDZmrvycmNNF6nIS2M6FmqJe9mfif18tNfB1MuMxyg5ItFsW5ICYls/dJxBUyI8aWUKa4vZWwIVWUGRtSxYbgLb+8StoXdc+te/eXtcZNEUcZTuAUzsGDK2jAHTShBQwkPMMrvDnaeXHenY9Fa8kpZo7hD5zPH8RvkE8=</latexit>

Because

Page 70: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Random Fourier feature maps

For any positive definite shift invariant kernel k(x, y) = k(x� y)<latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit>

Page 71: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Random Fourier feature maps

For any positive definite shift invariant kernel k(x, y) = k(x� y)<latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit>

Generate d functions fi(x) = cos(rTi x+ �i)<latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit><latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit><latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit>

Page 72: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Random Fourier feature maps

For any positive definite shift invariant kernel k(x, y) = k(x� y)<latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit>

Generate d functions fi(x) = cos(rTi x+ �i)<latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit><latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit><latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit>

ri 2 Rm<latexit sha1_base64="z/mciukMIoSSxNZrEekMqSQmfjM=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+kAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAc2JT/</latexit><latexit sha1_base64="z/mciukMIoSSxNZrEekMqSQmfjM=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+kAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAc2JT/</latexit><latexit sha1_base64="z/mciukMIoSSxNZrEekMqSQmfjM=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+kAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAc2JT/</latexit>

where is a uniform sample of the Fourier transform of k

Page 73: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Random Fourier feature maps

For any positive definite shift invariant kernel k(x, y) = k(x� y)<latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit>

Generate d functions fi(x) = cos(rTi x+ �i)<latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit><latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit><latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit>

ri 2 Rm<latexit sha1_base64="z/mciukMIoSSxNZrEekMqSQmfjM=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+kAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAc2JT/</latexit><latexit sha1_base64="z/mciukMIoSSxNZrEekMqSQmfjM=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+kAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAc2JT/</latexit><latexit sha1_base64="z/mciukMIoSSxNZrEekMqSQmfjM=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+kAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAc2JT/</latexit>

where is a uniform sample of the Fourier transform of k

�i ⇠ Unif(0, 2⇡]<latexit sha1_base64="0nDN44wl9Zv5x1bxp+eNbVWj77o=">AAACDHicbVDLSsNAFJ3UV62vqks3g0XoQkpSBF0W3bisYNpCE8rNdNIOnUnCzEQooR/gxl9x40IRt36AO//GSZuFth4YOJxzLnPvCRLOlLbtb6u0tr6xuVXeruzs7u0fVA+POipOJaEuiXksewEoyllEXc00p71EUhABp91gcpP73QcqFYujez1NqC9gFLGQEdBGGlRr3giEgAHDnmICewL0WIrMNZlZ3T5vegnzTcpu2HPgVeIUpIYKtAfVL28Yk1TQSBMOSvUdO9F+BlIzwums4qWKJkAmMKJ9QyMQVPnZ/JgZPjPKEIexNC/SeK7+nshAKDUVgUnmy6plLxf/8/qpDq/8jEVJqmlEFh+FKcc6xnkzeMgkJZpPDQEimdkVkzFIINr0VzElOMsnr5JOs+HYDefuota6LuoooxN0iurIQZeohW5RG7mIoEf0jF7Rm/VkvVjv1sciWrKKmWP0B9bnD8xbmsw=</latexit><latexit sha1_base64="0nDN44wl9Zv5x1bxp+eNbVWj77o=">AAACDHicbVDLSsNAFJ3UV62vqks3g0XoQkpSBF0W3bisYNpCE8rNdNIOnUnCzEQooR/gxl9x40IRt36AO//GSZuFth4YOJxzLnPvCRLOlLbtb6u0tr6xuVXeruzs7u0fVA+POipOJaEuiXksewEoyllEXc00p71EUhABp91gcpP73QcqFYujez1NqC9gFLGQEdBGGlRr3giEgAHDnmICewL0WIrMNZlZ3T5vegnzTcpu2HPgVeIUpIYKtAfVL28Yk1TQSBMOSvUdO9F+BlIzwums4qWKJkAmMKJ9QyMQVPnZ/JgZPjPKEIexNC/SeK7+nshAKDUVgUnmy6plLxf/8/qpDq/8jEVJqmlEFh+FKcc6xnkzeMgkJZpPDQEimdkVkzFIINr0VzElOMsnr5JOs+HYDefuota6LuoooxN0iurIQZeohW5RG7mIoEf0jF7Rm/VkvVjv1sciWrKKmWP0B9bnD8xbmsw=</latexit><latexit sha1_base64="0nDN44wl9Zv5x1bxp+eNbVWj77o=">AAACDHicbVDLSsNAFJ3UV62vqks3g0XoQkpSBF0W3bisYNpCE8rNdNIOnUnCzEQooR/gxl9x40IRt36AO//GSZuFth4YOJxzLnPvCRLOlLbtb6u0tr6xuVXeruzs7u0fVA+POipOJaEuiXksewEoyllEXc00p71EUhABp91gcpP73QcqFYujez1NqC9gFLGQEdBGGlRr3giEgAHDnmICewL0WIrMNZlZ3T5vegnzTcpu2HPgVeIUpIYKtAfVL28Yk1TQSBMOSvUdO9F+BlIzwums4qWKJkAmMKJ9QyMQVPnZ/JgZPjPKEIexNC/SeK7+nshAKDUVgUnmy6plLxf/8/qpDq/8jEVJqmlEFh+FKcc6xnkzeMgkJZpPDQEimdkVkzFIINr0VzElOMsnr5JOs+HYDefuota6LuoooxN0iurIQZeohW5RG7mIoEf0jF7Rm/VkvVjv1sciWrKKmWP0B9bnD8xbmsw=</latexit>

Page 74: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Random Fourier feature maps

For any positive definite shift invariant kernel k(x, y) = k(x� y)<latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit><latexit sha1_base64="g08mJemHIbiqQbYUE3mosN/i58I=">AAAB+HicbZDLSsNAFIZP6q3WS6Mu3QwWoQUtiQi6EYpuXFawF2hDmUyn7dDJJMxMxBj6JG5cKOLWR3Hn2zhts9DWHwY+/nMO58zvR5wp7TjfVm5ldW19I79Z2Nre2S3ae/tNFcaS0AYJeSjbPlaUM0EbmmlO25GkOPA5bfnjm2m99UClYqG410lEvQAPBRswgrWxenZxXH48SSroChk4TSo9u+RUnZnQMrgZlCBTvWd/dfshiQMqNOFYqY7rRNpLsdSMcDopdGNFI0zGeEg7BgUOqPLS2eETdGycPhqE0jyh0cz9PZHiQKkk8E1ngPVILdam5n+1TqwHl17KRBRrKsh80SDmSIdomgLqM0mJ5okBTCQztyIywhITbbIqmBDcxS8vQ/Os6jpV9+68VLvO4sjDIRxBGVy4gBrcQh0aQCCGZ3iFN+vJerHerY95a87KZg7gj6zPHyRxkXE=</latexit>

Generate d functions fi(x) = cos(rTi x+ �i)<latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit><latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit><latexit sha1_base64="uYCi7FOd4Sb0X9t5OdlR9mDMwV4=">AAACCXicbVDLSgMxFM3UV62vUZdugkVoEcqMCLoRim5cVugLOuOQSTNtaJIZkoy0lG7d+CtuXCji1j9w59+YtrPQ1gMXDufcy733hAmjSjvOt5VbWV1b38hvFra2d3b37P2DpopTiUkDxyyW7RApwqggDU01I+1EEsRDRlrh4Gbqtx6IVDQWdT1KiM9RT9CIYqSNFNgwCmhpWIZX0MOxKsmA3teH8BR6PcQ5Cmg5sItOxZkBLhM3I0WQoRbYX143xiknQmOGlOq4TqL9MZKaYkYmBS9VJEF4gHqkY6hAnCh/PPtkAk+M0oVRLE0JDWfq74kx4kqNeGg6OdJ9tehNxf+8TqqjS39MRZJqIvB8UZQyqGM4jQV2qSRYs5EhCEtqboW4jyTC2oRXMCG4iy8vk+ZZxXUq7t15sXqdxZEHR+AYlIALLkAV3IIaaAAMHsEzeAVv1pP1Yr1bH/PWnJXNHII/sD5/AHdSmEc=</latexit>

ri 2 Rm<latexit sha1_base64="z/mciukMIoSSxNZrEekMqSQmfjM=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+kAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAc2JT/</latexit><latexit sha1_base64="z/mciukMIoSSxNZrEekMqSQmfjM=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+kAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAc2JT/</latexit><latexit sha1_base64="z/mciukMIoSSxNZrEekMqSQmfjM=">AAAB/XicbVDLSsNAFL2pr1pf8bFzM1gEVyURQZdFNy6r2Ac0MUymk3boZBJmJkINxV9x40IRt/6HO//GSduFth4YOJxzL/fMCVPOlHacb6u0tLyyulZer2xsbm3v2Lt7LZVkktAmSXgiOyFWlDNBm5ppTjuppDgOOW2Hw6vCbz9QqVgi7vQopX6M+4JFjGBtpMA+kAFDHhPIi7EehGF+O76PA7vq1JwJ0CJxZ6QKMzQC+8vrJSSLqdCEY6W6rpNqP8dSM8LpuOJliqaYDHGfdg0VOKbKzyfpx+jYKD0UJdI8odFE/b2R41ipURyaySKjmvcK8T+vm+nows+ZSDNNBZkeijKOdIKKKlCPSUo0HxmCiWQmKyIDLDHRprCKKcGd//IiaZ3WXKfm3pxV65ezOspwCEdwAi6cQx2uoQFNIPAIz/AKb9aT9WK9Wx/T0ZI129mHP7A+fwAc2JT/</latexit>

where is a uniform sample of the Fourier transform of k

�i ⇠ Unif(0, 2⇡]<latexit sha1_base64="0nDN44wl9Zv5x1bxp+eNbVWj77o=">AAACDHicbVDLSsNAFJ3UV62vqks3g0XoQkpSBF0W3bisYNpCE8rNdNIOnUnCzEQooR/gxl9x40IRt36AO//GSZuFth4YOJxzLnPvCRLOlLbtb6u0tr6xuVXeruzs7u0fVA+POipOJaEuiXksewEoyllEXc00p71EUhABp91gcpP73QcqFYujez1NqC9gFLGQEdBGGlRr3giEgAHDnmICewL0WIrMNZlZ3T5vegnzTcpu2HPgVeIUpIYKtAfVL28Yk1TQSBMOSvUdO9F+BlIzwums4qWKJkAmMKJ9QyMQVPnZ/JgZPjPKEIexNC/SeK7+nshAKDUVgUnmy6plLxf/8/qpDq/8jEVJqmlEFh+FKcc6xnkzeMgkJZpPDQEimdkVkzFIINr0VzElOMsnr5JOs+HYDefuota6LuoooxN0iurIQZeohW5RG7mIoEf0jF7Rm/VkvVjv1sciWrKKmWP0B9bnD8xbmsw=</latexit><latexit sha1_base64="0nDN44wl9Zv5x1bxp+eNbVWj77o=">AAACDHicbVDLSsNAFJ3UV62vqks3g0XoQkpSBF0W3bisYNpCE8rNdNIOnUnCzEQooR/gxl9x40IRt36AO//GSZuFth4YOJxzLnPvCRLOlLbtb6u0tr6xuVXeruzs7u0fVA+POipOJaEuiXksewEoyllEXc00p71EUhABp91gcpP73QcqFYujez1NqC9gFLGQEdBGGlRr3giEgAHDnmICewL0WIrMNZlZ3T5vegnzTcpu2HPgVeIUpIYKtAfVL28Yk1TQSBMOSvUdO9F+BlIzwums4qWKJkAmMKJ9QyMQVPnZ/JgZPjPKEIexNC/SeK7+nshAKDUVgUnmy6plLxf/8/qpDq/8jEVJqmlEFh+FKcc6xnkzeMgkJZpPDQEimdkVkzFIINr0VzElOMsnr5JOs+HYDefuota6LuoooxN0iurIQZeohW5RG7mIoEf0jF7Rm/VkvVjv1sciWrKKmWP0B9bnD8xbmsw=</latexit><latexit sha1_base64="0nDN44wl9Zv5x1bxp+eNbVWj77o=">AAACDHicbVDLSsNAFJ3UV62vqks3g0XoQkpSBF0W3bisYNpCE8rNdNIOnUnCzEQooR/gxl9x40IRt36AO//GSZuFth4YOJxzLnPvCRLOlLbtb6u0tr6xuVXeruzs7u0fVA+POipOJaEuiXksewEoyllEXc00p71EUhABp91gcpP73QcqFYujez1NqC9gFLGQEdBGGlRr3giEgAHDnmICewL0WIrMNZlZ3T5vegnzTcpu2HPgVeIUpIYKtAfVL28Yk1TQSBMOSvUdO9F+BlIzwums4qWKJkAmMKJ9QyMQVPnZ/JgZPjPKEIexNC/SeK7+nshAKDUVgUnmy6plLxf/8/qpDq/8jEVJqmlEFh+FKcc6xnkzeMgkJZpPDQEimdkVkzFIINr0VzElOMsnr5JOs+HYDefuota6LuoooxN0iurIQZeohW5RG7mIoEf0jF7Rm/VkvVjv1sciWrKKmWP0B9bnD8xbmsw=</latexit>

z(x)i =p

2/dfi(x)<latexit sha1_base64="Ujq2DjGM/mH4pAC7xXQNhiY2OYA=">AAACA3icbVDLSsNAFJ3UV62vqDvdDBahbmpSBN0IRTcuK9gHtCFMJpN26GQSZyZiDQU3/oobF4q49Sfc+TdO2iy09cCFwzn3cu89XsyoVJb1bRQWFpeWV4qrpbX1jc0tc3unJaNEYNLEEYtEx0OSMMpJU1HFSCcWBIUeI21veJn57TsiJI34jRrFxAlRn9OAYqS05Jp7D5X7I5fCc9iTt0KltWN/DAOXZqpZtqrWBHCe2DkpgxwN1/zq+RFOQsIVZkjKrm3FykmRUBQzMi71EklihIeoT7qachQS6aSTH8bwUCs+DCKhiys4UX9PpCiUchR6ujNEaiBnvUz8z+smKjhzUsrjRBGOp4uChEEVwSwQ6FNBsGIjTRAWVN8K8QAJhJWOraRDsGdfnietWtW2qvb1Sbl+kcdRBPvgAFSADU5BHVyBBmgCDB7BM3gFb8aT8WK8Gx/T1oKRz+yCPzA+fwCWWJYx</latexit><latexit sha1_base64="Ujq2DjGM/mH4pAC7xXQNhiY2OYA=">AAACA3icbVDLSsNAFJ3UV62vqDvdDBahbmpSBN0IRTcuK9gHtCFMJpN26GQSZyZiDQU3/oobF4q49Sfc+TdO2iy09cCFwzn3cu89XsyoVJb1bRQWFpeWV4qrpbX1jc0tc3unJaNEYNLEEYtEx0OSMMpJU1HFSCcWBIUeI21veJn57TsiJI34jRrFxAlRn9OAYqS05Jp7D5X7I5fCc9iTt0KltWN/DAOXZqpZtqrWBHCe2DkpgxwN1/zq+RFOQsIVZkjKrm3FykmRUBQzMi71EklihIeoT7qachQS6aSTH8bwUCs+DCKhiys4UX9PpCiUchR6ujNEaiBnvUz8z+smKjhzUsrjRBGOp4uChEEVwSwQ6FNBsGIjTRAWVN8K8QAJhJWOraRDsGdfnietWtW2qvb1Sbl+kcdRBPvgAFSADU5BHVyBBmgCDB7BM3gFb8aT8WK8Gx/T1oKRz+yCPzA+fwCWWJYx</latexit><latexit sha1_base64="Ujq2DjGM/mH4pAC7xXQNhiY2OYA=">AAACA3icbVDLSsNAFJ3UV62vqDvdDBahbmpSBN0IRTcuK9gHtCFMJpN26GQSZyZiDQU3/oobF4q49Sfc+TdO2iy09cCFwzn3cu89XsyoVJb1bRQWFpeWV4qrpbX1jc0tc3unJaNEYNLEEYtEx0OSMMpJU1HFSCcWBIUeI21veJn57TsiJI34jRrFxAlRn9OAYqS05Jp7D5X7I5fCc9iTt0KltWN/DAOXZqpZtqrWBHCe2DkpgxwN1/zq+RFOQsIVZkjKrm3FykmRUBQzMi71EklihIeoT7qachQS6aSTH8bwUCs+DCKhiys4UX9PpCiUchR6ujNEaiBnvUz8z+smKjhzUsrjRBGOp4uChEEVwSwQ6FNBsGIjTRAWVN8K8QAJhJWOraRDsGdfnietWtW2qvb1Sbl+kcdRBPvgAFSADU5BHVyBBmgCDB7BM3gFb8aT8WK8Gx/T1oKRz+yCPzA+fwCWWJYx</latexit>

Page 75: Processing Big Datausers.isr.tecnico.ulisboa.pt/~csoares/BigDataDocs/kPCA.pdf · Example Let’s go 3D: R2!R3 AAACDXicbVC7SgNBFL0bXzG+Vi1tBqNgFXajoFgFbSyjmAdk1zA7mU2GzD6YmRXCkh

Complexityd = O((1/✏2) log n)

<latexit sha1_base64="A5xv37IZYuA5qFaNh5fDEtMinEQ=">AAACBXicbVC7SgNBFJ2Nrxhfq5ZaDAYhaeJuELQRgjZ2RjAPyK5hdnY2GTI7s8zMCmFJY+Ov2FgoYus/2Pk3Th6FJh64cDjnXu69J0gYVdpxvq3c0vLK6lp+vbCxubW9Y+/uNZVIJSYNLJiQ7QApwignDU01I+1EEhQHjLSCwdXYbz0Qqajgd3qYED9GPU4jipE2Utc+DOEFvCmV3BOPJIoywe+rZegx0YO83LWLTsWZAC4Sd0aKYIZ61/7yQoHTmHCNGVKq4zqJ9jMkNcWMjApeqkiC8AD1SMdQjmKi/GzyxQgeGyWEkZCmuIYT9fdEhmKlhnFgOmOk+2reG4v/eZ1UR+d+RnmSasLxdFGUMqgFHEcCQyoJ1mxoCMKSmlsh7iOJsDbBFUwI7vzLi6RZrbhOxb09LdYuZ3HkwQE4AiXggjNQA9egDhoAg0fwDF7Bm/VkvVjv1se0NWfNZvbBH1ifP7Rolic=</latexit><latexit sha1_base64="A5xv37IZYuA5qFaNh5fDEtMinEQ=">AAACBXicbVC7SgNBFJ2Nrxhfq5ZaDAYhaeJuELQRgjZ2RjAPyK5hdnY2GTI7s8zMCmFJY+Ov2FgoYus/2Pk3Th6FJh64cDjnXu69J0gYVdpxvq3c0vLK6lp+vbCxubW9Y+/uNZVIJSYNLJiQ7QApwignDU01I+1EEhQHjLSCwdXYbz0Qqajgd3qYED9GPU4jipE2Utc+DOEFvCmV3BOPJIoywe+rZegx0YO83LWLTsWZAC4Sd0aKYIZ61/7yQoHTmHCNGVKq4zqJ9jMkNcWMjApeqkiC8AD1SMdQjmKi/GzyxQgeGyWEkZCmuIYT9fdEhmKlhnFgOmOk+2reG4v/eZ1UR+d+RnmSasLxdFGUMqgFHEcCQyoJ1mxoCMKSmlsh7iOJsDbBFUwI7vzLi6RZrbhOxb09LdYuZ3HkwQE4AiXggjNQA9egDhoAg0fwDF7Bm/VkvVjv1se0NWfNZvbBH1ifP7Rolic=</latexit><latexit sha1_base64="A5xv37IZYuA5qFaNh5fDEtMinEQ=">AAACBXicbVC7SgNBFJ2Nrxhfq5ZaDAYhaeJuELQRgjZ2RjAPyK5hdnY2GTI7s8zMCmFJY+Ov2FgoYus/2Pk3Th6FJh64cDjnXu69J0gYVdpxvq3c0vLK6lp+vbCxubW9Y+/uNZVIJSYNLJiQ7QApwignDU01I+1EEhQHjLSCwdXYbz0Qqajgd3qYED9GPU4jipE2Utc+DOEFvCmV3BOPJIoywe+rZegx0YO83LWLTsWZAC4Sd0aKYIZ61/7yQoHTmHCNGVKq4zqJ9jMkNcWMjApeqkiC8AD1SMdQjmKi/GzyxQgeGyWEkZCmuIYT9fdEhmKlhnFgOmOk+2reG4v/eZ1UR+d+RnmSasLxdFGUMqgFHEcCQyoJ1mxoCMKSmlsh7iOJsDbBFUwI7vzLi6RZrbhOxb09LdYuZ3HkwQE4AiXggjNQA9egDhoAg0fwDF7Bm/VkvVjv1se0NWfNZvbBH1ifP7Rolic=</latexit>

` = O(1/✏)<latexit sha1_base64="W5BzBzpftMOnL9zHHfACZCqQyas=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5GSxC3dREBN0IRTfurGAf0IQymd60QyeTMDMRaij+ihsXirj1P9z5N07bLLR64MLhnHu5954g4Uxpx/myCguLS8srxdXS2vrG5pa9vdNUcSopNGjMY9kOiALOBDQ00xzaiQQSBRxawfBq4rfuQSoWizs9SsCPSF+wkFGijdS19zzgHF/gm4p77EGiGI/FUdcuO1VnCvyXuDkpoxz1rv3p9WKaRiA05USpjusk2s+I1IxyGJe8VEFC6JD0oWOoIBEoP5teP8aHRunhMJamhMZT9edERiKlRlFgOiOiB2rem4j/eZ1Uh+d+xkSSahB0tihMOdYxnkSBe0wC1XxkCKGSmVsxHRBJqDaBlUwI7vzLf0nzpOo6Vff2tFy7zOMoon10gCrIRWeohq5RHTUQRQ/oCb2gV+vRerberPdZa8HKZ3bRL1gf313Bk98=</latexit><latexit sha1_base64="W5BzBzpftMOnL9zHHfACZCqQyas=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5GSxC3dREBN0IRTfurGAf0IQymd60QyeTMDMRaij+ihsXirj1P9z5N07bLLR64MLhnHu5954g4Uxpx/myCguLS8srxdXS2vrG5pa9vdNUcSopNGjMY9kOiALOBDQ00xzaiQQSBRxawfBq4rfuQSoWizs9SsCPSF+wkFGijdS19zzgHF/gm4p77EGiGI/FUdcuO1VnCvyXuDkpoxz1rv3p9WKaRiA05USpjusk2s+I1IxyGJe8VEFC6JD0oWOoIBEoP5teP8aHRunhMJamhMZT9edERiKlRlFgOiOiB2rem4j/eZ1Uh+d+xkSSahB0tihMOdYxnkSBe0wC1XxkCKGSmVsxHRBJqDaBlUwI7vzLf0nzpOo6Vff2tFy7zOMoon10gCrIRWeohq5RHTUQRQ/oCb2gV+vRerberPdZa8HKZ3bRL1gf313Bk98=</latexit><latexit sha1_base64="W5BzBzpftMOnL9zHHfACZCqQyas=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5GSxC3dREBN0IRTfurGAf0IQymd60QyeTMDMRaij+ihsXirj1P9z5N07bLLR64MLhnHu5954g4Uxpx/myCguLS8srxdXS2vrG5pa9vdNUcSopNGjMY9kOiALOBDQ00xzaiQQSBRxawfBq4rfuQSoWizs9SsCPSF+wkFGijdS19zzgHF/gm4p77EGiGI/FUdcuO1VnCvyXuDkpoxz1rv3p9WKaRiA05USpjusk2s+I1IxyGJe8VEFC6JD0oWOoIBEoP5teP8aHRunhMJamhMZT9edERiKlRlFgOiOiB2rem4j/eZ1Uh+d+xkSSahB0tihMOdYxnkSBe0wC1XxkCKGSmVsxHRBJqDaBlUwI7vzLf0nzpOo6Vff2tFy7zOMoon10gCrIRWeohq5RHTUQRQ/oCb2gV+vRerberPdZa8HKZ3bRL1gf313Bk98=</latexit><latexit sha1_base64="W5BzBzpftMOnL9zHHfACZCqQyas=">AAAB/XicbVDLSsNAFJ3UV62v+Ni5GSxC3dREBN0IRTfurGAf0IQymd60QyeTMDMRaij+ihsXirj1P9z5N07bLLR64MLhnHu5954g4Uxpx/myCguLS8srxdXS2vrG5pa9vdNUcSopNGjMY9kOiALOBDQ00xzaiQQSBRxawfBq4rfuQSoWizs9SsCPSF+wkFGijdS19zzgHF/gm4p77EGiGI/FUdcuO1VnCvyXuDkpoxz1rv3p9WKaRiA05USpjusk2s+I1IxyGJe8VEFC6JD0oWOoIBEoP5teP8aHRunhMJamhMZT9edERiKlRlFgOiOiB2rem4j/eZ1Uh+d+xkSSahB0tihMOdYxnkSBe0wC1XxkCKGSmVsxHRBJqDaBlUwI7vzLf0nzpOo6Vff2tFy7zOMoon10gCrIRWeohq5RHTUQRQ/oCb2gV+vRerberPdZa8HKZ3bRL1gf313Bk98=</latexit>

Space: O((m log n)/✏2 + (log n)/✏3)<latexit sha1_base64="7ChuLVMfoAgvHdo3V9TTq6zMKjs=">AAACF3icbVDJSgNBEO2JW4xb1KOXxiBMEMaZKOgx6MWbEcwCyRh6Op2kSS9Dd48QhvyFF3/FiwdFvOrNv7GzHDTxQcHjvSqq6kUxo9r4/reTWVpeWV3Lruc2Nre2d/K7ezUtE4VJFUsmVSNCmjAqSNVQw0gjVgTxiJF6NLga+/UHojSV4s4MYxJy1BO0SzEyVmrnvRvX5bDFZA+K4kmLxJoyKe5L8Bi6C+ppsZ0v+J4/AVwkwYwUwAyVdv6r1ZE44UQYzJDWzcCPTZgiZShmZJRrJZrECA9QjzQtFYgTHaaTv0bwyCod2JXKljBwov6eSBHXesgj28mR6et5byz+5zUT070IUyrixBCBp4u6CYNGwnFIsEMVwYYNLUFYUXsrxH2kEDY2ypwNIZh/eZHUSl7ge8HtWaF8OYsjCw7AIXBBAM5BGVyDCqgCDB7BM3gFb86T8+K8Ox/T1owzm9kHf+B8/gDLlp00</latexit><latexit sha1_base64="7ChuLVMfoAgvHdo3V9TTq6zMKjs=">AAACF3icbVDJSgNBEO2JW4xb1KOXxiBMEMaZKOgx6MWbEcwCyRh6Op2kSS9Dd48QhvyFF3/FiwdFvOrNv7GzHDTxQcHjvSqq6kUxo9r4/reTWVpeWV3Lruc2Nre2d/K7ezUtE4VJFUsmVSNCmjAqSNVQw0gjVgTxiJF6NLga+/UHojSV4s4MYxJy1BO0SzEyVmrnvRvX5bDFZA+K4kmLxJoyKe5L8Bi6C+ppsZ0v+J4/AVwkwYwUwAyVdv6r1ZE44UQYzJDWzcCPTZgiZShmZJRrJZrECA9QjzQtFYgTHaaTv0bwyCod2JXKljBwov6eSBHXesgj28mR6et5byz+5zUT070IUyrixBCBp4u6CYNGwnFIsEMVwYYNLUFYUXsrxH2kEDY2ypwNIZh/eZHUSl7ge8HtWaF8OYsjCw7AIXBBAM5BGVyDCqgCDB7BM3gFb86T8+K8Ox/T1owzm9kHf+B8/gDLlp00</latexit><latexit sha1_base64="7ChuLVMfoAgvHdo3V9TTq6zMKjs=">AAACF3icbVDJSgNBEO2JW4xb1KOXxiBMEMaZKOgx6MWbEcwCyRh6Op2kSS9Dd48QhvyFF3/FiwdFvOrNv7GzHDTxQcHjvSqq6kUxo9r4/reTWVpeWV3Lruc2Nre2d/K7ezUtE4VJFUsmVSNCmjAqSNVQw0gjVgTxiJF6NLga+/UHojSV4s4MYxJy1BO0SzEyVmrnvRvX5bDFZA+K4kmLxJoyKe5L8Bi6C+ppsZ0v+J4/AVwkwYwUwAyVdv6r1ZE44UQYzJDWzcCPTZgiZShmZJRrJZrECA9QjzQtFYgTHaaTv0bwyCod2JXKljBwov6eSBHXesgj28mR6et5byz+5zUT070IUyrixBCBp4u6CYNGwnFIsEMVwYYNLUFYUXsrxH2kEDY2ypwNIZh/eZHUSl7ge8HtWaF8OYsjCw7AIXBBAM5BGVyDCqgCDB7BM3gFb86T8+K8Ox/T1owzm9kHf+B8/gDLlp00</latexit>

Train time: O((n log n)/✏2 + (n log n)/✏3)<latexit sha1_base64="gLDt3cGUvwVVfnKtCLBdb8LBtm0=">AAACGXicbVDLSgMxFM3UV62vUZdugkVoEepMFXRZdOPOCvYBnbFk0kwbmkmGJCOUob/hxl9x40IRl7ryb0zbWWjtgQuHc+7l3nuCmFGlHefbyi0tr6yu5dcLG5tb2zv27l5TiURi0sCCCdkOkCKMctLQVDPSjiVBUcBIKxheTfzWA5GKCn6nRzHxI9TnNKQYaSN1beemVOLQY6IPefnEI7GiTPD7KjyGi/TTctcuOhVnCvifuBkpggz1rv3p9QROIsI1ZkipjuvE2k+R1BQzMi54iSIxwkPUJx1DOYqI8tPpZ2N4ZJQeDIU0xTWcqr8nUhQpNYoC0xkhPVDz3kRc5HUSHV74KeVxognHs0VhwqAWcBIT7FFJsGYjQxCW1NwK8QBJhLUJs2BCcOdf/k+a1YrrVNzbs2LtMosjDw7AISgBF5yDGrgGddAAGDyCZ/AK3qwn68V6tz5mrTkrm9kHf2B9/QAKop3X</latexit><latexit sha1_base64="gLDt3cGUvwVVfnKtCLBdb8LBtm0=">AAACGXicbVDLSgMxFM3UV62vUZdugkVoEepMFXRZdOPOCvYBnbFk0kwbmkmGJCOUob/hxl9x40IRl7ryb0zbWWjtgQuHc+7l3nuCmFGlHefbyi0tr6yu5dcLG5tb2zv27l5TiURi0sCCCdkOkCKMctLQVDPSjiVBUcBIKxheTfzWA5GKCn6nRzHxI9TnNKQYaSN1beemVOLQY6IPefnEI7GiTPD7KjyGi/TTctcuOhVnCvifuBkpggz1rv3p9QROIsI1ZkipjuvE2k+R1BQzMi54iSIxwkPUJx1DOYqI8tPpZ2N4ZJQeDIU0xTWcqr8nUhQpNYoC0xkhPVDz3kRc5HUSHV74KeVxognHs0VhwqAWcBIT7FFJsGYjQxCW1NwK8QBJhLUJs2BCcOdf/k+a1YrrVNzbs2LtMosjDw7AISgBF5yDGrgGddAAGDyCZ/AK3qwn68V6tz5mrTkrm9kHf2B9/QAKop3X</latexit><latexit sha1_base64="gLDt3cGUvwVVfnKtCLBdb8LBtm0=">AAACGXicbVDLSgMxFM3UV62vUZdugkVoEepMFXRZdOPOCvYBnbFk0kwbmkmGJCOUob/hxl9x40IRl7ryb0zbWWjtgQuHc+7l3nuCmFGlHefbyi0tr6yu5dcLG5tb2zv27l5TiURi0sCCCdkOkCKMctLQVDPSjiVBUcBIKxheTfzWA5GKCn6nRzHxI9TnNKQYaSN1beemVOLQY6IPefnEI7GiTPD7KjyGi/TTctcuOhVnCvifuBkpggz1rv3p9QROIsI1ZkipjuvE2k+R1BQzMi54iSIxwkPUJx1DOYqI8tPpZ2N4ZJQeDIU0xTWcqr8nUhQpNYoC0xkhPVDz3kRc5HUSHV74KeVxognHs0VhwqAWcBIT7FFJsGYjQxCW1NwK8QBJhLUJs2BCcOdf/k+a1YrrVNzbs2LtMosjDw7AISgBF5yDGrgGddAAGDyCZ/AK3qwn68V6tz5mrTkrm9kHf2B9/QAKop3X</latexit>

Test time: O((m+ 1/✏)/✏2 log n)<latexit sha1_base64="btbTW2diLSRtRM1qXcm9PvAE8yQ=">AAACDHicbVDLSgMxFM3UV62vqks3wSK0CHWmCLosunFnBfuAzlgyadqG5jEkGaEM/QA3/oobF4q49QPc+Tem7SDaeiBwOOdcbu4JI0a1cd0vJ7O0vLK6ll3PbWxube/kd/caWsYKkzqWTKpWiDRhVJC6oYaRVqQI4iEjzXB4OfGb90RpKsWtGUUk4KgvaI9iZKzUyReui0V+7J34JNKUSVH6YXcV6DPZh6JkU27ZnQIuEi8lBZCi1sl/+l2JY06EwQxp3fbcyAQJUoZiRsY5P9YkQniI+qRtqUCc6CCZHjOGR1bpwp5U9gkDp+rviQRxrUc8tEmOzEDPexPxP68dm955kFARxYYIPFvUixk0Ek6agV2qCDZsZAnCitq/QjxACmFj+8vZErz5kxdJo1L23LJ3c1qoXqR1ZMEBOARF4IEzUAVXoAbqAIMH8ARewKvz6Dw7b877LJpx0pl98AfOxzf1X5mp</latexit><latexit sha1_base64="btbTW2diLSRtRM1qXcm9PvAE8yQ=">AAACDHicbVDLSgMxFM3UV62vqks3wSK0CHWmCLosunFnBfuAzlgyadqG5jEkGaEM/QA3/oobF4q49QPc+Tem7SDaeiBwOOdcbu4JI0a1cd0vJ7O0vLK6ll3PbWxube/kd/caWsYKkzqWTKpWiDRhVJC6oYaRVqQI4iEjzXB4OfGb90RpKsWtGUUk4KgvaI9iZKzUyReui0V+7J34JNKUSVH6YXcV6DPZh6JkU27ZnQIuEi8lBZCi1sl/+l2JY06EwQxp3fbcyAQJUoZiRsY5P9YkQniI+qRtqUCc6CCZHjOGR1bpwp5U9gkDp+rviQRxrUc8tEmOzEDPexPxP68dm955kFARxYYIPFvUixk0Ek6agV2qCDZsZAnCitq/QjxACmFj+8vZErz5kxdJo1L23LJ3c1qoXqR1ZMEBOARF4IEzUAVXoAbqAIMH8ARewKvz6Dw7b877LJpx0pl98AfOxzf1X5mp</latexit><latexit sha1_base64="btbTW2diLSRtRM1qXcm9PvAE8yQ=">AAACDHicbVDLSgMxFM3UV62vqks3wSK0CHWmCLosunFnBfuAzlgyadqG5jEkGaEM/QA3/oobF4q49QPc+Tem7SDaeiBwOOdcbu4JI0a1cd0vJ7O0vLK6ll3PbWxube/kd/caWsYKkzqWTKpWiDRhVJC6oYaRVqQI4iEjzXB4OfGb90RpKsWtGUUk4KgvaI9iZKzUyReui0V+7J34JNKUSVH6YXcV6DPZh6JkU27ZnQIuEi8lBZCi1sl/+l2JY06EwQxp3fbcyAQJUoZiRsY5P9YkQniI+qRtqUCc6CCZHjOGR1bpwp5U9gkDp+rviQRxrUc8tEmOzEDPexPxP68dm955kFARxYYIPFvUixk0Ek6agV2qCDZsZAnCitq/QjxACmFj+8vZErz5kxdJo1L23LJ3c1qoXqR1ZMEBOARF4IEzUAVXoAbqAIMH8ARewKvz6Dw7b877LJpx0pl98AfOxzf1X5mp</latexit>