Toward object-oriented deep reinforcement...
Transcript of Toward object-oriented deep reinforcement...
![Page 1: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/1.jpg)
Matthew BotvinickDeepMind, London UKGatsby Computational Neuroscience Unit, UCL
Toward object-oriented deep reinforcement learning
![Page 2: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/2.jpg)
+1
+1
atari
Mnih et al, Nature (2015)
![Page 3: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/3.jpg)
+1
+1
Jaderberg et al., Science, 2019
![Page 4: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/4.jpg)
+1
+1
dqn convnet
Mnih et al, Nature (2015)
![Page 5: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/5.jpg)
+1
+1lake
![Page 6: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/6.jpg)
+1
+1
![Page 7: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/7.jpg)
+1
+1objects — pic
![Page 8: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/8.jpg)
+1
+1human objects
Kahneman et al., 1992
![Page 9: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/9.jpg)
+1
+1
Egly, Driver, and Rafal (1994); Moore, Yantis, and Vaughan (1998)
Automatic spread of attention
![Page 10: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/10.jpg)
+1
+1
Roelfsema et al. Nature, 1998
![Page 11: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/11.jpg)
+1
+1
LO??? (Kanwisher)
Malach et al., PNAS, 1995
![Page 12: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/12.jpg)
+1
+1
objects — pic AGAIN
![Page 13: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/13.jpg)
+1
+1diuk (cf?)
cf. Keramati et al., 2018; Cobo et al., 2013; Garnelo et al., 2016; Lazaro-Gradillo et al., 2019; Zambaldi, et al., 2018
![Page 14: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/14.jpg)
![Page 15: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/15.jpg)
+1
+1
![Page 16: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/16.jpg)
+1
+1
E.g., Girshick, 2015; He et al., 2017; Redmon & Farhadi, 2018
![Page 17: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/17.jpg)
+1
+1
![Page 18: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/18.jpg)
Alex Lerchner Chris Burgess Loic Matthey Klaus Greff
Nick Watters Irina Higgins Rishabh Kabra Malcolm Reynolds
![Page 19: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/19.jpg)
![Page 20: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/20.jpg)
![Page 21: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/21.jpg)
half refrigerator
![Page 22: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/22.jpg)
other half refrigerator
![Page 23: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/23.jpg)
+1
+1
objects — pic AGAIN
![Page 24: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/24.jpg)
![Page 25: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/25.jpg)
![Page 26: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/26.jpg)
![Page 27: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/27.jpg)
![Page 28: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/28.jpg)
![Page 29: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/29.jpg)
![Page 30: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/30.jpg)
+1
![Page 31: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/31.jpg)
+1
+1
![Page 32: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/32.jpg)
+1
+1
![Page 33: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/33.jpg)
+1
+1
![Page 34: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/34.jpg)
+1
+1
Kahneman & Treisman, 1984: Object Files
Green, Edwin James, and Jake Quilty-Dunn. "what is an object file?." The British Journal for the Philosophy of Science (2017).
![Page 35: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/35.jpg)
+1
+1
![Page 36: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/36.jpg)
+1
+1
![Page 37: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/37.jpg)
+1
+1
![Page 38: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/38.jpg)
+1
+1
![Page 39: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/39.jpg)
+1
+1
![Page 40: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/40.jpg)
+1
+1
![Page 41: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4c1d32dbc3a5a2853d9d9/html5/thumbnails/41.jpg)
Alex Lerchner Chris Burgess Loic Matthey Klaus Greff
Nick Watters Irina Higgins Rishabh Kabra Malcolm Reynolds