No me indexes que me cacheo
-
Upload
chema-alonso -
Category
Technology
-
view
14.063 -
download
1
description
Transcript of No me indexes que me cacheo
![Page 1: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/1.jpg)
@chemaalonso
![Page 2: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/2.jpg)
Gracias por la invitación
![Page 3: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/3.jpg)
Index
Cache
Index
Cache
Other SearchEngines
Pub doc
Index & Cache
![Page 4: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/4.jpg)
Cache
http://www.elladodelmal.com/2013/08/una-anecdota-con-google-archive-y-la.html
![Page 5: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/5.jpg)
Primary Index & Secondary Index
http://www.elladodelmal.com/2014/07/cuantas-urls-se-pueden-extraer-con.html
![Page 6: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/6.jpg)
What is Robots.txt for?
• Evita la indexación de contenidos de las URLs protegidas
• Por tanto, no se realiza spidering• No evita que se indexen las URLS
![Page 7: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/7.jpg)
Robots.txt Security Issues
B14Ck S30 Pwn1nage!
![Page 8: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/8.jpg)
Robots.txt “leakeage”
![Page 9: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/9.jpg)
Robots.txt “leakeage” + Archive.org
![Page 10: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/10.jpg)
Robots.txt “leakeage” + Google
http://www.elladodelmal.com/2013/09/buscando-en-robotstxt-lo-que-esta.html
![Page 11: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/11.jpg)
Robots.txt “leakeage”+ Google+ Directory Listing
http://www.elladodelmal.com/2013/09/buscando-en-robotstxt-lo-que-esta.html
![Page 12: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/12.jpg)
Robots.txt “leakeage” + Google + IIS ShortName b
http://www.elladodelmal.com/2013/10/un-bug-de-iis-short-name-en-el-windows.html
![Page 13: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/13.jpg)
Robots.txt “leakeage” + Google + IIS ShortName + FOCA
http://www.elladodelmal.com/2012/07/listado-de-ficheros-en-iis-7-utilizando.html
![Page 14: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/14.jpg)
Indexing the Robots.txt
![Page 15: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/15.jpg)
Indexing the Robots.txt
![Page 16: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/16.jpg)
Indexing the Robots.txt
![Page 17: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/17.jpg)
Indexing the Robots.txt:Blogger preview + Cache
http://www.elladodelmal.com/2012/08/minority-report-pre-visualizando-el.html
![Page 18: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/18.jpg)
Indexing the Robots.txt:WordPress preview + Cache
http://www.elladodelmal.com/2014/07/wordpress-ten-cuidado-con-el-cacheo-de.html
![Page 19: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/19.jpg)
Indexing the Robots.txt:WordPress preview + Cache
http://www.elladodelmal.com/2014/07/wordpress-ten-cuidado-con-el-cacheo-de.html
![Page 20: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/20.jpg)
“GmailGate” with Robots.txt:Octubre de 2013
http://www.elladodelmal.com/2013/10/79400-urls-de-gmail-indexadas-en-google.html
![Page 21: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/21.jpg)
“GmailGate” with Robots.txt:Octubre de 2013
http://www.elladodelmal.com/2013/10/79400-urls-de-gmail-indexadas-en-google.html
![Page 22: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/22.jpg)
“GmailGate” with Robots.txt:Octubre de 2013
http://www.elladodelmal.com/2013/10/79400-urls-de-gmail-indexadas-en-google.html
![Page 23: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/23.jpg)
“GmailGate” with Robots.txt:Abril de 2014
http://www.elladodelmal.com/2014/05/gmail-borraste-en-google-pero-te-quedan.html
![Page 24: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/24.jpg)
“GmailGate” with Robots.txt:Julio de 2014
http://www.elladodelmal.com/2014/07/googe-si-usa-bing-para-borrar-las-urls.html
![Page 25: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/25.jpg)
“Facebookgate” with Robots.txt
http://www.elladodelmal.com/2013/09/facebook-tiene-problemas-con-la.html
![Page 26: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/26.jpg)
“Facebookgate” with Robots.txt
http://www.elladodelmal.com/2013/09/facebook-tiene-problemas-con-la.html
![Page 27: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/27.jpg)
“WhatsAppGate” with Robots.txt
http://www.elladodelmal.com/2013/09/problemas-de-privacidad-de-whatsapp-con.html
![Page 28: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/28.jpg)
Indexing the robots.txt + XSS = XSS Google-Persistentes
http://es.slideshare.net/chemai64/xss-google-persistentes
![Page 29: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/29.jpg)
Robots.txt
• Previene que se indexe a partir de las rutas puestas.
• Evita que se guarde contenido en el índice de Google/Bing/Otros
• No evita que la URL, el título, y las keywords del enlace se indexen.
• Puede ser un leak de información en ataques dirigidos y en ataques de dorking.
• No evita la indexación en el pasado.
![Page 30: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/30.jpg)
“Evernotegate” with Robots.txt
http://www.elladodelmal.com/2014/08/evernote-no-quiere-hacer-nada-cuida-tus.html
![Page 31: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/31.jpg)
Not cache, but it is in the index
http://www.elladodelmal.com/2014/08/evernote-no-quiere-hacer-nada-cuida-tus.html
![Page 32: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/32.jpg)
Evernote, Index & Cache
http://www.elladodelmal.com/2014/08/evernote-no-quiere-hacer-nada-cuida-tus.html
![Page 33: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/33.jpg)
“Evernotegate” with Robots.txt
http://www.elladodelmal.com/2014/08/evernote-no-quiere-hacer-nada-cuida-tus.html
![Page 34: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/34.jpg)
El “viagrate” del Albacete Balompié
![Page 35: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/35.jpg)
Demo: Brute Forcing the index
![Page 36: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/36.jpg)
How to manage the relationship?
http://www.slideshare.net/chemai64/black-seov3
![Page 37: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/37.jpg)
How to manage the relationship?
• Evitar rutas con contenido mixto (publico/privado)
• Evitar contenido no enlazado en rutas publicas
• Evitar rutas privadas conocidas (/etc/ /users/)• Evitar rutas privadas explícitas• Evitar configuraciones privadas automaticas • Evitar el uso de rutas privadas a fichero • Aplicar la misma configuración para todas las
aranas de todos los buscadores de Internet• Proteger las rutas privadas con listas de
control de acceso si es posible
http://www.slideshare.net/chemai64/black-seov3
![Page 38: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/38.jpg)
How to manage the relationship?(Google)
https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag
![Page 39: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/39.jpg)
HTML Meta Tags
https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag
![Page 40: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/40.jpg)
X-Robots-Tag HTTP header
https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag
![Page 41: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/41.jpg)
Google WebMaster Tools
https://www.google.com/webmasters/tools/home?hl=es
![Page 42: No me indexes que me cacheo](https://reader031.fdocuments.net/reader031/viewer/2022012916/55789493d8b42aaf518b48db/html5/thumbnails/42.jpg)
Faast: Persistent Pentesting
• BlackSEO– Cheap Viagra– Cheap Software– Etc…
• Robots.txt fingerprinting– SW versions
• Robots.txt leakeage– Testing all forbiden URLs
• Robots.txt indexation– Searching forbidden
URLS in Google/Bing
https://www.elevenpaths.com/technology/faast/index.html