DevCommerce Conference 2016: Busca e Data Lake Analytics
Transcript of DevCommerce Conference 2016: Busca e Data Lake Analytics
![Page 2: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/2.jpg)
• 786 lojas físicas
• 8 centros de distribuição
• +18 mil colaboradores
• +40 milhões de clientes
• ~16 milhões de visitantes únicos mês
![Page 3: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/3.jpg)
Big Data
![Page 4: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/4.jpg)
Data lake
![Page 5: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/5.jpg)
![Page 6: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/6.jpg)
• ~200MM linhas novas/dia
• 6 nós: 56 cores + 458GB RAM (AWS)
• 11 TB storage hot, 1 TB S3 arquivos comprimidos
• 1200 Jobs/dia
• 400MB/dia transfer S3 -> HDFS
Volume de informações - Datalake
![Page 7: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/7.jpg)
Recomendações
![Page 8: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/8.jpg)
Sistemas de recomendação
![Page 9: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/9.jpg)
Sistema de recomendação
• Grafo com informações sobre a interação do cliente
• Coleta de informações da navegação dos clientes no site
do magazine:
• Visualização de produtos
• Cálculo de frete
• Adições ao carrinho
• Compras
![Page 10: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/10.jpg)
• ~40k visitantes únicos hora
• ~300k interações com produtos por hora (~5k
minuto)
• Informações salvas no formato de grafo
• ~700 milhões de vértices
• ~ 2.8 bilhões de arestas
Volume de informações - Grafo
![Page 11: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/11.jpg)
Detalhe de produtos
![Page 12: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/12.jpg)
• Resultados iniciais:
• 30% de incremento de vendas em A/B teste com
a ferramenta anterior
Detalhe de produtos
![Page 13: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/13.jpg)
Home personalizada
![Page 14: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/14.jpg)
• Resultados:
• Mudança de layout e mensagem trouxe um
incremento de 7x a venda anterior
Home personalizada
![Page 15: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/15.jpg)
Emails personalizados
![Page 16: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/16.jpg)
• Resultados
• Taxa de abertura de ~24%
• Alguns emails com taxas ~35%
• Conversão 5x maior do que segmentados
Emails personalizados
![Page 17: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/17.jpg)
Push notification
![Page 18: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/18.jpg)
Busca
![Page 19: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/19.jpg)
• Coleta de todas as queries e resultados para o
datalake (~1.8K queries/min)
• Engines: SOLR e Elasticsearch
• Interface administrativa para edição de termos
• Realtime top queries com mais resultados e top
queries com resultado vazio (Intelie)
Busca de produtos
![Page 20: DevCommerce Conference 2016: Busca e Data Lake Analytics](https://reader031.fdocuments.net/reader031/viewer/2022030311/58ef2b671a28aba86c8b4661/html5/thumbnails/20.jpg)