Alta perfomance com Lucene
-
Upload
rodrigo-zaccara -
Category
Documents
-
view
1.252 -
download
0
description
Transcript of Alta perfomance com Lucene
![Page 1: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/1.jpg)
Alta performance com Lucene
ROdLéo Nickel
Saturday, October 22, 2011
![Page 2: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/2.jpg)
22/10/2011 2
ROd
Desenvolvedor
especialista em
arquiteturas de alta
performance
relacionadas a
publicação e busca
de conteúdo
@rzdrigo
Saturday, October 22, 2011
![Page 3: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/3.jpg)
22/10/2011 3
Léo Nickel
Desenvolvedor de
sistemas com
experiência de 6
anos em aplicações
Java EE em
telefonia, marketing
online e busca
orgânica
@leonardonickel
Saturday, October 22, 2011
![Page 4: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/4.jpg)
22/10/2011 4
LUCENE
• Apache Top Level Project desde 02/2005
• Orientado a documento
• Schemaless
• Instância única de IndexWriter por índice
• Quebra da API entre:
– 2.4 → 2.9
– 3.1 → 3.2 (incompatibilidade entre índices)
Saturday, October 22, 2011
![Page 5: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/5.jpg)
22/10/2011 5
MOTIVAÇÃO
• Substituição do motor de busca FAST SEARCH
• Requisitos não funcionais
- Baixa latência de indexação
- Alta disponibilidade de busca
- Sharding
Saturday, October 22, 2011
![Page 6: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/6.jpg)
22/10/2011 6
IMPLEMENTAÇÃO
Saturday, October 22, 2011
![Page 7: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/7.jpg)
22/10/2011 7
TIMELINE
SERVIDOR LINUX 1NFS PARAMS
24 QPS
Saturday, October 22, 2011
![Page 8: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/8.jpg)
22/10/2011 8
TIMELINE
OTIMIZAÇÃOCOMPOUND FILE
READ ONLY
FIELD SELECTOR
ESTRATÉGIA DE REOPEN
Saturday, October 22, 2011
![Page 9: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/9.jpg)
22/10/2011 9
TIMELINE
SERVIDOR LINUX 1NFS PARAMS
30 QPS
Saturday, October 22, 2011
![Page 10: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/10.jpg)
22/10/2011 10
TIMELINE
DESKTOP LINUXDISCO LOCAL
200 QPS
Saturday, October 22, 2011
![Page 11: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/11.jpg)
22/10/2011 11
TIMELINE
SERVIDOR LINUX 2DISCO LOCAL
500 QPS
Saturday, October 22, 2011
![Page 12: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/12.jpg)
22/10/2011 12
TIMELINE
SERVIDOR LINUX 2NFS DEFAULT + RO
450 QPS
Saturday, October 22, 2011
![Page 13: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/13.jpg)
22/10/2011 13
TIMELINE
QUESTÃO:
VÁRIOS SEARCHERS NO MESMO
NFS DEGRADA A PERFORMANCE?
Saturday, October 22, 2011
![Page 14: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/14.jpg)
22/10/2011 14
TIMELINE
SERVIDOR LINUX 1 e 2CARGA: SERVIDOR LINUX 3 e 4
900 QPS
Saturday, October 22, 2011
![Page 15: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/15.jpg)
22/10/2011 15
EM PRODUÇÃO...
SERVIDOR LINUX 1 e 2NFS DEFAULT + RO
27 → 413 → 962 QPS
Saturday, October 22, 2011
![Page 16: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/16.jpg)
22/10/2011 16
LIÇÕES APRENDIDAS
• Otimização constante do índice
• Configuração de File Descriptor no SO
• Diferentes politicas de Sharding
• Fieldable != Field
• MultiReader com ExecutorServices
• Evitar o uso de PrefixQuery
- Validar se campo existe
Saturday, October 22, 2011
![Page 17: Alta perfomance com Lucene](https://reader034.fdocuments.net/reader034/viewer/2022052311/559a3e6f1a28abbd398b458f/html5/thumbnails/17.jpg)
Obrigado =)http://www.slideshare.net/rzdrigo/alta-
performance-com-lucene
ROd ([email protected])Léo Nickel ([email protected])
@rzdrigo@leonardonickel
Saturday, October 22, 2011