Nutch - web-scale search engine toolkit
Munching & crunching - Lucene index post-processing
Nutch as a Web data mining platform