Eric Redmond@coderoshi

Riak Search 2.0

After two years, Riak 2.0 is out

┬─┬ ( ^_^ノ)

Setup

ノ

## The enabled Search set this 'on'. search = off !## The port number which Solr binds to. search.solr_port = 10014 !## The port number which Solr JMX binds to. search.solr_jmx_port = 10013 !## The arguments to pass to the Solr JVM. Non-standard ## arguments, i.e. -XX, may not be portable across JVM ## implementations. E.g. -XX:+UseCompressedStrings. search.solr_jvm_args = -Xms1g -Xmx1g -XX:+UseStringCache -XX:+UseCompressedOops

riak.conf

## The enabled Search set this 'on'. search = on !## The port number which Solr binds to. search.solr_port = 10014 !## The port number which Solr JMX binds to. search.solr_jmx_port = 10013 !## The arguments to pass to the Solr JVM. Non-standard ## arguments, i.e. -XX, may not be portable across JVM ## implementations. E.g. -XX:+UseCompressedStrings. search.solr_jvm_args = -Xms1g -Xmx1g -XX:+UseStringCache -XX:+UseCompressedOops

riak.conf

riak-admin cluster join riak@10.0.0.2 ... and so on... riak-admin cluster plan riak-admin cluster commit

riak-admin cluster join riak@10.0.0.2 ... and so on... riak-admin cluster plan riak-admin cluster commit

riak-admin security add-user eric 12345 riak-admin security add-user admin 123456 riak-admin security grant search.query ON index simple TO any riak-admin security grant search.admin ON schema TO admin

require 'riak' client = Riak::Client.new !

# create index client.create_search_index('simple') !

# tie the index to bucket 'cats' bucket = Riak::Bucket.new(client, 'cats') bucket.props = { search_index: 'simple' }

Indexing Datatypes

UTF8中搜索UTF8חיפוש בΑναζήτηση σε UTF8

Искать в UTF8

Search in UTF8

Advanced Searchaka. Next Level Search

solr = RSolr.connect(url: 'http://yokozuna01.bos1:8098/solr/docs')

resp = solr.get('select', params: {q: '*:*'})

Facets, Stats, and stuff

eDisMax

defType=edismax

Highlighting

Languages

Map/Reduce

{"inputs": { "module":"yokozuna", "function":"mapred_search", "arg":["docs","title_s:Key* AND language_s:en"] }, "query":[ {"map":{ "language":"javascript", "keep":false, "source":"function(v) { return [1]; }"}}, {"reduce"{ "language":"javascript", "keep":true, "name":"Riak.reduceSum" }} ] }

bucket = Riak::Bucket.new(client, 'people') bucket.props = { search_index: 'faces' } bucket.get('stuart').store bucket = Riak::Bucket.new(client, 'cats') bucket.props = { search_index: 'faces' } bucket.get('stuart').store bucket = Riak::Bucket.new(client, 'mountains') bucket.props = { search_index: 'faces' } bucket.get('hood').store

riak-admin bucket-type create faces '{"props":{"search_index":"faces"}}' !

bucket = Riak::Bucket.new(client, ‘people') bucket.get('stuart', type:'faces').store bucket = Riak::Bucket.new(client, ‘cats') bucket.get('stuart', type:'faces').store bucket = Riak::Bucket.new(client, ‘mountains') bucket.get('stuart', type:'faces').store

Schemas

<schema name="default" version="1.5"> <fields> <field name="kinds" type="string" indexed="true" stored="false" multiValued="true" /> <field name="name" type="string" indexed="true" stored="true" /> ! <dynamicField name="*_ss" type="string" indexed="true" stored="true" multiValued="true"/> ! <field name="_yz_ed" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_pn" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_fpn" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_vtag" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_node" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_rk" type="_yz_str" indexed="true" stored="true"/> <field name="_yz_rb" type="_yz_str" indexed="true" stored="true"/> </fields> </schema>

Fallen Nodes

Why?

Thermocline

Thermocline of Replication

Goals

But... why?

How

Yokozuna = Glue

Extractorsyz_extractor.erl yz_json_extractor.erl yz_noop_extractor.erl yz_text_extractor.erl yz_xml_extractor.erl yz_doc.erl

Interfaces

yz_pb_admin.erl yz_pb_search.erl yz_schema.erl yz_wm_extract.erl yz_wm_index.erl yz_wm_schema.erl yz_wm_search.erl

Security

Permissions

riak-admin security grant search.admin ON index TO admin

riak-admin security grant search.query ON index TO user

riak-admin security grant search.admin ON schema TO admin

riak-admin security grant search.query ON index wiki TO user

Administrationyz_pb_admin.erl yz_schema.erl yz_wm_index.erl yz_wm_schema.erl

Searchesyz_pb_search.erl yz_wm_search.erl

Backend Magicyokozuna.erl yz_app.erl yz_general_sup.erl yz_kv.erl yz_misc.erl yz_sup.erl

Distribution

yz_cover.erl yz_events.erl

AAEyz_entropy.erl yz_entropy_mgr.erl yz_exchange_fsm.erl yz_index.erl yz_index_hashtree.erl yz_index_hashtree_sup.erl

Solr code

EntropyData.java Monitor.java yz_solr.erl yz_solr_proc.erl yz_solr_sup.erl

Stats

yz_stat.erl yz_stat_worker.erl

“Testing only shows the presence, not

the absence, of bugs”

aae_test.erl yokozuna_essential.erl yz_errors.erl yz_fallback.erl yz_flag_transitions.erl yz_index_admin.erl yz_languages.erl yz_mapreduce.erl yz_monitor_solr.erl yz_pb.erl yz_rs_migration.erl yz_rt.erl yz_schema_admin.erl yz_siblings.erl yz_solr_start_timeout.erl yz_stat_test.erl yz_wm_extract_test.erl yz_component_tests.erl yz_json_extractor_tests.erl yz_kv_tests.erl yz_misc_tests.erl yz_text_extractor_tests.erl yz_xml_extractor_tests.erl yz_driver.erl yz_file_terms.erl

Dijkstra

Thanks

Eric Redmond@coderoshi