AWS Search Services
-
Upload
eiji-shinohara -
Category
Technology
-
view
2.372 -
download
5
Transcript of AWS Search Services
-
AWS Search ServicesEiji Shinohara
(2014A9CloudSearch)
-
{ "Name" : "", "Twitter" : "@shinodogg", "Profile" : { "Role" : "Solutions Architect", "Market": "Startup", "Services" : [ "Amazon CloudSearch", "Amazon Elasticsearch Service", "Amazon Simple Workflow Service, "AWS Elastic Beanstalk ] }}
New
-
AWS Amazon CloudSearch
https://aws.amazon.com/jp/cloudsearch/
Amazon Elasticsearch Service https://aws.amazon.com/jp/elasticsearch-service/
Amazon CloudSearch Amazon Elasticsearch Service
-
A9.com CloudSearch/Amazon ES
-
AmazonA9A9.com
-
Amazon CloudSearch
2011 API
A9 Amazon.com
2013 API on top of Apache Solr
n-gram 20143
-
Amazon CloudSearch
Auto Scaling / Auto Partitining
Auto Partitioning
Auto Scaling
-
Amazon CloudSearch
Amazon EMR
Index
Index P1
Index P2Amazon EMR
-
Amazon CloudSearch
schoo nanapi ChatWork SnapDish SmartInsight Lancers SmartNews
A9 x SmartNews in Palo Alto
CloudSearch Meetup at Amazon Meguro Office
-
SourceFieldDynamicField
SourceField Bi-Gram2
SolrCopy Field
-
SourceFieldDynamicField
DynamicField 1
-
SourceFieldDynamicField
DynamicField bar_txt
*_txt
-
(Morphological Analysis)
Amazon CloudSearch - Japanese Text Processing
-
(Morphological Analysis) : AWS CLI/SDK{ "AnalysisSchemeName": "jascheme", "AnalysisSchemeLanguage": "ja", "AnalysisOptions": { "JapaneseTokenizationDictionary": "[ [\\,\ \,\ \",\",,\]]" }}$ aws cloudsearch define-analysis-scheme --region us-east-1 --domain-name mydomain \--analysis-scheme file://jascheme.txt
Amazon CloudSearch - Japanese Text Processing
-
(Synonym Addition) Venice
Alias Pupil => Student Student => Pupil
Group 1st, first, one =>
CLI/SDK
http://ja.wikipedia.org/wiki/
Amazon CloudSearch - Japanese Text Processing
-
A/B
Amazon CloudSearch - Japanese Text Processing
-
Amazon CloudSearch
CloudSearch
CloudSearch5MB
Amazon CloudSearchAPI SolrSolrAPI Solr
-
Elasticsearch
Google Trends
-
ELK stack
ELK is Elasticsearch, Logstash, and Kibana
Elasticsearch is a distributed, schema-free search and analytics engine
Logstash is a tool for collecting and managing events and logs.
Kibana is a browser-based analytics and search dashboard for Elasticsearch.
-
Kibana
-
http://www.binpress.com/blog/2014/10/21/binpress-podcast-episode-16-shay-banon-elasticsearch/
-
Amazon Elasticsearch Service
-
Shay()
-
Amazon Elasticsearch Service
Elasticsearch AWS Elasticsearch
Elasticsearch/
Kibana Management ConsoleURL
EC2 EBSEBS
Amazon ES
-
Amazon Elasticsearch Service
http://aws.typepad.com/aws_japan/2015/10/amazon-elasticsearch-service.html
-
Amazon Elasticsearch Service
-
Elasticsearch
Apache Lucene(Apache SolrCloudSearch)
Kibana Amazon Elasticsearch Service
AWS1) CloudWatch LogsLambdaElasticsearchKibana2) DynamoDB StreamsLogstashElasticsearch AZ
Zone Awareness
Amazon Elasticsearch Service
-
Amazon ES CloudSearch
ElasticIAMCloudWatch
Amazon Elasticsearch Service
-
Amazon ESCloudWatch Metricsq Cluster status(Green/Yellow/Red)q Node countq Searchable documentsq Deleted documentsq CPU utilizationq Free storage spaceq JVM memory pressureq Automatic snapshot failuresq Master CPU utilization
q Master free storage spaceq Master JVM memory pressureq Read IOPSq Write IOPSq Read latencyq Write latencyq Read throughputq Write throughputq Disk queue depth
Amazon Elasticsearch Service
-
Amazon Elasticsearch Service
IAM Integration IP
) KibanaIP Signed requests with SigV4
) AWS Logstash: https://github.com/awslabs/logstash-output-amazon_es
Fine-grained ) Index
Logstash Amazon ESData Source
-
Amazon Elasticsearch Service
ICUKuromoji
analysis-icu() analysis-kuromoji()
(201510)
-
Amazon Elasticsearch Service
Deployment
node node
EBS EBS
node
node
nodeMaster Nodes Data Nodes
ClusterNode
Shard 3
node
EBS
node
EBS
Shard 1
Shard 4
Shard 2
-
Amazon Elasticsearch Service
AWS
Daily: 11 Automated snapshot start hour AWS
w/ _snapshot API ElasticsearchS3
-
Amazon Elasticsearch Service
2016
) Elasticsearch2.0Kibana4.2 9300Elasticsearch VPC
AmazonES