Postgres the best tool you're already using
-
Upload
liquidplanner -
Category
Technology
-
view
1.218 -
download
0
description
Transcript of Postgres the best tool you're already using
![Page 1: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/1.jpg)
Postgres
The Best Tool You're Already Using
• Adam Sanderson
• LiquidPlanner
1
![Page 2: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/2.jpg)
Adam Sanderson
I have been a full stack engineer at LiquidPlanner for 5 years.
• I got off in Kansas*, and that's ok!
• Github: adamsanderson
• Twitter: adamsanderson
• Blog: http://monkeyandcrow.com
* Seattle
2
![Page 3: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/3.jpg)
Online project management with probabilistic scheduling.
• Started in 2007 with Rails 1.x
• Used Postgres from the beginning
• We have learned some great techniques along the way
3
![Page 4: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/4.jpg)
Topics
• Tagging
• Hierarchy
• Custom Data
• Full Text Search
4
![Page 5: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/5.jpg)
Method
For each topic, we'll cover the SQL before we cover its use in ActiveRecord.
We will use Postgres 9.x, Ruby 1.9 syntax, and ActiveRecord 4.0.
If you understand the SQL you can use it in any version of ActiveRecord, 4.0 just
makes it easier.
5
![Page 6: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/6.jpg)
Backstory
You just built a great new social network for hedgehog lovers around the world,
HedgeWith.me.
Everything is going well. You have a few users, but now they want more.
6
![Page 7: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/7.jpg)
My hedgehog is afraid of grumpy hedgehogs, but likes cute ones how can I find him
friends?
hedgehogs4life
Tagging
People want to be able to tag their hedgehogs, and then find other hedgehogs with
certain tags.
“
7
![Page 8: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/8.jpg)
Defining Arrays in SQL
CREATE TABLE hedgehogs (
id integer primary key,
name text,
age integer,
tags text[]
);
8
![Page 9: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/9.jpg)
Defining Arrays in ActiveRecord
create_table :hedgehogs do |t|
t.string :name
t.integer :age
t.text :tags, array: true
end
ActiveRecord 4.x introduced arrays for Postgres, use array:true
9
![Page 10: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/10.jpg)
Heads Up
Define array columns as t.text instead of t.string to avoid casting.
Postgres assumes that ARRAY['cute', 'cuddly'] is of type text[] and will
require you to cast, otherwise you will see errors like this:
ERROR: operator does not exist: character varying[] && text[]
10
![Page 11: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/11.jpg)
Boolean Set Operators
You can use the set operators to query arrays.
• A @> B A contains all of B
• A && B A overlaps any of B
11
![Page 12: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/12.jpg)
Querying Tags in SQL
Find all the hedgehogs that are spiny or prickly:
SELECT name, tags FROM hedgehogs
WHERE tags && ARRAY['spiny', 'prickly'];
A && B A overlaps any of B
12
![Page 13: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/13.jpg)
Querying Tags in SQL
name tags
Marty spiny, prickly, cute
Quilby cuddly, prickly, hungry
Thomas grumpy, prickly, sleepy, spiny
Franklin spiny, round, tiny
13
![Page 14: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/14.jpg)
Querying Tags in SQL
Find all the hedgehogs that are spiny and prickly:
SELECT name, tags FROM hedgehogs
WHERE tags @> ARRAY['spiny', 'prickly'];
A @> B A contains all the B
14
![Page 15: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/15.jpg)
Querying Tags in SQL
name tags
Marty spiny, prickly, cute
Thomas grumpy, prickly, sleepy, spiny
15
![Page 16: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/16.jpg)
Querying Tags in ActiveRecord
Find all the hedgehogs that are spiny and prickly
Hedgehog.where "tags @> ARRAY[?]", ['spiny', 'prickly']
16
![Page 17: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/17.jpg)
Querying Tags in ActiveRecord
Create scopes to encapsulate set operations:
class Hedgehog < ActiveRecord::Base
scope :any_tags, -> (* tags){where('tags && ARRAY[?]', tags)}
scope :all_tags, -> (* tags){where('tags @> ARRAY[?]', tags)}
end
17
![Page 18: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/18.jpg)
Querying Tags in ActiveRecord
Find all the hedgehogs that are spiny or large, and older than 4:
Hedgehog.any_tags('spiny', 'large').where('age > ?', 4)
18
![Page 19: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/19.jpg)
Hi, I run an influential hedgehog club. Our members would all use HedgeWith.me,
if they could show which hogs are members of our selective society.
Boston Spine Fancy President
Hierarchy
Apparently there are thousands of hedgehog leagues, divisions, societies, clubs, and
so forth.
“
19
![Page 20: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/20.jpg)
Hierarchy
We need to efficiently model a club hierarchy like this:
• North American League
• Western Division
• Cascadia Hog Friends
• Californian Hedge Society
How can we support operations like finding a club's depth, children, or parents?
20
![Page 21: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/21.jpg)
Materialized Path in SQL
Encode the parent ids of each record in its path.
CREATE TABLE clubs (
id integer primary key,
name text,
path integer[]
);
21
![Page 22: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/22.jpg)
Querying a Materialized Path
id name path
1 North American League [1]
2 Eastern Division [1,2]
4 New York Quillers [1,2,4]
5 Boston Spine Fancy [1,2,5]
3 Western Division [1,3]
6 Cascadia Hog Friends [1,3,6]
7 California Hedge Society [1,3,7]
... 22
![Page 23: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/23.jpg)
Materialized Path: Depth
The depth of each club is simply the length of its path.
• array_length(array, dim) returns the length of the array
dim will always be 1 unless you are using multidimensional arrays.
23
![Page 24: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/24.jpg)
Materialized Path: Depth
Display the top two tiers of hedgehog clubs:
SELECT name, path, array_length(path, 1) AS depth
FROM clubs
WHERE array_length(path, 1) <= 2
ORDER BY path;
array_length(path, 1) is the depth of record
24
![Page 25: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/25.jpg)
Materialized Path: Depth
name path depth
North American League [1] 1
Eastern Division [1,2] 2
Western Division [1,3] 2
South American League [9] 1
25
![Page 26: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/26.jpg)
Materialized Path: Children
Find all the clubs that are children of the California Hedge Society, ID: 7.
SELECT id, name, path FROM clubs
WHERE path && ARRAY[7]
ORDER BY path
A && B A overlaps any of B
26
![Page 27: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/27.jpg)
Materialized Path: Children
id name path
7 Californian Hedge Society [1,3,7]
8 Real Hogs of the OC [1,3,7,8]
12 Hipster Hogs [1,3,7,12]
Apparently it is illegal to own hedgehogs in California
27
![Page 28: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/28.jpg)
Materialized Path: Parents
Find the parents of the California Hedge Society, Path: ARRAY[1,3,7].
SELECT name, path FROM clubs
WHERE ARRAY[id] && ARRAY[1,3,7]
ORDER BY path;
A && B A overlaps any of B
28
![Page 29: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/29.jpg)
Materialized Path: Parents
id name path
1 North American League [1]
3 Western Division [1,3]
7 Californian Hedge Society [1,3,7]
29
![Page 30: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/30.jpg)
ActiveRecord: Arrays & Depth
With ActiveRecord 4.x, path is just ruby array.
class Club < ActiveRecord::Base
def depth
self.path.length
end
...
30
![Page 31: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/31.jpg)
Querying in ActiveRecord
Encapsulate these conditions as instance methods:
class Club < ActiveRecord::Base
def children
Club.where('path && ARRAY[?]', self.id)
end
def parents
Club.where('ARRAY[id] && ARRAY[?]', self.path)
end
31
![Page 32: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/32.jpg)
Querying in ActiveRecord
Now we have an easy way to query the hierarchy.
@club.parents.limit(5)
@club.children.joins(:hedgehogs).merge(Hedgehog.any_tags('silly'))
These features can all work together.
Mind blown?
32
![Page 33: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/33.jpg)
I need to keep track of my hedgehogs' favorite foods, colors, weight, eye color, and
shoe sizes!
the Quantified Hedgehog Owner
If I am forced to enter my hedgehog's shoe size, I will quit immediately!
the Unquantified Hedgehog Owner
Custom Data
Your users want to record arbitrary data about their hedgehogs.
“
33
![Page 34: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/34.jpg)
Hstore
Hstore provides a hash column type. It is a useful alternative to ActiveRecord's
serialize where the keys and values can be queried in Postgres.
34
![Page 35: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/35.jpg)
Hstore
Hstore needs to be installed manually. Your migration will look like this:
class InstallHstore < ActiveRecord::Migration
def up
execute 'CREATE EXTENSION hstore'
end
...
35
![Page 36: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/36.jpg)
Heads Up
Although hstore is supported by ActiveRecord 4.x, the default schema format does
not support extensions.
Update config/application.rb to use the SQL schema format, otherwise your
tests will fail.
class Application < Rails::Application
config.active_record.schema_format = :sql
end36
![Page 37: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/37.jpg)
Defining an Hstore in SQL
CREATE TABLE hedgehogs (
id integer primary key,
name text,
age integer,
tags text[],
custom hstore DEFAULT '' NOT NULL
);
37
![Page 38: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/38.jpg)
Defining an Hstore in ActiveRecord
hstore is supported in ActiveRecord 4.x as a normal column type:
create_table :hedgehogs do |t|
t.string :name
t.integer :age
t.text :tags, array: true
t.hstore :custom, :default => '', :null => false
end
38
![Page 39: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/39.jpg)
Heads Up
Save yourself some hassle, and specify an empty hstore by default:
t.hstore :custom, :default => '', :null => false
Otherwise new records will have null hstores.
39
![Page 40: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/40.jpg)
Hstore Format
Hstore uses a text format, it looks a lot like a ruby 1.8 hash:
UPDATE hedgehogs SET
custom = '"favorite_food" => "lemons", "weight" => "2lbs"'
WHERE id = 1;
Be careful of quoting.
40
![Page 41: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/41.jpg)
Hstore Operators
Common functions and operators:
• defined(A, B) Does A have B?
• A -> B Get B from A. In ruby this would be A[B]
41
![Page 42: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/42.jpg)
Query Hstore in SQL
Find all the favorite foods of the hedgehogs:
SELECT name, custom -> 'favorite_food' AS food
FROM hedgehogs WHERE defined(custom, 'favorite_food');
defined(A, B) Does A have B?
A -> B Get B from A. In ruby this would be A[B]
42
![Page 43: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/43.jpg)
Query Hstore in SQL
name food
Horrace lemons
Quilby pasta
Thomas grubs
43
![Page 44: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/44.jpg)
Query Hstore in ActiveRecord
Create scopes to make querying easier:
class Hedgehog < ActiveRecord::Base
scope :has_key, -> (key){ where('defined(custom, ?)', key) }
scope :has_value, -> (key, value){ where('custom -> ? = ?', key, value) }
...
44
![Page 45: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/45.jpg)
Query Hstore in ActiveRecord
Find hedgehogs with a custom color:
Hedgehog.has_key('color')
45
![Page 46: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/46.jpg)
Query Hstore in ActiveRecord
Find hedgehogs that are brown:
Hedgehog.has_value('color', 'brown')
46
![Page 47: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/47.jpg)
Query Hstore in ActiveRecord
Find all the silly, brown, hedgehogs:
Hedgehog.any_tags('silly').has_value('color', 'brown')
47
![Page 48: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/48.jpg)
Updating an Hstore with ActiveRecord
With ActiveRecord 4.x, hstore columns are just hashes:
hedgehog.custom["favorite_color"] = "ochre"
hedgehog.custom = {favorite_food: "Peanuts", shoe_size: 3}
48
![Page 49: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/49.jpg)
Heads Up
Hstore columns are always stored as strings:
hedgehog.custom["weight"] = 3
hedgehog.save!
hedgehog.reload
hedgehog.custom['weight'].class #=> String
49
![Page 50: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/50.jpg)
Someone commented on my hedgehog. They said they enjoy his beady little eyes,
but I can't find it.
hogmama73
Full Text Search
Your users want to be able to search within their comments.
“
50
![Page 51: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/51.jpg)
Full Text Search in SQL
CREATE TABLE comments (
id integer primary key,
hedgehog_id integer,
body text
);
51
![Page 52: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/52.jpg)
Full Text Search Data Types
There are two important data types:
• tsvector represents the text to be searched
• tsquery represents the search query
52
![Page 53: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/53.jpg)
Full Text Search Functions
There are two main functions that convert strings into these types:
• to_tsvector(configuration, text) creates a normalized tsvector
• to_tsquery(configuration, text) creates a normalized tsquery
53
![Page 54: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/54.jpg)
Full Text Search Normalization
Postgres removes common stop words:
select to_tsvector('A boy and his hedgehog went to Portland');
-- boy, hedgehog, portland, went
select to_tsvector('I need a second line to fill space here.');
-- fill, line, need, second, space
54
![Page 55: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/55.jpg)
Full Text Search Normalization
Stemming removes common endings from words:
term stemmed
hedgehogs hedgehog
enjoying enjoy
piping pipe
55
![Page 56: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/56.jpg)
Full Text Search Operators
Vectors:
• V @@ Q Searches V for Q
Queries:
• V @@ (A && B) Searches V for A and B
• V @@ (A || B) Searches V for A or B
56
![Page 57: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/57.jpg)
Full Text Search Querying
Find comments about "enjoying" something:
SELECT body
FROM comments
WHERE to_tsvector('english', body)
@@ to_tsquery('english','enjoying');
V @@ Q Searches V for Q
57
![Page 58: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/58.jpg)
Full Text Search Querying
• Does he enjoy beets? Mine loves them
• I really enjoy oranges
• I am enjoying these photos of your hedgehog's beady little eyes
• Can I feed him grapes? I think he enjoys them.
Notice how "enjoying" also matched "enjoy" and "enjoys" due to stemming.
58
![Page 59: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/59.jpg)
Full Text Search Wildcards
• to_tsquery('english','cat:*') Searches for anything starting with cat
Such as: cat, catapult, cataclysmic.
But not: octocat, scatter, prognosticate
59
![Page 60: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/60.jpg)
Full Text Search Wild Cards
Find comments containing the term "oil", and a word starting with "quil" :
SELECT body
FROM comments
WHERE to_tsvector('english', body)
@@ ( to_tsquery('english','oil')
&& to_tsquery('english','quil:*')
);
V @@ (A && B) Searches V for A and B
60
![Page 61: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/61.jpg)
Full Text Search Querying
• What brand of oil do you use? Have you tried QuillSwill?
61
![Page 62: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/62.jpg)
Heads Up
tsquery only supports wildcards at the end of a term.
While quill:* will match "QuillSwill", but *:swill will not.
In fact, *:swill will throw an error.
62
![Page 63: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/63.jpg)
Even More Heads Up!
Never pass user input directly to to_tsquery, it has a strict mini search syntax. The
following all fail:
• http://localhost : has a special meaning
• O'Reilly's Books Paired quotes cannot be in the middle
• A && B & and | are used for combining terms
You need to sanitize queries, or use a gem that does this for you.
63
![Page 64: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/64.jpg)
Full Text Search With ActiveRecord
We can wrap this up in a scope.
class Comment < ActiveRecord::Base
scope :search_all, -> (query){
where("to_tsvector('english', body) @@ #{sanitize_query(query)}")
}
You need to write sanitize_query, or use a gem that does this for you.
64
![Page 65: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/65.jpg)
Full Text Search With ActiveRecord
Find the comments about quill oil again, and limit it to 5 results:
Comment.search_all("quil* oil").limit(5)
Since search_all is a scope, we chain it like all the other examples.
65
![Page 66: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/66.jpg)
Full Text Search Indexing
Create an index on the function call to_tsvector('english', body):
CREATE INDEX comments_gin_index
ON comments
USING gin(to_tsvector('english', body));
The gin index is a special index for multivalued columns like a text[] or a
tsvector
66
![Page 67: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/67.jpg)
Heads Up
Since we are indexing a function call, to_tsvector('english', body), we must
call it the same way every time.
You don't have to use english, but you do need to be consistent.
67
![Page 68: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/68.jpg)
In Summary
• Arrays can model tagging and hierarchies
• Hstore can be used to model custom data
• Postgres supports full text search
You can now enjoy the happy hour!
SELECT * FROM beers WHERE
traits @> ARRAY['hoppy', 'floral']
68
![Page 69: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/69.jpg)
Any Questions?
Possible suggestions:
• Why not normalize your database instead of using arrays?
• Can I see how you implemented sanitize_query?
• What is a good gem for full text search?
• What about ActiveRecord 2 and 3?
• Why hstore instead of JSON?
• Can I buy you coffee?
69
![Page 70: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/70.jpg)
Extra Resources
• ActiveRecord Queries & Scopes
• Postgres Array Operators
• Postgres Hstore Documentation
• Postgres Full Text Search
• Ruby Gems for Full Text Search
• Textacular Supports Active Record 2.x and 3.x
• pg_search Supports Active Record 3.x, but has more features
• My Blog, Github, and favorite social network
• How to draw a hedgehog.70
![Page 71: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/71.jpg)
Bonus
Here's sanitize_query:
def self.sanitize_query(query, conjunction=' && ')
"(" + tokenize_query(query).map{|t| term(t)}.join(conjunction) + ")"
end
It breaks up the user's request into terms, and then joins them together.
71
![Page 72: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/72.jpg)
Bonus
We tokenize by splitting on white space, &, |, and :.
def self.tokenize_query(query)
query.split(/(\s|[&|:])+/)
end
72
![Page 73: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/73.jpg)
Bonus
Each of those tokens gets rewritten:
def self.term(t)
# Strip leading apostrophes, they are never legal, "'ok" becomes "ok"
t = t.gsub(/̂'+/,'')
# Strip any *s that are not at the end of the term
t = t.gsub(/\*[̂$]/,'')
# Rewrite "sear*" as "sear:*" to support wildcard matching on terms
t = t.gsub(/\*$/,':*')
... 73
![Page 74: Postgres the best tool you're already using](https://reader034.fdocuments.net/reader034/viewer/2022052522/554c4b82b4c90570648b5695/html5/thumbnails/74.jpg)
...
# If the only remaining text is a wildcard, return an empty string
t = "" if t.match(/̂[:* ]+$/)
"to_tsquery('english', #{quote_value t})"
end
74