Post on 09-Mar-2021
WiredTiger Backend forOpenLDAP
Open Source Solution Technology CorporationTsukasa Hamano <hamano@osstech.co.jp>
LDAPCon 2015 Edinburgh November 2015
Open Source Solution Technology Corporation 1
WiredTiger Backend for OpenLDAP
About OSSTech
• ID Management leading company inJapan.
• Storage Solution• Open Source Contribution
Open Source Solution Technology Corporation 2
WiredTiger Backend for OpenLDAP
What’s back-wt
• New OpenLDAP Backend• WiredTiger Database
Open Source Solution Technology Corporation 3
WiredTiger Backend for OpenLDAP
About WiredTiger
• Embedded database• High performance• High scalability
Open Source Solution Technology Corporation 4
WiredTiger Backend for OpenLDAP
Lock Free
• Hazard pointer• Optimistic concurrency control
Open Source Solution Technology Corporation 5
WiredTiger Backend for OpenLDAP
Data Structure
Open Source Solution Technology Corporation 6
WiredTiger Backend for OpenLDAP
bdb_next_id()� �int bdb_next_id( BackendDB *be, ID *out )
{
struct bdb_info *bdb=(struct bdb_info*)be->be_private;
ldap_pvt_thread_mutex_lock(&bdb->bi_lastid_mutex);
*out = ++bdb->bi_lastid;
ldap_pvt_thread_mutex_unlock(&bdb->bi_lastid_mutex);
return 0;
}� �Open Source Solution Technology Corporation 7
WiredTiger Backend for OpenLDAP
wt_next_id()� �int wt_next_id(BackendDB *be, ID *out){
struct wt_info *wi = (struct wt_info *)be->be_private;
*out = __sync_add_and_fetch(&wi->wi_lastid, 1);
return 0;
}� �
Open Source Solution Technology Corporation 8
WiredTiger Backend for OpenLDAP
fsync(2) is slow
Open Source Solution Technology Corporation 9
WiredTiger Backend for OpenLDAP
Durability Levels
1. in-memory txn log – fastest but nodurability
2. write txn log file, no sync3. write txn log file, sync per every
commit
Open Source Solution Technology Corporation 10
WiredTiger Backend for OpenLDAP
New Benchmark Tool - lb
• SLAMD is dead• Command line interface• Written in Go
Open Source Solution Technology Corporation 11
WiredTiger Backend for OpenLDAP
Installation of lb
$ export GOPATH=~/go
$ go get github.com/hamano/lb
Open Source Solution Technology Corporation 12
WiredTiger Backend for OpenLDAP
Benchmark Environment
• 12 Core CPU• No RAID Card• SAS Disk
Open Source Solution Technology Corporation 13
WiredTiger Backend for OpenLDAP
ADD Benchmark Script
� �for c in 1 2 4 8 16 32 64 128 256 512; do
lb add -c $c -n 10000 --uuid ldap://targethost/
done� �
Open Source Solution Technology Corporation 14
WiredTiger Backend for OpenLDAP
ADD (nosync) Benchmarks
0
5000
10000
15000
20000
25000
30000
1 2 4 8 16 32 64 128 256 512
Requests
per
second
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 15
WiredTiger Backend for OpenLDAP
ADD (sync) Benchmarks
0
1000
2000
3000
4000
5000
6000
1 2 4 8 16 32 64 128 256 512
Requests
per
second
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 16
WiredTiger Backend for OpenLDAP
BIND Benchmark Script� �for c in 1 2 4 8 16 32 64 128 256 512; do
lb bind -c $c -n 100000 \
-D "cn=user%d,dc=example,dc=com" -w secret \
--last 10000 ldap://targethost/
done� �
Open Source Solution Technology Corporation 17
WiredTiger Backend for OpenLDAP
BIND Benchmark Result
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
1 2 4 8 16 32 64 128 256 512
Requests
per
second
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 18
WiredTiger Backend for OpenLDAP
SEARCH Benchmark Script� �for c in 1 2 4 8 16 32 64 128 256 512; do
lb search -c $c -n 100000 \
-a "(cn=user%d)" \
--last 10000 ldap://targethost/
done� �
Open Source Solution Technology Corporation 19
WiredTiger Backend for OpenLDAP
SEARCH Benchmark Result
0
5000
10000
15000
20000
25000
30000
1 2 4 8 16 32 64 128 256 512
Requests
per
second
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 20
WiredTiger Backend for OpenLDAP
Tests
$ make -C tests wt
5465 Succeed
Open Source Solution Technology Corporation 21
WiredTiger Backend for OpenLDAP
Tasks
• Hot-backup• alias and glue entry
Open Source Solution Technology Corporation 22
WiredTiger Backend for OpenLDAP
Thank You!
Open Source Solution Technology Corporation 23