EFSanFrancisco2013-MongooseIM

7/23/2019 EFSanFrancisco2013-MongooseIM

1/54

1999-2013 Erlang Solutions Ltd.

MongooseIM - Messaging that ScalesMicha!"laski

Monday, 8 April 13


2/54


What is MongooseIM?

2

Monday, 8 April 13


3/54


What is MongooseIM?

Instant messagingfor Social Media, Gaming andTelecommunications

2

Monday, 8 April 13


4/54


What is MongooseIM?


Solution designed for high volume

2

Monday, 8 April 13


5/54


What is MongooseIM?



Easily scalabledistributed system

2

Monday, 8 April 13


6/54


What is MongooseIM?



Easily scalabledistributed system

Highly customisableplatform thanks to industrystandards and Open Source technology

2

Monday, 8 April 13


7/54


Target domains

3

Monday, 8 April 13


8/54


Target domains

3

Telecoms

(chat features are a part of

standard commmunication

software)

Monday, 8 April 13


9/54


Target domains

3

Telecoms



software)

Social Media

(messaging is the key part of

social web sites)

Monday, 8 April 13


10/54


Target domains

3

Telecoms



software)

Social Media

(messaging is the key part of

social web sites)

Gaming(multi-user chats are an integral

part of the gameplay and

enhance the user experience)

Monday, 8 April 13


11/54


Easy to work with

Support of XMPP

- a protocol used for Facebook chat and Google Talk

- seamless integration with many libraries and clientapplications

4

Monday, 8 April 13


12/54


XMPP Properties

Open standard

Secure

Flexible

Decentralised

Efficient

Proven

5

Monday, 8 April 13


13/54


XMPP Properties

Open standard

Secure

Flexible

Decentralised

Efficient

Proven

5

Standard formalised by the IETF through RFCsand extensions are published by the XSF

Monday, 8 April 13


14/54


XMPP Properties

Open standard

Secure

Flexible

Decentralised

Efficient

Proven

5

Channel encryption, strong authentication, may beisolated from the public network

Monday, 8 April 13


15/54


XMPP Properties

Open standard

Secure

Flexible

Decentralised

Efficient

Proven

5

Custom functionality can be built on top of XMPP

Monday, 8 April 13


16/54


XMPP Properties

Open standard

Secure

Flexible

Decentralised

Efficient

Proven

5

Anyone can run XMPP serverand there is no central master server

Monday, 8 April 13


17/54


XMPP Properties

Open standard

Secure

Flexible

Decentralised

Efficient

Proven

5

Solves issues of polling approaches

Monday, 8 April 13


18/54


XMPP Properties

Open standard

Secure

Flexible

Decentralised

Efficient

Proven

5

In use since 1998, many implementations anddeployments, millions of end users

Monday, 8 April 13


19/54


XMPP Properties

Open standard

Secure

Flexible

Decentralised

Efficient

Proven

5

Monday, 8 April 13


20/54


Features

Support for WebSockets- ready for modern chat-enabled web applications

Multi-User Chat(XEP-0045)

Service Discovery (XEP-0030)

vCard (XEP-0054)

Privacy Lists (XEP-0016)

Private XML Storage (XEP-0049)

6

Monday, 8 April 13


21/54


Extensible

Customisable - provides a base for bespokesolution to ones specific needs

- push notifications to mobile devices

- authentication plugins

- reliable message delivery in mobile networks

Has open sourced regression tests

- github.com/esl/ejabberd_tests

7

Monday, 8 April 13


22/54


Scalable

Configurable database backends

- Mnesia for simple deployments

- MySQL for persistent data

- Mnesia or Redis for transient data

8

Monday, 8 April 13


23/54


Scalable

9

Monday, 8 April 13


24/54


Load tests - environment

Test environmentErlang/OTP R15B02

MongooseIM 1.1

MySQL 5.5.24Ubuntu 12.04 LTS 3.2.0-23-generic

Tsung 1.5.0a

redis 2.6.10

ejabberd 2.1.11

10

Monday, 8 April 13


25/54


Load tests - dedicated box

11

MongooseIM

AMD 8x3.3GHz, 32GB RAM

MySQL

Monday, 8 April 13


26/54


Load tests - max users

User arrival rate: 150/s

User count: 400k

Roster size: 100

No message exchange

The aim was to check resource usage with maximumpossible count of online users connected to server.

12

Monday, 8 April 13


27/54



13

Monday, 8 April 13


28/54



14

Monday, 8 April 13


29/54


Load tests - max message rate


User count: 75k

Roster size: 100

Message rate: ~21k per second

The aim was to check resource usage when maximummessage traffic is generated.

15

Monday, 8 April 13


30/54



16

Monday, 8 April 13


31/54



17

Monday, 8 April 13


32/54


Load tests - Amazon EC2

M1 Extra Large Instance

- 15 GiB memory

- 8 EC2 Compute Units

(4 virtual cores with 2 EC2 Compute Units each)- I/O Performance: High

18

Monday, 8 April 13


33/54



19

MongooseIM

m1.xlarge

MongooseIM

m1.xlarge

MongooseIMm1.xlarge

Monday, 8 April 13


34/54


Load tests - 3 MongooseIM + Mnesia


User count: 50k

Roster size: 100

Presence rate: 120 per second (12k msg/s)

The aim was to check resource usage with all user datalike passwords and rosters in Mnesia.

20

Monday, 8 April 13


35/54

1999-2013 Erlang Solutions Ltd. 21

!

#$!

%$!

&$!

'$!

($!

)$!

*$!

+$!

,$!

#$$!

#

%&

'(

)*

+,

###

#&&

#((

#**

#,,

%%#

%'&

%)(

%+*

&$,

&

&(&

&*(

&,*

'#,

''#

')&

'+(

($*

(%,

((#

(*&

(,(

)#*

)&,

))#

)+&

*$(

*%*

*',

**#

*,&

+#(

+&*

+(,

++#

,$&

,%(

,'*

,),

,,#

#$#&

#$&(

#$(*

#$*,

##$#

##%&

##'(

##)*

##+,

#%##

#%&&

#%((

#%**

#%,,

#&%#

#&'&

#&)(

#&+*

!"#

#%

&'()%*+,-.

/'01 ,2.

-.- # -.- % -.- &

Monday, 8 April 13


36/54


!"!!

$%!!!%!!!"!!

&%!!!%!!!"!!

'%!!!%!!!"!!

(%!!!%!!!"!!

)!%!!!%!!!"!!

)$%!!!%!!!"!!

)&%!!!%!!!"!!

)'%!!!%!!!"!!

)

$'

*)

+'

)!)

)$'

)*)

)+'

$!)

$$'

$*)

$+'

,!)

,$'

,*)

,+'

&!)

&$'

&*)

&+'

*!)

*$'

**)

*+'

'!)

'$'

'*)

'+'

+!)

+$'

+*)

++'

(!)

($'

(*)

(+'

-!)

-$'

-*)

-+'

)!!)

)!$'

)!*)

)!+'

))!)

))$'

))*)

))+'

)$!)

)$$'

)$*)

)$+'

),!)

),$'

),*)

),+'

!"##%#%&"'

()*+

,-%# (.+

./. ) ./. $ ./. ,

Monday, 8 April 13


37/54



23

MySQLm1.xlarge

MongooseIM

m1.xlarge

MongooseIMm1.xlarge

MongooseIMm1.xlarge

Monday, 8 April 13


38/54


Load tests - 3 MongooseIM + MySQL


User count: 330k

Roster size: 100

Presence rate: 120 per second (12k msg/s)

The aim was to check resource usage with all user datalike passwords and rosters in MySQL

24

Monday, 8 April 13


39/54


!

#$!

%$!

&$!

'$!

($!

)$!

*$!

+$!

,$!

#$$!

#

)%

#%&

#+'

%'(

&$)

&)*

'%+

'+,

(($

)##

)*%

*&&

*,'

+((

,#)

,**

#$&+

#$,,

##)$

#%%#

#%+%

#&'&

#'$'

#')(

#(%)

#(+*

#)'+

#*$,

#**$

#+

#+,%

#,(&

%$#'

%$*(

%#&)

%#,*

%%(+

%,

%&+$

%''#

%($%

%()&

%)%'

%)+(

%*')

%+$*

%+)+

%,%,

%,,$

&$(#

#%

*&

&%&'

&%,(

&&()

&'#*

&'*+

&(&,

&)$$

&))#

&*%%

&*+&

&+''

&,$(

&,))

!"##%&'()%*+,-.

/'01 ,2.

-.- # -/- % -/- &

Monday, 8 April 13


40/54


!"!!

$%!!!%!!!"!!

&%!!!%!!!"!!

'%!!!%!!!"!!

(%!!!%!!!"!!

)!%!!!%!!!"!!

)$%!!!%!!!"!!

)&%!!!%!!!"!!

)'%!!!%!!!"!!

)

'*

)

$+

)

+,

$

*-

,

$)

,

(*

&

&+

*

),

*

--

'

&)

-

!*

-

'+

(

,,

(

+-

+

')

)!

$*

)!

(+

))*,

)$

)-

)$

()

),

&*

)&

!+

)&-,

)*

,-

)'

!)

)'

'*

)-

$+

)-+,

)(

*-

)+

$)

)+

(*

$!

&+

$)),

$)

--

$$

&)

$,

!*

$,

'+

$&,,

$&

+-

$*

')

$'

$*

$'

(+

$-*,

$(

)-

$(

()

$+

&*

,!

!+

,!-,

,)

,-

,$

!)

,$

'*

,,

$+

,,+,

,&

*-

,*

$)

,*

(*

,'

&+

,-),

,-

--

,(

&)

,+

!*

,+

'+

!"##%#%&"'()*+

,-%# (.+

./. ) ./. $ ./. ,

Monday, 8 April 13


41/54



27

redism1.xlarge

MongooseIMm1.xlarge

MongooseIMm1.xlarge

MongooseIMm1.xlarge

MySQLm1.xlarge

Monday, 8 April 13


42/54


Load tests - 3 MIM + redis + MySQL


User count: 380k

Message rate: 8k per second

The aim was to check characteristics of a clusterof 3 MongooseIM nodes with session data in redisinstead of mnesia.

28

Monday, 8 April 13


43/54


Monday, 8 April 13


44/54


Monday, 8 April 13


45/54

31

ooVoo is the largest independent video

communication service provider

70mm users

Free, up to 12way high-quality service. Leader in group calling

Apps for PC. Mac. Web. iPhone. iPad. Android. Facebook

10+ billion video minutes in 2012

Platform for Shared Experiences

courtesy of Alex Fok, System Architect at ooVoo

Monday, 8 April 13


46/54

32

Main Challenge - Scale

2010

600K online users

10M registered users

500 messages/sec

Due to constant and rapid usage grow infinite scale requirement


Monday, 8 April 13


47/54

33

Today (2 years after )

11 scale units in 2 geographical locations

x10 Usage growth

2.2M online users (connected)

70M registered users

5,000+ messages per second

99.98% uptime


Monday, 8 April 13


48/54

34


Monday, 8 April 13


49/54

35


Monday, 8 April 13


50/54


Benefits

Proven solution from an Erlang-focused companywith expertise in instant messaging (over 300clients since 1999)

Rapid deployment Interoperability ensured by industry standards

Minimal capital expenditure due to efficientruntime platform

Pluggable authentication for ease of integration

36

Monday, 8 April 13


51/54


MongooseIM

Downloadwww.erlang-solutions.com/downloads/

Fork and contributehttps://github.com/esl/ejabberd

Contact [email protected]

37

Monday, 8 April 13


52/54


Load tests - WebSockets vs. BOSH


User count: 10k

Message rate: 6k/s

The aim was to compare resource usageof WebSockets vs. BOSH.

38

Monday, 8 April 13


53/54



39

Monday, 8 April 13


54/54

1999 2013 E l S l ti Ltd


40

EFSanFrancisco2013-MongooseIM

Documents

Transcript of EFSanFrancisco2013-MongooseIM