Batch/Queue APIWhat we can learn from Drupal 7 for CiviCRM 4.1

About myselfChang Xiao

Emotive LLC

chang@emotivellc.com

IRC: changx

Forum handle: xcf33

Batch. Yum….A Batch of cookies

Characteristics of BatchWorks on right after the series of action desired

to be performed is submitted.

Provide feedbacks to user on completion status.

Example: 3 records imported, etc

etc

Submit import form Import user 1

Import user 2

Import user 3

BenefitsProvide user feed back on the progress.

(Usually user take it for granted).

Good and smooth user experience.

Preventing user errors.

Potential PitfallsWaiting, Waiting and Waiting.

PHP settings (timeout, memory, etc)

Database settings (MySQL has gone away)

Conclusion:Not suitable for large scale operations.

Queue, errrA line on Black Friday

Characteristics of QueueFirst in First out (FIFO)

Processes a series of tasks in sequence

Example:

3 records imported,

etc etc

Submit import form

Imp

ort

user1 im

port

user2 im

port

user3

Import Queue

BenefitsCan be stopped and resumed at any time.

Provides granular scalability and robustness.

Separate user input and actual process

Potential PitfallsComplex design

Resource consumption

Current CiviCRM applicationsBulk mailing

Each mailing is in a queue. Each mailing contains X number of emails and sits

in a mailing queue

Contact Import

ETC

When to use?Batch:

Tasks that will exceed php time out Tasks with tolerable time frame (i.e. 5 minutes) Tasks that needs to provide continuous feedback

Queue: Tasks that takes a lot of time Tasks that can be stopped and resumed Distribute processing.

Where to use itExporting contacts

Importing process

Running large reports

CiviMail bulk mailing process

Dedupe processes

Synchronizing processes

Installation/Upgrade

Conclusion Batch Queue

Why notHave your cookie and ….Eat it too?