The QS

How did we build the new query system?

Development Team

Andrew Gilmartin, Chuck Koscher, Evans Lin, Hisham Shahtout

1. Interesting parts of the QS

2. Experiences building the QS

3. Lessons learned for the DS

Tools

modern, but not too modern

NetBeans 6.[89]

technique sharing, tool building

Java 6

generics, annotations, oh my!

Tomcat 5.5

servlets are components too*

HAProxy 1.4

http://haproxy.1wt.eu/

Oracle && MySQL

valuable || expendable

ActiveMQ 5.4

wicked fast loose coupling

Spring 3.0

Really?

Spring || EJB || ???

reaction, refinement, replacement, re-implement, rejoice

POJOs || Deployment Descriptors

i just ( do || describe ) objects

Dependency Injection || Coded Configuration

more than one main()

Functional Interfaces || Featureful Interfaces

a little well done is enough

Spring Experiences

some tales are told a little later

Design

and not doing architecture

Controllers, Services && Coordination

insides and outsides

“Where is this article?”!

“Does this DOI have meta-data?”!

!"!!#$%&'!'&(%)*!"#$%&'!()*!++*!!!+!!!!!!,-&..)*/012,0/..0%324.2,/'50/--%0.2,-./*0$0*)12#$%&'/)10%)33#%*6!!7!!!!!!#80/8%059!'&(%)*,:5&5:/';%&0,<=//0>:'&5/0*!0%3)*&)44)1/*0$0*)12#$%&'/))%(*1$0)%*?6!!@!!!!!!222!!A!!#?$%&'6!!B!!#$%&'!'&(%)*!"#%53#0!67#%8!++*!!!C!!!!!!,-&..)*/012,0/..0%324.2,/'50/--%0.297#%8/)10%)33#%*6!!D!!!!!!#80/8%059!'&(%)*,:5&5:/';%&0,<=//0>:'&5/0*!0%3)*&)44)1/*0$0*)12#$%&'/))%(*1$0)%*?6!!E!!!!!!222!"F!!#?$%&'6!

""!!#$%&'!'&(%)*&)44)1/*0$0*)12#$%&'/))%(*1$0)%*!!"+!!!!!!!!,-&..)*/012,0/..0%324.2,:5&5:/'.%&0,<2/*0$0*)12#$%&'/))%(*1$0)%*6!"7!!!!!!222!"@!!#?$%&'6!

"A!!#$%&'!'&(%)*()*/*0$0*)197#%82#%5*&#*!!"B!!!!!!!!,-&..)*/012,0/..0%324.2>/:,:5&5:/'.%&0,<2,)*/*0$0*)197#%82#%5*&#*6!"C!!!!!!#80/8%059!'&(%)*4G%09;%0H:,%I%1:.509*!0%3)*&)44)1/*0$0*)12#$%&'/))%(*1$0)%*?6!"D!!!!!!222!!"E!!#?$%&'6!+F!!#$%&'!'&(%)*4#0$,$0$/*0$0*)197#%82#%5*&#*!!+"!!!!!!!!,-&..)*/012,0/..0%324.2(%5&>&5&,:5&5:/'.%&0,<2:#0$,$0$/*0$0*)197#%82#%5*&#*6!++!!!!!!#80/8%059!'&(%)*4G%09;%0H:,%I%1:.509*!0%3)*&)44)1/*0$0*)12#$%&'/))%(*1$0)%*!?6!+7!!!!!!222!+@!!#?$%&'6!

Service Engine

actual heavy lifting

Management

x-rays, JMX

Health

…, lub-dub, lub-dub, …

8G$-:,!:'5%03&,%!J%&05$%&5KL%&'!M!

!!!!$//-%&'!:.L%&5:'1NOP!Q!

<558R??,0"F2,0/..0%32/01?<%&05$%&5?>/:S,:5&5:/'S.%&0,<!

<558R??,0"F2,0/..0%32/01?<%&05$%&5?(%5&S>&5&S,:5&5:/'S.%&0,<!

!"!!#$%&'!!!+!!!!!!'&(%)*!'#$%0;#$0!++*!!!7!!!!!!,-&..)*/012,0/..0%324.2,/'50/--%0.2J%&05$%&5=/'50/--%0*6!!@!!!!#80/8%059!'&(%)*<%&05$%&5.*6!!A!!!!!!#(&86!!B!!!!!!!!#%'509!!!C!!!!!!!!!!!!T%9)*()*<&*0$0*)1<"#$%&'*!!!D!!!!!!!!!!!!H&-G%S0%3)*>/:=:5&5:/';%&0,<J%&-5<J%&05$%&5*!?6!!E!!!!!!!!#%'509!!"F!!!!!!!!!!!!T%9)*4#0$<($0$<&*0$0*)1<"#$%&'*!!""!!!!!!!!!!!!H&-G%S0%3)*(%5&U&5&=:5&5:/';%&0,<J%&-5<J%&05$%&5*!?6!"+!!!!!!!!222!"7!!!!!!#?(&86!"@!!!!#?80/8%0596!"A!!#?$%&'6!

Artifacts

secondary (lossy) data flow(s)

8G$-:,!:'5%03&,%!V05:3&,5!%W5%'>.!;%0:&-:X&$-%!M!

Q!

8G$-:,!:'5%03&,%!V05:3&,5;%0H:,%!M!

!!!!8G$-:,!H/:>!,/'50:$G5%V05:3&,5N!V05:3&,5!&05:3&,5!OP!Q!

Interim Storage

bigger response, expiration

<558R??,0/..0%32/01?0%50:%H%?Y7VCU="LSL"@ES@DC+SE@"DSE+FBBA+77E7=!

!"!!8G$-:,!:'5%03&,%!;5/0%!M!!+!!7!!!!$//-%&'!,0%&5%N!!@!!!!!!ZZ[U!77*(\!!A!!!!!!-/'1!>%8/.:5/0[>\!!B!!!!!!-/'1!0%,:8:%'5[>\!!C!!!!!!]W8:0&5:/'^/-:,9!%W8:0&5:/'^/-:,9\!!D!!!!!!;50:'1!,/'5%'5_98%\!!E!!!!!!.1=7020%#$4>,/'5%'5['8G5;50%&(O!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P!"F!""!!!!-70=7020%#$4>,0%&5%N!"+!!!!!!ZZ[U!77*(\!"7!!!!!!-/'1!>%8/.:5/0[>\!"@!!!!!!-/'1!0%,:8:%'5[>\!"A!!!!!!]W8:0&5:/'^/-:,9!%W8:0&5:/'^/-:,9\!"B!!!!!!;50:'1!,/'5%'5_98%O!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P!"C!"D!!!!a:.5#[5%(6!3:'>V--NO!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P!"E!+F!!!!.0#4>3:'>L9ZG:>NZZ[U!GG:>O!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P!+"!++!!!!H/:>!8G01%]W8:0%>[5%(.=/'5%'5NO!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P!+7!+@!!!!H/:>!,-/.%NO!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P!+A!!Q!

!"!!8G$-:,!:'5%03&,%![5%(!M!!+!!!!7!!!!8G$-:,!ZZ[U!1%5ZG:>NOP!!@!!!!A!!!!8G$-:,!_:(%.5&(8!1%5U%8/.:5%>_:(%.5&(8NOP!!B!!C!!!!8G$-:,!-/'1!1%5U%8/.:5/0[>NOP!!D!!E!!!!8G$-:,!-/'1!1%5I%,:8:%'5[>NOP!"F!""!!!!8G$-:,!;50:'1!1%5=/'5%'5_98%NOP!"+!"7!!!!8G$-:,!['8G5;50%&(!1%5=/'5%'5NO!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P!"@!"A!!!!8G$-:,!]W8:0&5:/'^/-:,9!1%5]W8:0&5:/'^/-:,9NOP!"B!"C!!!!8G$-:,!$//-%&'!:.]W8:0%>NOP!"D!"E!!!!8G$-:,!_:(%.5&(8!1%5]W8:0%>_:(%.5&(8NOP!+"!++!!Q!

Notifications

urgency, ( now || later ), aggregates

!"!!8G$-:,!:'5%03&,%!b/5:3:,&5:/';%0H:,%!M!!+!!!!7!!!!!!H/:>!'/5:39Nb/5:3:,&5:/'!'/5:3:,&5:/'OP!!@!!!!A!!!!!!b/5:3:,&5:/'!1%5b/5:3:,&5:/'NZZ[U!50&,T:'1[>OP!!B!!!!C!!!!!!b/5:3:,&5:/';5&5%!1%5b/5:3:,&5:/';5&5%NZZ[U!50&,T:'1[>OP!!D!!!!E!!!!!!b/5:3:,&5:/'!0%(/H%b/5:3:,&5:/'NZZ[U!50&,T:'1[>OP!"F!!Q!

!"!!8G$-:,!:'5%03&,%!b/5:3:,&5:/'!M!!+!!!!7!!!!!!ZZ[U!1%5_0&,T:'1[>NOP!!@!!!!A!!!!!!$//-%&'!:.U%-:H%09Z01%'5NOP!!B!!!!C!!!!!![>%'5:59!1%5;%'>%0NOP!!D!!!!E!!!!!!;%5#[>%'5:596!1%5I%,:8:%'5.NOP!"F!!!""!!!!!!$//-%&'!<&.U%-:H%09V110%1&5:/'b&(%NOP!"+!!!!!!;50:'1!1%5U%-:H%09V110%1&5:/'b&(%NOP!"7!!!"@!!!!!!$//-%&'!:.U%-:H%09V,T'/`-%>1%>NOP!"A!!!"B!!!!!!$//-%&'!:.U%-:H%09]W8:0&5:/'V,T'/`-%>1%>NOP!"C!!!"D!!!!!!;50:'1!1%5;G$c%,5NOP!"E!!!+F!!!!!!$//-%&'!<&.K%..&1%NOP!+"!!!!!!;50:'1!1%5K%..&1%=/'5%'5NOP!++!!!!!!;50:'1!1%5K%..&1%=/'5%'5_98%NOP!+7!!!+@!!!!!!$//-%&'!<&.['5%0:(;5/0&1%ZZ[U.NOP!+A!!!!!!;%5#ZZ[U6!1%5['5%0:(;5/0&1%ZZ[U.NOP!+B!!!+C!!!!!!$//-%&'!<&.]W8:0&5:/'_:(%.5&(8NOP!+D!!!!!!_:(%.5&(8!1%5]W8:0&5:/'_:(%.5&(8NOP!+E!!Q!

!"!!8G$-:,!:'5%03&,%!b/5:3:,&5:/';5&5%!M!!+!!!!7!!!!!!ZZ[U!1%5_0&,T:'1[>NOP!!@!!!!A!!!!!!$//-%&'!:.U%-:H%09^%'>:'1NOP!!B!!!!C!!!!!!$//-%&'!:.U%-:H%0%>NOP!!D!!!!E!!!!!!$//-%&'!:.]W8:0%>NOP!"F!!!""!!!!!!$//-%&'!:.I%(/H%>NOP!"+!!!"7!!!!!!_:(%.5&(8!1%5_:(%.5&(8NO!5<0/`.![--%1&-;5&5%]W,%85:/'P!"@!!!"A!!!!!!K&8#[>%'5:59\!;%5#ZZ[U66!1%5U%-:H%09V110%1&5%NO!5<0/`.!222P!"B!!Q!

Spring Experiences

Coming up to speed

books are useful … for about 5 hours

Is that answer for Spring 1.x || 2.x || 3.x?

rummaging around the net

Deployment Descriptors

they’re back and now called contexts

Spring Beans && Java Properties

Where do I defined this thing?

XML

XSDs, id(s) || name(s), conditional sections

Runtime Configuration Errors

scripting without the support tools

Bondage

bound to Spring as much as you are to any framework

It might be there

but not designed as expected

Development lessons from the QS to apply

with the DS

Undervalue your own ideas

the view from the backside of the leading edge

Existing wheels are generally well designed

use code libraries … which are sometimes your own

Use the API correctly

150/second to 2000/second

Stopping buck-passing

artifacts, costs, choices

A does not follow B right away

synchronization and the ReadWriteLock are your friends

Pick a monitoring tool as-soon-as-possible

instills design for management and monitoring

The data is already stale

design data for management and convert data for use

Priming takes a long time

pool XML parsers

Data conversion takes a longer time

break it up into multiple parts

Spring Contexts need support tools

we have one, we need more

Design small interfaces

federation of cliques

Front-Controller vs Servlets

I want my componentization back

Keep talking

especially with distant developers

:)

agilmartin@crossref.org