Post on 10-Apr-2017
Preparing for System Failure:How Pearson used AppDynamics to Drive Improved Student Experience
Mike Jackson, VP of Software Engineering, Pearson
Tim Boberg, Sr. Manager of Application Engineering & DevOps, Pearson
Pearson is the world’s leading learning company
40,000 employees in over 70 countries
All working to help people of all ages to make measurable progress in their lives through all kinds of learning
We provide learning materials, technologies, assessments and services to teachers and students in order to help people everywhere aim higher and fulfill their potential.
We put the learner at the center of everything we do.
Who is Pearson?
Who is Pearson?
Pearson is the world’s leading learning company
40,000 employees in over 70 countries
All working to help people of all ages to make measurable progress in their lives through all kinds of learning
We provide learning materials, technologies, assessments and services to teachers and students in order to help people everywhere aim higher and fulfill their potential.
We put the learner at the center of everything we do.
The role of technology in education is expanding
87%of parents say
effective implementation of technology is important for
their child’s successSource: EdTechReview 2013
6.7mstudents are now taking
at least one online courseSource: Online Learning Consortium 2012
92%of educators rely on
internet content in the classroomSource: Tomorrow Project – Speak up survey 2013
Helping people make progress in their lives through learning
PEARSON DIGITAL TRANSFORMATION
6
Digital Applications at Pearson
Multiple platforms with disparate technology stacks- Microsoft, Oracle, NoSQL- .Net, Java, Opensource
Pearson Apps – XL Platform Only
Copyright © 2015 AppDynamics. All rights reserved. 7
1B+Pageview
sOver 2 months
1.3MDaily
Logins
40KConcurren
t users
8
Example App - MathXL
9
Preparing for system failure
“Enterprise software must be cynical. Cynical software expects bad things to happen and is never surprised when they do.”
Release It! Design and Deploy Production-Ready Software by Michael T Nygard
Copyright © 2015 AppDynamics. All rights reserved.10
11
Failure to prepare is preparing to fail
Surviving a failure - Think before you fail
Copyright © 2015 AppDynamics. All rights reserved.12
“A resilient system keeps processing transactions, even when there are transient impulses, persistent stresses, or component failures disrupting normal processing”
“A highly stable design usually costs the same to implement as the unstable one.”
- Release It! Design and Deploy Production-Ready Software by Michael T Nygard
The Exceptional Fall
13
The Monitoring Story
Copyright © 2015 AppDynamics. All rights reserved. 14
The Monitoring Story
Copyright © 2015 AppDynamics. All rights reserved. 15
Why AppDynamics?
- One ubiquitous tool that handles our disparate tech stacks
- Scalable Architecture
- We love all of the AppDynamics features:- System level monitoring and alerting- Business transaction monitoring and alerting- Call Stack visibility for slow or failing health checks- Visual representation of system
16
Even VP’s can use it !!
Partnership between Pearson and AppDynamics
Copyright © 2015 AppDynamics. All rights reserved. 17
Preparing for the Exceptional Fall
18
“AppDynamics turns our application into a living, breathing entity” Pearson Software Engineer
Preparing for the Exceptional Fall
19
Copyright © 2015 AppDynamics. All rights reserved. 20
Exceptional Results with AppDynamics
Preparing for success!!!
21
● Establish CoE○ Engage AppD Professional Services○ Ongoing Hands on Training
● Organize your applications● Instrument key business transactions● Eyes On The Glass – Watch Production● Action what you see
○ If you see it, say it● Continuous improvement
○ Keep current on controller and agent○ Evaluate key systems and transactions
Next Steps with AppDynamics & our initiative
Copyright © 2015 AppDynamics. All rights reserved. 22
Takeaways
• Success at scale requires standardization and preparation• Systems fail, Implement Stability Patterns• Adopt and Implement vibrant full stack Monitoring • Watch and Learn from Production • Avoid silos, work cross functionally• Train First Responders and DevOps Teams
Copyright © 2015 AppDynamics. All rights reserved. 23
Thank You
Agenda
Pearson Overview Preparing for system failureSurviving a system failure The Exceptional FallResults Takeaways
Copyright © 2015 AppDynamics. All rights reserved. 25
Preparing for system failure
“It is a fantasy to expect every single bug to be driven out. Bugs will happen. They cannot be eliminated, so they must be survived instead.”
- Release It! Design and Deploy Production-Ready Software by Michael T Nygard
Copyright © 2015 AppDynamics. All rights reserved. 26
Surviving a failure: Think before you failMemory: You can never have enough
Users consume resources
Common logging tools create objects to represent a log event. These objects require memory. If no memory is available to create a log event, these logs do not get created
This is a great argument for external monitoring in addition to log file scrapping
27
28
- Over 1 billion pageviews since September 1- Over 1.3m daily logins- Over 40k concurrent users during peak
- For the XL Platform ALONE
- Multiple platforms with disparate technology stacks- Microsoft, Oracle, NoSQL- .Net, Java, Opensource
Pearson - few stats...
Copyright © 2015 AppDynamics. All rights reserved. 29