Minimal effort server give you more noteworthy security and genuine feelings of serenity.
Developing Highly Instrumented Applications with Minimal Effort
-
Upload
tim-hobson -
Category
Technology
-
view
2.249 -
download
0
description
Transcript of Developing Highly Instrumented Applications with Minimal Effort
![Page 1: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/1.jpg)
Tim HobsonPrincipal Engineer, Chief Caffeine Officer - Intuit
Developing highly instrumented applications with minimal effort
![Page 2: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/2.jpg)
2
About Me
Work
Play
![Page 3: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/3.jpg)
3
Agenda
Theory• Patterns• Supporting
Frameworks• Supporting
Components
Practice• .Net Example • Java Example• Node.js Example
Concepts• Quality Data• Application
Instrumentation
• Application Logging
GOAL: You should be able to apply this to your application today
![Page 4: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/4.jpg)
Concepts
![Page 5: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/5.jpg)
5
If You Do Nothing…
OR
![Page 6: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/6.jpg)
6
There’s No Free Lunch
Garbage In, Garbage OutWe can
parse
anything!
Insight!
![Page 7: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/7.jpg)
7
Quality In, Insight Out
Most apps start with only the framework or app server logging (or nothing!)
None of the above is interesting to the business or the developer.
You can’t get if you don’t give – there are many ways to give, and many classes of data to provide.
![Page 8: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/8.jpg)
8
Classes of System Output
App Instrumentation
• Cross-cutting (free)• App Activity• Passively triggers
alerts• Source of
performance data• Source of usage data
App Logging• Intentional• Business
Transactions• Overtly triggers
alerts• Source of business
metrics• Aids in
troubleshooting failures, bugs
System Instrumentation• JMX/WMI/SNMP monitoring• Apache/IIS/nginx access logs
![Page 9: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/9.jpg)
Theory and Best Practices
![Page 10: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/10.jpg)
10
Best Practices (© splunk>)
Create human readable events
Clearly timestamp events
Use key-value pairs
Be aware of multi-value fields
Log unique identifiers
![Page 11: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/11.jpg)
11
Best Practices (© Tim)
Global timestamps (UTC – 2013-08-21 22:43:31,990)
Context setting (who/what/where/when/how)
Categories/taxonomy (what tier, what component)
Timing (time everything!)
Security (never log sensitive data: password=***)
Consistency in naming – (action=purchase; sale=oct13;
productId=123123)– (action=buy; promo=oct13; sku=123123)
![Page 12: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/12.jpg)
12
Example Output
2013-08-21 22:55:36,504; LogLevel=INFO; sid=q3prv41kt511vzojytnx1d42; rid=6500583; userLogin=(null); ipAddress=0.0.0.0; thread=249; category=Web.Controllers.BaseWebController; msg=RequestInfo; server=ws001prod; url=https://myapp.com/account/logon; method=GET; languages=en-US,en;q=0.8; referrer=https://myapp.com/members/dashboard; userAgent=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36; userId=123456; controller=Account; action=Logon;
2013-08-21 22:55:36,519; LogLevel=INFO; sid=q3prv41kt511vzojytnx1d42; rid=6500583; userLogin=(null); ipAddress=0.0.0.0; thread=249; category=TraceInterceptor; timeTracing=Service.Users.GetUser, time=3;
Context
Metrics
![Page 13: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/13.jpg)
13
Patterns
Pipeline– Filter– Interceptor
Dependency injection– Proxy
Pointcuts/aspects
![Page 14: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/14.jpg)
14
HTTP Pipelines
Java (from Oracle Docs)
ASP.Net (from MSFT Docs)
express.js
var app = express();app.use(func1);app.use(func2);app.use(func3);app.use(func4);app.use(func5);app.use(func6);app.use(func7);app.use(func8);app.listen(8080);
![Page 15: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/15.jpg)
15
Dependency Injection
• Enforces interface-based programming
• Forbids circular references• Lends itself to testability• Flexibility in changing
implementations• For our purposes: cross-cutting
capabilities
![Page 16: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/16.jpg)
Intercepting Calls With Dynamic Proxies
16
Implementation
Caller
DI Container
Dependency A
Dependency B
Proxy
Inte
rcep
tor
Inte
rcep
tor
Inte
rcep
tor
Proxy
Inte
rcep
tor
Inte
rcep
tor
Inte
rcep
tor
Proxy
Inte
rcep
tor
Inte
rcep
tor
Inte
rcep
tor
![Page 17: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/17.jpg)
Supporting Frameworks
17
The Inner Sanctum• Unity Dynamic
Proxies• Spring
@AspectJ Pointcuts
• JavaScript Mixins
Annotations & Attributes• @Instrumenta
ble• [Instrumentabl
e]
The Front Door• ASP.Net
Modules/ActionFilters
• Java Servlet Filters/Spring Interceptors
• Node.js interceptors
![Page 18: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/18.jpg)
Supporting Components
18
Java• LogBack• SLF4J• Log4J
node.js• SenchaLab
s Connect• Winston
.Net• Log4Net• NLog
![Page 19: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/19.jpg)
Intervention!
![Page 20: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/20.jpg)
20
Sample App Intervention
3 Sample Apps
3 Interventions3 Platforms
+ + +
![Page 21: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/21.jpg)
21
Goals
Minimize impact on existing code
Apply best practices
Leverage aspects for interception
Leverage an HTTP pipeline for context
![Page 22: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/22.jpg)
22
MVC Music Store Intervention (Demo)
Missing dependency injection/interception framework: UnityMissing logging library: Log4Net, Buche
1. Add DI framework (Unity)2. Configure aspects (Interception)3. Configure controller factory, container locator 4. Set up logger (Log4Net)5. Configure log pattern and targets (Log4Net.xml)6. Wire up logging interceptor7. Wire up LogActionFilter and BaseLoggingController8. Deploy!
![Page 23: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/23.jpg)
23
Spring Pet Clinic Intervention (Demo)
1. Set up loggera. Configure log pattern (to support context data)b. Configure targets (rotating file)
2. Create LogAspect.java and @Instrumentable attributea. Capture calling contextb. Start/stop timing
3. Create LogInterceptor.javaa. Capture request metadatab. Set MDCc. Start/stop timing
4. Configure aspect bean
5. Configure interceptor bean
6. Annotate the methods we care about
![Page 24: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/24.jpg)
24
NodeCellar Intervention (Demo)
Missing Interception Framework: ScarletMissing Logging Library: Winston
1. Set up Scarlet 2. Configure log pattern3. Configure method interception4. Create logging interceptor5. Create LoggingFilter6. Deploy!
![Page 25: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/25.jpg)
Closing Thoughts
![Page 26: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/26.jpg)
Got Log? Now Get Intimate With Your App
26
Ops Dashboards Pro-Active Service
Degradation Alerting
Quality Assurance
PerformanceMetricsBusiness
Dashboards
SLA Tracking
Pre-ReleaseSanity TestingSecurit
y Alertin
g
Customer
Support
![Page 27: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/27.jpg)
27
Key Takeaways
• It is YOUR responsibility as a developer to provide useful operational and business data.
• It is not hard, and most of it is for free once you have the patterns in place.
• The same patterns and practices can be applied to practically any platform, and any type of application or service.
• When you provide consistent and predictable data others can build on your greatness
![Page 28: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/28.jpg)
28
Get the Code@hoserdude
Spring Pet Clinic Intervention:https://github.com/hoserdude/spring-petclinic-instrumented
MVC Music Store Intervention:https://github.com/hoserdude/mvcmusicstore-instrumented
NodeCellar Intervention:https://github.com/hoserdude/nodecellar-instrumented
![Page 29: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/29.jpg)
Intuit Speakers @ Silicon Code Camp 2013:SATURDAY9:45 a.m. - Ramakrishna Kollipara – “Complete Automation of Performance Testing” 1:45 p.m. - Joe Wells - “QBO: Journey From legacy Java app to a Client-side HTML5 app”3:30 p.m. - Naga Addagadde & Sangeeta Narang – “Intuit APIs for Financial Transaction Aggregation”5:00 p.m. Ted Drake –“Hitting the Accessibility High Notes with ARIA”
SUNDAY9:15 a.m. - Eugene Krivopaltsev –“Building Native Mobile Apps with Custom Views” 1:15 p.m. - Tim Hobson – “Developing Highly Instrumental Applications with Minimal Effort”
You don't want to miss out on a chance to win this cool headset. Stop by our booth to enter!
For more information about joining our organization visit our booth or connect with our onsite recruiter:
![Page 30: Developing Highly Instrumented Applications with Minimal Effort](https://reader035.fdocuments.net/reader035/viewer/2022062708/558cdb21d8b42acb118b4648/html5/thumbnails/30.jpg)
30
THANK YOU
Want to talk more?
I’ll be at the Intuit booth today 3-5 PM.