Un-broken Logging - Operability.io 2015 - Matthew Skelton
-
Upload
skelton-thatcher-consulting-ltd -
Category
Software
-
view
3.897 -
download
0
Transcript of Un-broken Logging - Operability.io 2015 - Matthew Skelton
Un-Broken Loggingthe foundation of software operability
Operability.io conference #OIO15Friday 25th September 2015
Matthew SkeltonSkelton Thatcher Consulting
@matthewpskelton
Logging is often unloved
1. Discontinuous
2. Errors only, or arbitrary
3. ‘Bolted on’
4. No aggregation & search
5. Specify severity up front
How to make logging awesome
1. Continuous event IDs
2. Transaction tracing
3. Log aggregation & search tools
4. Design for logging
5. Decoupled severity
reduce time-to-detectincrease team engagement
increase configurabilityenhance DevOps collaboration
#operability
Logging is often unloved
1. Discontinuous
2. Errors only, or arbitrary
3. ‘Bolted on’
4. No aggregation & search
5. Specify severity up front
How to make logging awesome
1. Continuous event IDs
2. Transaction tracing
3. Log aggregation & search tools
4. Design for logging
5. Decoupled severity
public enum EventID
{
// Badly-initialised logging data
NotSet = 0,
// An unrecognised event has occurred
UnexpectedError = 10000,
ApplicationStarted = 20000,
ApplicationShutdownNoticeReceived = 20001,
PageGenerationStarted = 30000,
PageGenerationCompleted = 30001,
MessageQueued = 40000,
MessagePeeked = 40001,
BasketItemAdded = 60001,
BasketItemRemoved = 60002,
CreditCardDetailsSubmitted = 70001,
// ...
}
Technical
Domain
public enum EventID
{
// Badly-initialised logging data
NotSet = 0,
// An unrecognised event has occurred
UnexpectedError = 10000,
ApplicationStarted = 20000,
ApplicationShutdownNoticeReceived = 20001,
PageGenerationStarted = 30000,
PageGenerationCompleted = 30001,
MessageQueued = 40000,
MessagePeeked = 40001,
BasketItemAdded = 60001,
BasketItemRemoved = 60002,
CreditCardDetailsSubmitted = 70001,
// ...
}
APM gives us application insightBUT
How much do we learn? Is APM available on the Dev box?
It’s not just ‘an Ops problem’!
{
"eventmappings": {
"events": {
"event": [ {
"id": "CacheServiceStarted",
"severity": { "level": "Information" }
}, {
"id": "PageCachePurged",
"severity": { "level": "Debug" },
"state": { "enabled": false }
}, {
"id": "DatabaseConnectionTimeOut",
"severity": { "level": "Error" }
} ]
}
}
}
Event tracing
Use enumerations (or closest thing)
Technical and Domain event types
Distributed systems: debuggers less useful
Trace calls with ‘unique-enough’ handles
Tune log levels via config
Logging is often unloved
1. Discontinuous
2. Errors only, or arbitrary
3. ‘Bolted on’
4. No aggregation & search
5. Specify severity up front
How to make logging awesome
1. Continuous event IDs
2. Transaction tracing
3. Log aggregation & search tools
4. Design for logging
5. Decoupled severity
‘Structured Logging’TW: “Adopt” (May 2015)
https://www.thoughtworks.com/radar/techniques/structured-logging
http://gregoryszorc.com/
.NET: http://serilog.net/Java: https://github.com/fluent/fluent-logger-java
More
Ditch the Debugger and Use Log Analysis Instead
Matthew Skelton
https://blog.logentries.com/2015/07/ditch-the-debugger-and-use-log-analysis-instead/
More
Using Log Aggregation Across Dev & Ops: The Pricing Advantage
Rob Thatcher
https://blog.logentries.com/2015/08/using-log-aggregation-across-dev-ops-the-pricing-
advantage/
Thank you
http://skeltonthatcher.com/[email protected]
@SkeltonThatcher
+44 (0)20 8242 4103
@matthewpskelton