Graylog2 (MongoBerlin/MongoHamburg 2010)
-
Upload
lennartkoopmann -
Category
News & Politics
-
view
2.839 -
download
0
Transcript of Graylog2 (MongoBerlin/MongoHamburg 2010)
Graylog2 Syslog with Rails and MongoDB candy
Manage your logs in the dark and have lasersgoing and make it look like you're from space
Lennart Koopmann, 2010
About me
Lennart Koopmann22 years oldLiving in Hamburg, working at Jimdo
lennartkoopmann.net / @_lennart
phpLogCon let's not talk about thatGraylog1: August 2009Graylog2: August 2010
Syslog daemon
(TCP/UDP)
GELF
(UDP)
Graylog extended log format
UDP - Max (GZIP) 8192 byte per message(Chunking is supported)
{ }
{ 'message':'Exception: Something went wrong.' }
{ 'message':'Exception: Something went wrong.', 'full_message':'Stacktrace.\nSome env vars' }
{ 'message':'Exception: Something went wrong.', 'full_message':'Stacktrace.\nSome env vars','host':'www19' }
{ 'message':'Exception: Something went wrong.', 'full_message':'Stacktrace.\nSome env vars','host':'www19', 'file':'/var/www/index.php' }
{ 'message':'Exception: Something went wrong.', 'full_message':'Stacktrace.\nSome env vars','host':'www19', 'file':'/var/www/index.php','line':2638 }
{ 'message':'Exception: Something went wrong.', 'full_message':'Stacktrace.\nSome env vars','host':'www19', 'file':'/var/www/index.php','line':2638, 'level':1 }
Chunking
Use Cases
Plain syslog
Collect everything of your /var/log from all servers, aggregate, analyze and get a warning if something goes wrong
GELF
Embed it into the logging class of your application and trigger with every error. Include stacktrace and env vars in full message. Later filter by file:line to get statistics.
Aggregate, analyze and get a warning if something goes wrong
Notifications
Currently only Nagios hook (rake task) Triggered at too many messages in last X minutes
Fine grained notifications coming with incident management. (Email, SMS APIs, XMPP)
Average page generation time
Graylog1 / MySQL: ~12000ms
Average page generation time
Graylog1 / MySQL: ~12000msGraylog2 / MongoDB: ~250ms
Roadmap
Incident management
Define chained rules that explain an incident. Trigger several actions when incident occurs.
Fine grained notifications
Be informed via Email, XMPP, SMS by Graylog2 directly without Nagios hook
More analyzing
Thank you.
www.graylog2.org / @Graylog2www.lennartkoopmann.net / @_lennart
(Slides are on Slideshare)