Python Diamond

By,Vishwanath Kulkarni

1ms12cs135

March 2016

visit www.improvepresentation.com to grab more slides

1

2

3

4

5

INDEX

Introduction

Python Diamond

Fabrics

Controller and Logging

Conclusion

IntroductionDiamond is a python daemon that collects system metrics and publishes them to Graphite (and others). It is capable of collecting cpu, memory, network, i/o, load and disk metrics. Additionally, it features an API for implementing custom collectors for gathering metrics from almost any source.

History• Brightcove Open

Source Projects• The Google Analytics

plugin for Brightcove Player sends player and video event data to your Google Analytics account. This allows you to see some of your player/video data along with any other analytic metrics you are collecting all in one place.

History• Brightcove, Inc. (

NASDAQ: BCOV) is a Boston, Massachusetts–based software company that produces an online video platform.

• Brightcove Smart Player plugin to allow tracking with Google Analytics in a Brightcove HTML5 player. 

History• Then Came Python

Diamond.• Brightcove was founded

in 2004 by Jeremy Allaire.

Working• Graphite Web• Carbon Cache• Whisper handler• Legends

Coding standards• Genaral• Unit tests are highly

recommended

• PEP-8

• Documentation should be done docstr style at the start of the collector.

• The use of positional arguments for functions is strongly discouraged.

Coding standards

Collectors• Collector default configuration should

be in the get default config method of the class

• All collectors should verify before doing. For example, if the collector requires the existence of a file, check to verify that it exists and is readable before opening.

• use small high-resolution units as default (such as bits or bytes). this gives you most options afterwards. note that graphite will show k/M/G etc prefixes by default if needed.

Installation• Installation/Building

Dependencies• make• rpm-build

Core Dependencies• CentOS or Ubuntu• Python 2.4+• python-configobj• python-setuptools• Python Psutil for non linux system

metrics

Succuss stories• Diamond has successfully been

deployed to a cluster of 1000 machines pushing 3 million points per minute.

• Diamond is deployed on Fabric's infrastructure, polling hundreds of metric sources and pushing millions of points per minute.

Logging• Diamond defaults to logging

at /var/log/diamond/diamond.log. The following is a sample configuration to have log sent to both /var/log/diamond/diamond.log AND local syslog daemon:

Collector Settings• Every collector has some default

options

FabricThe Tools You Need to Build the Best Apps

Fabric was crafted with both power and ease-of-use in mind.

creating different accounts, configuring the SDKs, among others. With Fabric, you’ll never have to worry about tedious configurations or juggling different accounts.

It let you get right into coding and building the next big app.

FabricThe Pulse of All Great Apps

FabricThe Daily users are more than 2.4thousands.

It is rapidly increasing.

9.1 thousands Monthly Active Users.

Sessions 106 thousands.

FabricBest-in-class SDKs.

Modular Kits for Building the Best Apps.

Developers can verifying users, engaging them, generating revenue, among others.

Each presents a unique set of hurdles.

FabricProactive Alerting.

Intelligent Alerts in Real-time.

Nothing is more important than getting timely and actionable data.

Fabrics provide it.

FabricNo need to keep revisiting dashboard to stay on top of apps.

All get actionable crash reports, daily performance metrics and fabrics intelligent analyses.

Every number and metric been displayed is up-to-date and accurate in real-time.

Related Projects• puppet-diamond - diamond module for

puppet.

• chef-diamond - diamond cookbook for chef (Reported Broken).

• chef-diamond - Alternative diamond cookbook for chef (Reported Broken).

• graph-explorer - graphite dashboard that comes with templates for diamond plugins.

• graph-index - index of graphs for diamond.

AdvantagesAll get actionable crash reports, daily performance metrics and fabrics intelligent analyses.

Developers can verifying users, engaging them, generating revenue, among others.

Real-time logging with 98.5% accuracy with python diamond and its products.

Analysing all desktops and servers with just few installations.

DisadvantagesIt’s being deployed in only linux based application.

Windows apps and desktops cannot be analysed.

Has been modified by various organization which has deplicated in its performance.

No documentation.

Thank you

Any Questions?