Aug 2012 HUG: Hug BigTop
37
What's stacking? Or: How I Learned to Stop Worrying and Love the BigTop Dr. Konstantin Boudnik Apache Hadoop committer since 2009 Co-author of iTest (BigTop's kernel) Senior architect @Karmasphere [email protected]
-
Upload
yahoo-developer-network -
Category
Technology
-
view
1.145 -
download
0
Transcript of Aug 2012 HUG: Hug BigTop
What's stacking?
Or: How I Learned to Stop Worrying
and Love the BigTop
Dr. Konstantin BoudnikApache Hadoop committer since 2009
Co-author of iTest (BigTop's kernel)Senior architect @Karmasphere
Distributed under CC3.0https://creativecommons.org/licenses/by/3.0/
Disclaimer!
All characters appearing in this work are fictitious. Any resemblance to real creatures, living or dead, is purely
coincidental
Can you see the forrest behind the trees?
Not a technical talk...
What's the real intrinsic value of BigTopWhat is our philosophical proposition
Why Hadoop data stack is important?
The elephant is yesterday's news...
An article over at Forbes:
“...it is not a database, it is not an analytics environment, and it is not a visualization tool. By itself, it is not a solution for helping businesses make better decisions.”
Some self-proclaimed hot-shot dude
Sorry, I am allergic to bullshit...
There are limits, all right...
● Maximum size of the file system● High availability● Recoverability
But there is a lot to pick from...
● Do you want to experiment with– CEPH
– Cassandra
– Ganglia
– Puppet
Why I am wasting your time?
● You are an ASP and need to ship a stack
Why I am wasting your time?
● You are an ASP and need to ship a stack● You need a higher level abstraction for MR
Why I am wasting your time?
● You are an ASP and need to ship a stack● You need a higher level abstraction for MR● You are in need of virtual deployment
Why I am wasting your time?
● You are an ASP and need to ship a stack● You need a higher level abstraction for MR● You are in need of virtual deployment
Hey!
“I am gonna just pick up a distro from B or S”1)
1) “B” - Big Ego Distributor “S” - Small Ego Distributor
Yeah, right! (case study)
Commercial vendor “BS” packs Hive 0.7.1 into a release
We know how it is full of it
Good news!
“Our next release comes with Hive 0.8.1”BS' VP of Product
But may be they just forgot this?..
public int executeUpdate(String sql, String[] columnNames) throws SQLException {
throw new SQLException("Method not supported");
}
So, no worries! We can deliver!
But can they?
Do you REALLY have to be at mercy of a distribution vendors?
“I have no resource to do it myself”
DIY: and there's always “but”...
Maybe ASP or ISV business isn't for you?
Or maybeBigTop stack: take it and go...
Modify a stack BOM– Build
– Deploy
– Configure with Puppet (included)
– Test (scenarios are provided or easy to add)
– Grab a virtual appliance if short of hardware
Hit a problem?Just rinse and repeat (C)
Many thanks to the good folks from Hadoop community!
● Many components● Gazillions of versions● Lot of patches if you like it hot and dirty
Where is the philosophy?
WAIT!
Competition is great!● Competition enabling open platform● Diversity of stacks● No more vendors lock-ups● Don't be afraid to try● Be your own Sprechstallmeister
May be something wrong with this?
News And Updates
5 releases of BigTop!
Three are out Two more in the work
Strong community and growing!
20 committers and many contributorsWe are looking for more!
And sorry: no more binaries for you
BigTop won't be publishing convenience binary artifacts anymore
Might as well use this expertise:http://www.magnatempusgroup.net
Come and join us!
Powered By Bigtop:ClouderaEMCHortonWorksMagnaTempusGroupTrend Micro
https://incubator.apache.org/bigtop/
https://blogs.apache.org/bigtop/
May be one day...
Q & A
