Security in OSGi applications: Robust OSGi Platforms, secure Bundles
The Cathedral, the Bazaar and the Commissar -...
Transcript of The Cathedral, the Bazaar and the Commissar -...
The Cathedral, the Bazaar and the Commissar
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited.
The Evolution of Innovation in Enterprise Java
Topics
• The Sources of Innovation
• A History of Innovation in Enterprise Java
• Where next?
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 2
• Where next?
Assumption:
Innovation matters and is essential to
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited.
Innovation matters and is essential to the flourishing of a technology platform
What Drives Innovation?
• There has been much research into the drivers of innovation
• Key themes identified include
– Creativity
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 4
– Creativity
– Experimentation
– Competition
– Economic motivation
Sources of Creativity
• Source: National Center of Education and the Economy (2006)
• Creative Thinking
– Comfort in disagreeing with others and trying
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 5
– Comfort in disagreeing with others and trying solutions that depart from the status quo
– Combining knowledge from previously disparate fields
– Ability to persevere through difficult problems and dry spells
– Ability to step away from an effort and return later with a fresh perspective (“incubation”)
Value of Experimentation in Innovation
• Stefan Thomke of Harvard Business School argues that every company’s ability to innovate depends on a series of experiments [successful or not], that help create new products and services or improve old ones. That period between the earliest point in the design cycle and the final release should be filled with experimentation,
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 6
final release should be filled with experimentation, failure, analysis, and yet another round of experimentation.
• Experimentation can only work if failure is punished and failed ideas are killed off
Innovation and competition
• The Department of Justice and the Federal Trade Commission have frequently raised innovation concerns as reasons to challenge mergers
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 7
• Harvard Business School – “Competition and Innovation” paper
– Empirical work…has pointed to a positive correlation between product market competition and innovative output
The ages of Enterprise Java
• Before J2EE
• The promise of J2EE
• The decline of J2EE
• The rise of open source
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 8
• The rise of open source
Before J2EE
• Mid 1990s
– Java gradually moves to the server side
• Largely unregulated
• Many competing products in different areas
– NetDynamics
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 9
– NetDynamics
– TopLink
– Silverstream
– Persistence PowerTier
– Apple WebObjects
Before J2EE…
• Good and Bad– Innovation and choice of approaches
–Fragmented server-side market
–Real danger of vendor lock-in
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 10
–Real danger of vendor lock-in
–Many solutions very expensive,• No impact from open source
The Glory Days of J2EE
• 1999-2003
– The JCP becomes dominant in the space
• TopLink and other “non-standard” technologies cannot compete with J2EE
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 11
technologies cannot compete with J2EE standards
– ORM versus EJB entity beans
– Velocity vs JSF
– WebObjects vs web tier
The Glory Days of J2EE
• Good and bad
– A market is created
– Vendor lock-in is reduced, but not eliminated
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 12
– Increasing thought control strangles innovation
– Flaws in the model take years to be resolved, while many projects fail
The decline of J2EE
• 2003-
• Move away from traditional application server towards lighter-weight
• Proportion of enterprise Java users using Tomcat
40
50
60
70
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 13
lighter-weight solutions such as Tomcat
– Tomcat now clear leader in enterprise Java deployments
0
10
20
30
40
WAS JBoss WLS Tomcat
Springframework.orgBZ Research
The rise of open source
• Fewer and fewer organizations develop enterprise Java applications without using open source
– Those that do face increasing competitive
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 14
– Those that do face increasing competitive disadvantage
• Numerous open source projects help to shape the future
– Eclipse
– Spring
– AspectJ
The three key sources of innovation in Enterprise Java
• The Cathedral
• The Bazaar
• The Commissar
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 15
• Let’s understand each
• Understand why none alone is sufficient
The Cathedral or Bazaar?
• Cathedral model has software built by relatively few people, with centralized design
• Bazaar has many deveolopers who lay out their wares
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 16
• Classic statement on open source by Eric Raymond, 1996– Linux is subversive. Who would have thought
even five years ago (1991) that a world-class operating system could coalesce as if by magic out of part-time hacking by several thousand developers scattered all over the planet, connected only by the tenuous strands of the Internet?
Reality check
• As Linux has matured into enterprise use, it’s no longer predominantly developed this way
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 17
Case Study: Linux
• Dispelling the perception that Linux is cobbled together by a large cadre of lone hackers working in isolation, the individual in charge of managing the Linux kernel said that most Linux improvements now come from corporations.
“People’s stereotype [of the typical Linux developer] is of a male computer geek working in his basement writing code in his spare time, purely for the love of his craft. Such people were a significant force up until about five years ago,” said Andrew Morton, whose role is maintaining the Linux kernel in
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 18
Andrew Morton, whose role is maintaining the Linux kernel in its stable form.
Morton said contributions from such enthusiasts, “is waning.” Instead, most code is generated by programmers punching the corporate time clock.
About 1,000 developers contribute changes to Linux on a regular basis, Morton said. Of those 1,000 developers, about 100 are paid to work on Linux by their employers. And those 100 have contributed about 37,000 of the last 38,000 changes made to the operating system.
Not everyone wants to play
• As open source becomes mainstream, more and more users of open source won’t want to contribute and shouldn’t need to
• “The myth of open source software is the aura of freedom that surrounds it. Download the source code and play with it if you wish. And best of all, you won’t have to pay the freight.
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 19
Although that’s the public image of open source, the reason why open source software is growing popular within enterprises has nothing to do with open source itself. Few enterprises care about whether they can monkey around with source code because the relative minority that still have active internal software development staffs have more important things to do.”
Tony Baer, Datamonitor Computerwire
Myth: Community magically generates open source software
• All we need is lots of developers
• With all those fingers on all those keyboards, great software just happens
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 20
happens
Reality check
• Innovation is not a numbers game
• The bazaar model encourages competition in implementation but may not produce innovation
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 21
not produce innovation
• The cathedral model is more likely to produce innovation
• Neither is a complete solution
The Commissar
• Java has its own somewhat unique model
– The Commissar
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 22
• In this model, the politburo knows what’s best for the proletariat (you)
The Commissar Knows Best
• JCP expert groups talk largely in private
• Typically composed of software vendors
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 23
software vendors
• Relatively slow pace of change, like Soviet 5 year plans
Why standards are needed
• Standards can create markets
• Standards can provide a base on which competing open source and commercial alternatives can flourish
– JTA
– Servlet API
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 24
– Servlet API
– JMS
• Standards can protect customers from lock in to a proprietary technology
• To ensure interoperability
– Web Services
– IIOP
How much standardization is too much?
• In the Java world we have an unhealthy obsession with standards
• Desire to standardize everything
• Failure to critically evaluate standard
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 25
• Failure to critically evaluate standard technologies
Where Standards Don’t Work
• Jim Waldo (Sun Distinguished Engineer)
– Kowtowing to the god of standards is, I believe, doing great damage to our industry, or craft, and our science. It turns technical discussions into political debates. It misunderstands the role that standards have played in the past. Worst of all, it is leading us
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 26
have played in the past. Worst of all, it is leading us down absurd technological paths in the quest to follow standards which have never been implemented and aren’t the right thing for the problems at hand.
Where standards don’t work
• CORBA history (1990s)
– Death by committee
– Attempts to innovate by committee (distributed persistent objects)
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 27
• When they’re too slow
• When they’re divorced from reality
– Ivory castle
• When they are about politics, not technology
Politics
• Like the Soviet Union, the JCP has also seen its great purges
• JDO
– Arguably still the best technology choice for ORM
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 28
ORM
– JPA is promising, but JDO 2.x remains superior
– Yet JDO is dead in the market
• Database vendors didn’t like it and they are influential in the JCP
• No application server vendor liked it as it competed with EJB technology
• …so it was taken out and shot
Today’s Political Struggle: OSGi vs JSR-277
• OSGi: Proven technology for dynamic modularization
• From the OSGi Alliance
• Yet JSR-277 attempts to reinvent modularization for Java
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 29
modularization for Java
• Make or break issue for the JCP
• IBM, Oracle, BEA and many other vendors are building their middleware strategy on OSGi
– They are not going to walk away just because Sun doesn’t like it
Enterprise Java is no longer a one party state
• Not just the Party (the JCP)
• OASIS
– SCA
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 30
– SCA
– Web Services standards
• OSGi Alliance
– Dynamic modularization standards
– More enterprise standards
• Open source projects
The JCP must look at wider world and accept that it doesn’t need to reinvent everything
• History of failure when this has not happened
JCP technology Ignored existing technology
Negative consequences
Entity beans TopLink and all other ORM solutions
•Two complete failures (EJB 1.x and 2.x) •ORM in Java loses at least 6 years•Billions of dollars of wasted development effort from customers
Commons Log4J Added complexity of pointless abstraction
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 31
Commons Logging
Log4J Added complexity of pointless abstraction layers such as Commons Logging
EJB (DI) Spring, PicoContainer, Hivemind
Limited DI functionality in EJB 3 specification misses opportunity to match best practice
EJB3 (interception)
Spring, AOP Alliance, AspectJ, AspectWerkz
Lack of knowledge of AOP in the expert group produces fragile, clunky API missing central AOP concepts
JSR 277 (modularization)
OSGi •Ignoring input and experience from OSGi•May split JCP as many organizations are deeply committed to OSGi
The standards check list
1. Will the pace of change and innovation required by met in a standards process cycle
2. Do we benefit from competing implementations?
3. Does this affect wire protocols (in which standards are probably outside Java)
4. Is there an entrenched open source solution, in
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 32
4. Is there an entrenched open source solution, in which case competition may not occur?
5. Is the field mature and well understood
1. Has the proposal been tested in the market?
2. Do not want design by committee?
6. Is the standards committee representative of the users of the technology?
Does the standard exist to lock out newcomers?
• When standards become too complex, like J2EE, they effectively lock out new entrants and benefit existing franchises, not consumers
• Study of the NHS
– In open markets the threat of entry by newcomers
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 33
– In open markets the threat of entry by newcomers not only puts pressure on prices; it also acts as a pressure towards innovation because if, say, a hospital – or new provider – pioneered new techniques that provide higher quality, more cost-effective, care, it should both attract more patients and make greater profit
Competition is diminishing in Java EE
• Oracle purchase of BEA leaves IBM, Oracle dominant in Java EE
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 34
• Since Red Hat acquisition, JBoss momentum has stalled
Just the right amount of competition
• There must be a reward for innovation
• Bad solutions must die
• Standard economic theory predicts that
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 35
• Standard economic theory predicts that innovation should decline with competition, as more competition reduces the monopoly rents that reward successful innovators
The JCP and creativity
Creativity enabler JCP Open source
Cathedral
Comfort in disagreeing with others and trying solutions that depart from the status quo
Combing knowledge from previously disparate fields
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 36
disparate fields
Ability to persevere through difficult problems and dry spells
Ability to step away from an effort and return later with a fresh perspective
Competition and elimination of bad ideas through market pressure
Create market
JCP: Conclusion
• The JCP is unlikely to produce innovation but should focus on what it can succeed at
– Creating a market where innovators can
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 37
– Creating a market where innovators can compete above fundamental stanards
• Innovation by committee is a bad idea, and has traditionally produced poor results
Innovation needs Competition: External Forces
• Ruby on Rails
• .NET
• External competition is very important
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 38
• External competition is very important to Java
• Good to Great
– One of the companies profiled succeeded because they chose to compete with a highly efficient larger competitor, with the aim of lifting their own performance
The Commissar doesn’t like Competition
• Communism views competition as wasteful and the root of all evil
– The proletarian liberates himself by abolishing competition, private property, and all class differences
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 39
and all class differences
• Friedrich Engels - The Principles of Communism (1847)
The Reality
• Competition leads to the best performance, even among communists
• Ironically, the Soviet Union only ever succeeded when forced to compete to survive
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 40
But lack of inherent competition in the system brought it down
• What really destroyed the USSR was the rapid advancement of technology developed within the fires of cooperative competition among Western nations. The Soviets had brilliant scientists and researchers, but they couldn’t
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 41
scientists and researchers, but they couldn’t outmaneuver the efforts of thousands of individual companies, whose efforts were being selected by merit in a free market by millions of consumers.
Things need to become faster
• Competition and experimentation needs to occur rapidly
• As with the Soviet Union, technology change and the increasing pace of
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 42
change and the increasing pace of business leaves 2-3 years committee-driven cycles looking less and less relevant
Open Source implications: MySQL
• Open source produces fast experimentation/review cycles
• Biggest event in the future of the JCP is
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 43
• Biggest event in the future of the JCP is not connected to Java
• Sun is becoming an open source company
• The JCP is not a true standards body, so is inevitably shaped by Sun’s agenda
Evidence of positive change
• JCP sessions, chaired by Patrick Curran (JCP chair) show that Sun is opening up
• The JCP has gradually evolved over time to be more open and less bureaucratic
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 44
to be more open and less bureaucratic
• Needs to change much more and much faster, but there is real hope…
What does Tomorrow Look Like?
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited.
The standardization cycle
Breakdown of standards process
Breakdown of standards process
No standardsNo standards
Lack of innovationLack of innovation
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 46
StandardizationStandardization
Create coherent market
Create coherent market
Lock out new entrants /
Produce lowest common denominator
Lock out new entrants /
Produce lowest common denominator
The future
• No longer a one party state
– No one vendor or organization (even Sun) will control all the pieces
• OSGi is shaping up as a make-or-break test
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 47
• Three sources of innovation
– Cathedral (proprietary vendors)
– Bazaar (open source)
– The Commissar (JCP)
• All three are needed
The future
• Open source innovation will continue
– Apache
– Eclipse
– Spring
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 48
– Spring
• Change needs to be more rapid
– JCP needs to adapt to survive
– Likely to be run according to true open source
The future
• Remember that you can be more than a spectator
1. Evaluate technologies on merit, not necessarily because of where they come from
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 49
from
2. Choose the technologies you want to use
3. Take the opportunity to participate in the JCP if you want to help keep it relevant
1. It’s not just a question of Sun “fixing” the JCP