Using Service-Oriented Methodologies to Create SharePoint Applications by Eugene Rosenfeld -...
description
Transcript of Using Service-Oriented Methodologies to Create SharePoint Applications by Eugene Rosenfeld -...
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Using SOA to Create Using SOA to Create SharePoint ApplicationsSharePoint ApplicationsThis presentation focuses on applying the methodologies and architectural concepts of Services Oriented Architecture (SOA) to the task of creating complex SharePoint applications. We will see how a SOA approach can help us achieve a stable and scalable application or product, decrease support and maintenance costs by decreasing the application's complexity, and reduce the effort required to upgrade the application to the next version of SharePoint.
Eugene RosenfeldCTO, Black Blade Associates, Inc.
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
About Eugene Rosenfeld CTO, Black Blade Associates Two-time SharePoint MVP
www.BlackBladeInc.com twitter.com/erosen03
ThingsThatShouldbeEasy.blogspot.com [email protected]
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
About Black BladeSpecialize in SharePoint document and records management, information architecture, distributed systems, and systems integration
www.BlackBladeInc.com twitter.com/BlackBladeInc www.blackbladeinc.com/pages/blog.aspx [email protected]
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
About this Presentation Intended audience
Application architects (Visio, not Visual StudioSome distributed application design
experience is helpful Not a session on writing the ultimate
web part Focused on HOW the application
components are organized, not WHAT the application does
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
About You How do you use SharePoint today? Why do you want to learn about using
SOA to create SharePoint apps? If you could make just one problem go
away with the snap of your fingers, what would it be?
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Overview Problems with Traditional SharePoint
Application Architecture SOA Intro Rethink the Application as Services Case Study: Document Section Manager
application to docBlock service
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Problems with Traditional Problems with Traditional SharePoint Application SharePoint Application ArchitectureArchitecture
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Sample SharePoint Application
Sha
reP
oint
Far
m
Application Data Storage
SQL Server
Application Interface
Web Front End
Application Logic
Data
Data
Application Page
Web Part
Application Logic
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Aspects of the Architecture Entire codebase must reside on each
web front end server Web front end servers bear the bulk of
the load for executing the code Most code runs in the w3wp.exe
process Most code runs in the context of an
HTTP request
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Problems with the Architecture These are the “bad practices” of
traditional SharePoint application design Entire codebase must reside on each
web front end serverDeployment complexityConfiguration complexityPotential application compatibility issuesOne more thing to worry about when
upgrading to next version of SharePoint
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Problems with the Architecture Web front end servers bear the bulk of
the load for executing the codeDifficult to determine load application
places on farmMay require scaling out the web front ends
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Problems with the Architecture Most code runs in the w3wp.exe
processRecycling the w3wp.exe process causes all
applications to restart
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Problems with the Architecture Most code runs in the context of an
HTTP requestHTTP has a 60 second timeoutBad code can break the HTTP request
pipeline
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Problems with the Architecture Difficult to copy protect application
Many files must be un-protected:○ Site and list definitions and templates○ Content types and other XML files○ Branding customizations (images, CSS, etc)
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Problems with the Architecture Difficult to apply licensing
No extensible license management in SharePoint framework
Cannot easily deploy custom license management components to SharePoint
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Problems with the Architecture Summary
Main issue is that all of the application’s components are being deployed to the customer’s SharePoint farm
The application has no control over the platform on which it is running
The fixSeparate the application from the farmLet the farm access the application as a
service
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
SOA IntroSOA Intro
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
SOA Intro SOA: Services Oriented Architecture SOA is architecture
Not a technologyNot a product
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
SOA Myths Myth: SOA is new Myth: SOA needs an Enterprise Service Bus Myth: SOA requires a particular technology
Myth: SOA requires Web ServicesMyth: SOA requires XMLMyth: SOA requires text-based interfaceMyth: SOA requires HTTP
Proof: Look at WCF in .Net 3.0
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
What’s Left in SOA Definition? SOA is a design principle that abstracts
a business capability from the interface used to access the capability.
SOA services should:Scale upScale outBe distributable
○ Have remotable interfaces○ Be topology-aware
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
SOA Service, not Web ServiceS
OA
Ser
vice
SOA Service Data StorageSOA Service LogicSOA Service Interfaces
Data
Data
Data
HTML
Web Service
RSS
XML
SMTP
Service Logic
DCOM
Terminal
CORBA
Interface Business Capability
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
What about the Clients? The User is not the client HTML Clients: IE, Firefox, Safari RSS Clients: Outlook, NewsGator SMTP Clients: Outlook, Notes, Evolution Terminal Client: HyperTerminal, x3270 Web Service Client: ? – Custom Code XML Client: ? – Custom Code
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Who cares about the Clients? Users
The service client is the users’ only interaction with the service, so make the service client shine.
Sales peopleIt’s very difficult to sell a back-end service.
Much easier to sell the service client. Marketing people
You can’t make brochures or screencasts of a service. You can of a service client.
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Rethink the Application as Rethink the Application as ServicesServices
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Design Goals Increased stability
Customers don’t like applications that periodically stop working
Increased scalabilityIf your product doesn’t scale, your customer
has no reason to buy more than 1 Increase likelihood of getting paid for
your product
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
What Impedes Stability? Platform updates and configuration
changes Platform variations
Windows Server 2003 / R2 / 2008 / R2 / 2012
WSS V3 / V4 / V5 SharePoint Server 2007 / 2010 / 2013x86 / x64
Permissions changes
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
What Impedes Scalability? Poor application architecture
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
What Impedes Getting Paid? Poor adoption Piracy (intentional and unintentional) Unlicensed users, servers, site
collections No upgrade purchased because original
product was not usedDifficult to deployStability problems
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Achieving the Design Goals Re-architect the application as a SOA
Service Get as much code off of the SharePoint
farm and out of W3WP process as possible Secure, copy-protect, and license the
service Spend as much or more effort on the
service client as on the service Distribute the service client freely
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Our Service Client The SharePoint farm is the service client
SO
A S
erv
ice
SOA Service Data StorageSOA Service LogicSOA Service Interfaces
Data
Data
Data
HTML
Web Service
RSS
XML
SMTP
Service Logic
DCOM
Terminal
CORBA
Interface Business Capability
Sha
reP
oint
Far
m
Application Data Storage
SQL Server
Application Interface
Web Front End
Application Logic
Data
Data
Application Page
Web Part
Application Logic
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Achieving the Design Goals Separate processing into 2 categories:
Requires SharePoint infrastructure Does not require SharePoint infrastructure
Separate processing duration into 3 categoriesFast (< 1 sec)Medium ( > 1 sec, < 60 sec) long running ( > 60 sec)
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Design Implementation
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
My code fits more than one box Of course it does. It should. Most non-trivial applications will use at
least 2-3 boxes.
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
For Example… Box 1: A click in a web part causes a
document property in SharePoint to change. Box 2: The property change triggers an
asynchronous event receiver to call a remote web service.
Box 4: The remote web service starts a long-running operation.
Box 6: The long-running operation completes and publishes the results back to SharePoint.
Box 1: The web part displays the new state of the data when a user revisits the page.
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Re-architecting an Application Process flows are the key to re-
architecture Reuse conceptual processes, not code Create good remotable service APIs for
the service client to invoke
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Case Study: Re-Case Study: Re-architecting the Virtual architecting the Virtual Document ServiceDocument Service
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Document Section Manager App What it did
Software that allowed multiple people to edit the same document at the same time
How it did itConverted a physical document into a
virtual document with multiple physical sections
Merged section changes back into main document so main document was always current
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Document Section Manager App Implementation
Document section changes trigger an asynchronous event receiver
Event receiver invokes a custom Windows service to process document sections
Windows service invokes Merger Engine through DCOM to perform the section merges
Another Windows service uploads the processed document back to SharePoint
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
DSM Process FlowD
ocum
ent S
ectio
n M
anag
er P
roce
ssin
g
Assembler Merger Engine UploaderEvent
Receiver
Section Change Detected
Processing Request Received
Merger Engine Initialized
Engine Merges Sections
Assembler service passes merge
results to uploader service
Uploader service uploads merged
document to SharePoint
Updated Virtual Document in SharePoint
Virtual Document is “Processing”
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
DSM’s Place in SharePoint Farm
Redundant Failover HTTP
Load Balancers
Document Section Manager
Load Balanced SharePoint 2007 Web
Servers
Clustered SQL 2005 Database
Back End Servers
SharePoint Users
SharePoint Index and
Application Servers
Merger Engine
Event Receiver
Document Assembler
Document Uploader
Management Pages
Content Types
List Definitions
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Issues with the Architecture All processing ran on each web front
end Deployment was a nightmare
Deploy and configure Merge Engine to each WFE
Deploy and configure custom Windows service to each WFE
System reliability was sub-optimal, mostly due to constant DCOM issues
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Issues with the Architecture Application did not work when
SharePoint was installed on Windows Server 2008, due to DCOM differences
Application did not work when SharePoint was installed on Windows Server 2003 x64
Document processing put substantial load on WFEs
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Issues with the Architecture Main issue:
The Document Section Manager was too intrusive in the SharePoint system. This caused the deployment, configuration, stability, and maintenance issues.
This is an architectural issue that required a fundamental redesign of the application.
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Re-architecting the DSMD
ocum
ent S
ectio
n M
anag
er P
roce
ssin
g
AssemblerService
Merger EngineUploaderService
Event Receiver
Section Change Detected
Processing Request Received
Merger Engine Initialized
Engine Merges Sections
Assembler service passes merge
results to uploader service
Uploader service uploads merged
document to SharePoint
Updated Virtual Document in SharePoint
Virtual Document is “Processing”
1 3 4 & 6
3 4 & 6
3 6
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
docBlock’s Place in SharePoint
Redundant Failover HTTP
Load Balancers
Load Balanced SharePoint 2007
Web Servers
Clustered SQL 2005 Database
Back End Servers
SharePoint Users
SharePoint Index and
Application Servers
Service Client
Event Receiver
Management Pages
Content Types
List Definitions
Service Interface
Merger Engine
Document Assembler
Document Uploader
Virtual Document Processing Service
15% 85%
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Re-architecting the DSM Document Assembler Service remains
mostly unchanged but is moved from SharePoint farm to docBlock server appliance.
Uploader Service remains mostly unchanged but is moved from SharePoint farm to docBlock server appliance.
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Re-architecting the DSM New SOAP Service interface created for
Document Assembler and Uploader services on docBlock server appliance.
Event receiver remains mostly unchanged but now invokes the Document Assembler Service remotely rather than locally.
Merger Engine is removed from SharePoint WFEs.
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
docBlock Client / Service Flow
docB
lock
Ser
vice
Pro
cess
ing
AssemblerService
Merger EngineUploaderService
docBlock Web Service Interface
Processing Request Received
Processing Request Received
Merger Engine Initialized
Engine Merges Sections
Assembler service passes merge
results to uploader service
Uploader service uploads merged
document to SharePoint
Web Service Returns
Processing Complete
docB
lock
Clie
nt P
roce
ssin
g
Event Receiver
Section Change Detected
Virtual Document is “Processing”
Data is prepared for docBlock
service invocation
docBlock service invoked
ServiceClient
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Tech Benefits of Re-architecture Deployment is 80% easier 85% less code running on SharePoint
WFEs 95% less load on SharePoint WFEs 75% increase in processing reliability
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Tech Benefits of Re-architecture Support for SharePoint on all Windows
Server versions, including x86 and x64 Support for all versions of SharePoint,
including 2013 Cloud-ready architecture – important for
Office 365
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Non-Tech Benefits Easier to find developers Easier to sell Companies willing to pay us to help
them similarly re-architect their applications.
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Non-Tech Benefits Application can now be distributed as a
virtual or physical appliance. Application can now be hosted “in the
cloud.”
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
ClosingClosing
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
Summary If processing doesn’t require SharePoint,
don’t run the processing in SharePoint. Minimal intrusion into SharePoint yields
a more stable, reliable, performant platform.
Utilizing SOA service interfaces yields more manageable, upgradable, and supportable code.
The SOA service is “the product”.
www.BlackBladeInc.com | [email protected] | +1-703-260-1111
About Black BladeSpecialize in SharePoint document and records management, information architecture, distributed systems, and systems integration
www.BlackBladeInc.com twitter.com/BlackBladeInc www.blackbladeinc.com/pages/blog.aspx [email protected]