Succeeding with Hybrid SharePoint and Search:Strategy and Implementation

Jeff Fried CTO, BA Insight

Mike GregoryDirector Systems Engineering, BA Insight

SPS ConnecticutOctober 2015

Succeeding with HybridFail to plan and you are planning to fail

Planning starts with understanding

KCTCS (background)

Focused on Search and SharePoint since 2004

Longtime Search Nerd• CTO, BA Insight• Senior PM,

Microsoft• VP, FAST• SVP, LingoMotors

Who is this guy? Jeff FriedPassionate About• Search• SharePoint• Search-driven

applications• Information

Strategy

Blog: DoMoreWithSearch.comTechnet Column“A View from the Crawlspace”

jeff.fried@bainsight.com

Longtime Nerd• BA Insight• 25+ years in

enterprise information storage and retrieval: databases, middleware, social platforms

• Sybase, Iona, etc.

Who is that other guy? Mike GregoryPassionate About• Applying

technology to solve real business problems

• Making complex concepts understandable

Broad industry / application experience: • Government / Defense• Call Center• Content Management• Knowledge Management

About BA Insight We provide connectivity, classification, and application software that accelerates

and future-proofs the implementation of on-premise and cloud-based SharePoint portals.

– Secure connectivity to a wide variety of content systems– Increased findability using auto-tagging, metadata generation, and text analytics– Applications and UI components to help with the creation of knowledge centers

Hundreds of successful implementations at Global 5000 companies:

Hybrid SharePoint - The What, the Why, & the How

Hybrid SharePoint and Search- Cloud Hybrid Search and the Cloud SSA- Scenarios, Configurations, & Gotchas

Succeeding With Hybrid SharePoint and Search - Projects and Tools

New

Succeeding with HybridStart with your goals and strategy

Move to the cloud without– Breaking customizations– Raising security concerns

Use features not available with SharePoint Online– extensibility models, search on external content, cross-site publishing….

Reduce risk by migrating in steps– Separation of workloads

Keep up with Microsoft’s cloud-first/cloud-only roadmap– and/or hedge your bets

Why Hybrid SharePoint?

9

Delve

PowerBIYammer

Next-Gen PortalsOffice 365 APIs

Cloud-only

OneDrive

Office VideoEquivio Zoom(e-Discovery)

Migrate remote users

Two Hybrid Adoption Strategies

Pilot Online Service with a subset of users Off-board exchange

mailboxes

MigrateMove everything to the cloud

at your own pace

Co-ExistMaintain a hybrid model

Keep using On-Premises systems & customizations; mix according to need

The Evolutionof SharePoint:

HYBRID

Management ExtensibilityExperiences

| Server

Experiences Management Extensibility

| Server | Server

HYBRID

Team Sites

Portals

SearchEnterprise

Content Mngt

BI

Succeeding with HybridYour workloads determine approach

Key building blocks: identity and search

SharePoint “Hybrid Classic”

Search: Bidirectional Business Connectivity Services: SupportedDuet Enterprise for SharePoint and SAP: Supported

IntranetMicrosoft data center InternetMicrosoft Office 365 tenant

SharePoint Online

Federated search results

Site collection

SharePoint Online can query SharePoint Server

SharePoint Server 2013

SharePoint

Primary web app

Federated search results

SharePoint Server can query SharePoint Online

Outbound

Inbound

Customer networkPerimeter network

15

Essential building block:Authentication with Hybrid SharePoint

http://blogs.office.com/2014/05/13/choosing-a-sign-in-model-for-office-365/

Approaches to Hybrid – by Workload

Split Workload

different tools indifferent places

Split User

task uses content or sites across ‘the divide’

Exchange, SharePoint, LyncOneDrive, Yammer, PowerBI, Delve

Extranet, Mysites, Team Sites, Project SitesPortals, Intranet, Services/Applications

Links Search

Search Provides a Unified ViewSeamless experience for users• Don’t need to know where content is• Effective for “split user” hybrid

Bring in content from many systems• Not “just” SharePoint• Tap Business Critical content where it lives• Secure; reduces load & risk on LoB systems

Unified Index drives the experience

SharePoint 2016 HybridFacilities built in for “split user”

Cloud Hybrid Search

User Profiles Following

ExtranetCompliance

(DLP/e-Discovery)

Config Experience

Built on Search

Succeeding with HybridImportant new mechanism:

Cloud Hybrid Search

Cloud Hybrid Search

Cloud Search Service Application • Unified index with on-premises and cloud

content• Feeds Office Graph/Delve experiences • Supports Search as a Service • Reduces search crawl footprint

Audio textAnd search

indexpropertiessignalsMetadata extraction and processing

Cloud SSA O365Search Index

External Content(on-premises

and/orin the cloud)

SharePoint Server(On-premises or Hosted)

Office 365

SharePoint Online Content

Onedrive for Business Content

Conn

ecto

rs

SharePoint Content

Hybrid Connectivity (index in O365)

Content and Collaboration Systems• EMC Documentum• EMC eRoom• HP Trim• IBM Filenet F8• IBM Content Manager• IBM Connections• Objective DMS• OpenText LiveLink/RM• OpenText Hummingbird / eDocs• Oracle CMS/Stellent• IBM Lotus Notes • Xerox DocuShare• Confluence• Alfresco• Jive• CuadraSTAR

BA Insight ConnectorsMailbox and Archiving Systems• Microsoft Exchange • Microsoft Exchange Online• IBM Lotus Notes • Symantex Evault• Autonomy EAS / (Zantaz)

Practice Management Systems• Aderant Practice Management• Autonomy Worksite (iManage)• Elite/3E• KnowledgeMill OnePlace• LegalKey • NetDocuments• Practical Law • AutoElite Prolaw • RealPractice

CRM Systems• Salesforce.com/Force.com• Microsoft Dynamics CRM• LexisNexis Interaction CRM• Any SQL Based CRM

Databases• Microsoft SQL Server• MySQL• IBM DB2• Oracle Databases

ERP and Portal Systems• SAP Business Suite• SAP DMS• IBM WebSphere• Oracle WebCenter • Interaction (PlumTree

Search and Cloud Systems• Microsoft SharePoint Online• Google Drive• SharePoint 2013, 2010, 2007,

FAST Search for SharePoint• Microsoft Search Server• Box• Scopus• PharmaCircle

Plus a proven architecture and process for creating new connectors to complex systems

OOTB Search Center, plus any and all tailored search experiences

Shows external content (Connectors) and consistent metadata (AutoClassifier)

External Content in O365 UX

Unified view across all content - on-premises and on-line- inside and outside SharePoint

Cloud SSA

External Content(on-premises

and/orin the cloud)

SharePoint Server(On-premises or Hosted)

SPO ContentOneDrive Content

Conn

ecto

rs SharePoint Content

ConnectorFramework

AutoClassifier

Office 365

AutoClassifier(app

version)

O365Search Index

CEWS

Hybrid Connectivity and Classification

MMS Sync: add-on for SP2013, coming OOB for SP2016

DLP Sensitive Data Search works with hybrid

Search for sensitive data across on-premises and SharePoint Online

All Built-in sensitive types

Identification and export

Extends to data in OneDrive

Sensitive Information type detection through KQL searches

Get instant statistics

Preview & export results

Cloud SSA

External Content(on-premises

and/orin the cloud)

SharePoint Server(On-premises or Hosted)

SharePoint Online (SPO) Search

Delve & InfoPediaData Loss Prevention (DLP)

SPO ContentOneDrive Content

Conn

ecto

rs SharePoint Content

ConnectorFramework

AutoClassifier

Office 365

AutoClassifier(app

version)

OfficeGraphO365

Search Index

CEWS

Content and Metadata in O365 indexalso appears in the Office Graph

External content from BA Insight Connectors also surfaces in Delve, InfoPedia, etc. via the Office Graph

ScalingOn-Prem Footprint scales with content volume

Even the large farm is small

External Content(on-premises

and/orin the cloud)

Connectors

FileHandlers

/iFilters

CrawlerContent

Processing Index

CustomProcessing

Content Flow – on-prem indexing

CEWS

Bottlenecks:1) Source systems2) Content Processing3) Indexer….

External Content(on-premises

and/orin the cloud)

Connectors

FileHandlers

/iFilters

CrawlerContent

Processing Index

Content Flow – Cloud Hybrid Search OOB

CloudSSA

Bottlenecks:1) Uplink2) Source systems….

31

PerformanceUplink is nowthe bottleneck

Want a tool?

External Content(on-premises

and/orin the cloud)

OOBConnectors

FileHandlers

/iFilters

CrawlerContent

Processing Index

Content Flow – Gateway

BA InsightConnector

s + Framewor

k

SmartPipeline

CEWS

AutoClassifier Preview

CustomProcessing

Bottlenecks:1) Uplink2) Source systems3) Content Processing….

Succeeding with HybridSearch (split workloads) can need more than one mechanism

Walk through your scenarios

Mechanisms

Cloud SSA

Remote Result Source

Cloud App model

Applications

External Content

Federator

Identity and Directory Sync

• Security principals can be managed on-premises and synched to the cloud by using the AAD sync tool.

• The object in the cloud (AAD) directory now mirrors the object in the on-premises (AD) directory.

Directory Synchronization

AD AAD

AccountName

CORP\jaden

SID S-1-5-21-1212121212-1212121212-1212

AccountName

jaden@corp.hybridsearch.com

msOnline-OnPremiseSecurityIdentifier

S-1-5-21-1212121212-1212121212-1212

PUID PUID-XXXX-XXXXXXXXXX

Mapping of Access Control ListsAs items are indexed in Office 365, the access control entries are looked up in the cloud directory service.

Allow: S-1-5-21-1212121212-1212121212-1212 Allow: PUID-XXXX-

XXXXXXXXXX

• User SIDs are mapped to PUIDs• Group SIDs are mapped to Object IDs• «Everyone» and «Authenticated users» are mapped to

«Everyone except external users»

1) SaaS service will have less control and extensibility

2) Set of features unavailable with Cloud SSA/O365 index

3) No query-side “hook” for extensibility

Cloud Hybrid Search Limitations

37

Feature OOB Limitation BA Insight CEWS not available with Cloud SSA available via connector frameworkEntity Extraction not available with Cloud SSA available via autoclassifierCustom Security Trimming not available with O365 index can 'map down' to AD groups

ThesaurusSharePoint Online doesn't support a thesaurus can use Federator - with SP server-

based search centerRemoval of on-premises search results not available with Cloud SSA (could provide a custom solution)

Cloud SSA

External Content(on-premises

and/orin the cloud)

SharePoint Server(On-premises or Hosted)

SPO ContentOneDrive Content

Conn

ecto

rs SharePoint Content

ConnectorFramework

Office 365

AutoClassifier(app

version)

O365Search Index

CEWS

No CEWS?Use BA Insight Framework/Smart Pipeline

CustomProcessing

Advanced Security – Workaround for: no custom security trimming with Sloud SSA– Crawling SP2010 with Classic Authentication– Multiple Domains

Export Restricted Content Specialized Content Processing

NOT OOB …. but there’s a way to handle them all

Scenarios

39

40

Connectors

Federator

Check & block

Data Export Restrictions:Crawl local content locally and federate

OOB Federated Search User Experience

Results from Cloud

Results from SharePoint On-Premise

Refiners from Cloud only

No termsetsynchronization

Result Blocks(not interleaved)

BA Insight Federator

44

Full Range of Hybrid Search Configurations

ScenarioMost systems and portals

hosted on-premisesMost systems and portals

hosted in the cloudMust work across borders but

maintain data residencySingle Single MultiSearch

Search Index Search Index Across Multiplein SP Server in SP Online * Search Indices

How it worksCrawls SP Online and

other sources from SP Server

Crawls SP Server and other sources from Cloud SSA, pushes

text & metadata to SP Online

Searches SP Online and SP Server simultaneously; combines the results

AdvantagesSimplest approach;

best search experience

Low footprint on-premises;can use online features

(Delve, DLP)

Only solution for some scenarios

BA InsightImprovement over OOB

no OOB solutionAdds content outside SP

Preview content outside SP2013Supports content enrichment

Provides single interleaved result set and refiners

* requires Microsoft Cloud SSA

Approaches for Hybrid SharePoint

Configuration

45

Customer Example: ACE

Built on SharePoint 2013 – but couldn’t run as-is in O365

SharePoint Server in Azurein hybrid configuration with O365 Tenant

Virtual Network

Cloud Service

Availability Set

Active Directory & DNS

Cloud Service Cloud Service

Availability Set

Front End

Availability Set

App server

Availability Set

Database

Microsoft Azure

Gatewaysubnet

Active VPN

On-premises environment

Optional!

BA Insight

Apps

Example: Using Search-First Migration with Hybrid

Cloud Service

Availability Sets

SharePoint Services Farm

Microsoft Azure

SharePoint Online

Site collections

Office 365 Tenant

SharePoint 2013 Content Farms

SharePoint 2010 Farm(s)

2) Migrate / UpgradeContent Farms

Each site collection can be moved independentlyCan be on-premises, in O365, or hosted in Azure

3) Decommission old farm(s)

1) Establish Search Service(using Azure IaaS)

Succeeding with HybridIt’s a project

There are resources and tools to help

Expect to iterate

Lay out your strategy and approach– Understand your scenarios and select the appropriate

configuration and implementation options up front Ready your team for the change before you execute

– Test and Train with Azure & O365 Take advantage of your move to improve

– Don’t just ‘lift and shift’; do smart migration

Getting Ready

50

Key Considerations for Hybrid: Workloads, Environment, Data, Customizations

Availability of features Online versus On-Premises on particular workloads

Significant investments in customization of On-Premises workloads

Concerns over global network performance with remote sites

Regulatory considerations

Manageability concerns

Tools Cloud Migration Assistant Hybrid Scenario Picker Bandwidth Calculator Pros/Cons Map

53

New Sites to bookmarkHybrid.office.com

(useful soon)

Domorewithsearch.com(will post links)

54

Succeeding with HybridStart with your goals and strategyLay out your workloads, scenarios, & constraintsLeverage the new cloud hybrid searchAdd in external contentRun a phased project

You can get started hands-on now

Contact:Jeff.Fried@BAinsight.comwww.BAinsight.com

Questions /Discussion