HAUFE CONTENT HUB – TURN CONTENT INTO APIS · SCENE SETTING: HAUFE GROUP Seite 3 2016-06-14...

HAUFE CONTENT HUB – TURN CONTENT INTO APIS

Christoph Ludwig [email protected]

HAUFE CONTENT HUB

2016-06-14 Christoph Ludwig: Haufe Content Hub – Turn Content into APIs Seite 2

› Central content discovery and exchange platform

› within the Haufe Group and

› with partners.

› Developed under new Haufe Technology Strategy

› First release planned summer 2016

SCENE SETTING: HAUFE GROUP


Akademie Trainings,

e-Learnings

CCP Consultants,

Corporations & Public Sector

SBC Small

Businesses & Consumer

RLN Real Estate, Lawyers & Non Profit

H2/H3 Product

Innovation

Umantis HR Software

LSL Literature

Procurement

VCW Publisher

(Controlling)

HLRE Real Estate Software

Hammonia Publisher

(Real Estate)

Haufe Inc. US Market (HR)

New Times Corporate Publishing

Schäffer-Poeschel

Publisher (Taxes, Law, Economics)

Smart-steuer

Taxes SaaS

Semigator Trainings

Procurement

HAUFE: THE ROOTS

Christoph Ludwig: Haufe Content Hub – Turn Content into APIs

Books

Loose-leaf editions

Desktop content databases (1990s)

2016-06-14 Seite 4

HAUFE TODAY

Christoph Ludwig: Haufe Content Hub – Turn Content into APIs

Online Content Databases Haufe.de News Site

Booking platforms for seminars & trainings

Books & Print Products

2016-06-14 Seite 5

SaaS solutions (accounting, tax, HR, …)

… and more!

Academy Shop Content

SBC Online Trainings

CCP Online Trainings

SAME OLD STORY: CONTENT SILOS EVERYWHERE


Bulk-produced Content Products

Subject-specific News

Partner Content

Academy Training Materials

Customer Content

SBC Shop Content

CCP Shop Content

Academy Shop Content

SBC Online Trainings

CCP Online Trainings

WELL-TRIED APPROACH: CONTENT HUB


Bulk-produced Content Products

Subject-specific News

Partner Content

Academy Training Materials

Customer Content

SBC Shop Content

CCP Shop Content

Content Hub

Composable Enterprise

DIGRESSION: HAUFE TECH STRATEGY


Evolvable

Rapid Onboarding

Operation at Scale

Efficiency at Scale

Open to Partners

Customer-centric

Interoperability

http://www.freeimages.com/photo/toy-blocks-1227451

STRATEGY IMPLEMENTATION: APIS


Ease Integration

Partnership Development, Extend Reach

Innovate through Product Unbundling

Decouple Service Consumers and Providers

New Business Models

Insight and Control over API success and usage

APIs

CONTENT HUB BUILDING BLOCKS


Content Producer Content Consumer

API Gateway

Content Hub Implementation

Ingest API Search / Retrieval API

Cross-cutting concerns: › Authorization › Rate limiting & metering › Usage analytics › …



Bulk Content Producer Content Consumer

API Gateway

Content Hub Implementation

Search / Retrieval API

Incremental Content Producer

Bulk Ingest API

Single Doc Ingest API

File Storage API



Bulk Content Producer Content Consumer

API Gateway Bulk Ingest API



Incremental Content Producer

Ingest Service Search & Retrieval Service

Admin UI Tmp Bulk

File Storage

File Storage API

CONTENT HUB API PRINCIPLES


Guiding Principles:

› Outside-in approach: Client convenience drives the API, don’t leak implementation details!

› Ease of use: Make client developers quickly productive:

› adhere to common conventions;

› provide reasonable defaults;

› provide developer guides and comprehensive documentation;

› Self-descriptive: Clients learn about potential next steps from embedded hypermedia controls.

Public at http://dev.haufe.com/api-style-guide/

http://dev.haufe.com/api-style-guide/

CLIENT-DRIVEN SEARCH API


› “Customer” interviews:

› Standard features (full-text search, wildcards, constraints, faceting, pagination, …)

› Group-by / result aggregation

› Suggest

› Search expression language:

› Inspired by well-known search APIs

› Lends itself to OpenSearch descriptions

› Results returned as OpenSearch v1.1 response based on Atom collections

Hit # Doc Id Score Property

1 23.4.67 0.9746 a

2 20.1.08 0.9255 b

3 43.9.64 0.8843 a

4 30.9.53 0.8802 c

5 20.2.30 0.7945 b

6 18.7.42 0.7769 d

Hit # Doc Id Score Property

1 (1) (3)

23.4.67 43.9.64

0.9746 0.8843

a a

2 (2) (5) (9)

20.1.08 20.2.30 21.5.23

0.9255 0.7945 0.5810

b b b

3 (4) 30.9.53 0.8802 c

CONTENT HUB SINGLE DOC INGEST API


› Synchronous

› Based on the Atom Publishing Protocol

› Every producing application has its own set of Atom Pub feeds.

› Document ingest uses the usual Atom publishing operations.

› UBER (Uniform Basis for Exchanging Representations) data embedded into Atom service document provides request template for Atom extension (document lookup).

› Supports (but does not require) XML-binary Optimized Packaging (XOP) for attached blobs.

Single Doc Ingest Service Document

Producer App A Atom Workspace

Producer App B Atom Workspace

App A Content Docs

Atom Collection

App B Content Docs

Atom Collection

App A Contract Docs

Atom Collection

App B Contract Docs

Atom Collection

BULK INGEST API


› Asynchronous

› Bulk ZIP archives: Multiple GByte (up to millions of XML documents + some binaries)

› Producer does not necessarily control publication

Bulk Content Producer

Ingest Service

Tmp Bulk File Storage

API

Gat

eway

Bul

k In

gest

API

Fi

le S

tora

ge

API

Publisher

1. Create ingest job Response: job ID, storage URL

2. Upload archive to storage URL

3. Approve ingestion

4. Ingest archive when scheduled

DEPLOYMENT & OPERATIONS


Composable Enterprise

Evolvable

Rapid Onboarding

Operation at Scale

Efficiency at Scale

Open to Partners

Customer-centric

Interoperability

APIs

APIs APIs

APIs APIs

Tech Strategy: Efficiency and operation at scale by

› fully automated

› cloud deployments.

http://www.freeimages.com/photo/toy-blocks-1227451

In May 2016, Haufe Api Management Deployment took

15 secs

WHAT DO WE WANT TO ACHIEVE?


Speed

Business Value In Manual Deployments?

Reduce Error

Prepare For Change

Make it repeatable Write it down Include testing Start early in the process Automate

Hoster, on premise, cloud(s) Different technologies Business scenarios change

AZURE DEPLOYMENT: Building Blocks Become…


API Gateway Bulk Ingest API



Ingest Service Search & Retrieval Service

Admin UI Tmp Bulk

File Storage

File Storage API

Doc

ker

Hos

t Ingest Service

Search & Retrieval Service

Admin UI

Doc

ker

Hos

t Ingest Service


Admin UI

Doc

ker

Hos

t Ingest Service


Admin UI

AZURE CLOUD DEPLOYMENT


Azure API Mgmt Gateway Bulk Ingest API



Azure Blob Storage Azure Premium Storage

MarkLogic VM

Azure Premium Storage

MarkLogic VM

Azure Premium Storage

MarkLogic VM Stateless Docker

Containers w/ Spring Boot Apps

IaaS Offerings

PaaS Offerings

DEPLOYMENT AUTOMATION W/ GO.CD


Maven Repo Docker Registry

Build Service Jars

Build Docker Images

Deploy Test Env

Deploy Prod Env

Build Base VM Images

Azu

re

Azure VM Images

Content Hub Test Environment

Content Hub Prod Environment

Security Scan

SUMMARY


Exposing APIs only: › Decoupled Content Hub implementation from clients;

› Freed implementation from cross-cutting requirements (authorization, rate limiting, usage analytics etc.)

› Simplifies on-boarding of new clients and reaching out to new partners

Fully automated cloud deployments: › Developed in collaboration with operations partner Ebcont;

› Up-front effort, expected to amortize soon;

› Reproducible environments;

› Unburdens dev / ops team;

› Security scans shifted into the build pipeline;

› Precondition for operating and evolve the application at scale (with a small team)

HAUFE CONTENT HUB – TURN CONTENT INTO APIS · SCENE SETTING: HAUFE GROUP Seite 3 2016-06-14...

Documents

Transcript of HAUFE CONTENT HUB – TURN CONTENT INTO APIS · SCENE SETTING: HAUFE GROUP Seite 3 2016-06-14...