Pentaho Google Hangout - Simplifying Analytics Architecture for Big Data
EWD: Simplifying Web Application Architecture
Transcript of EWD: Simplifying Web Application Architecture
EWD:
Simplifying WebApplication Architecture
Rob TweedM/Gateway Developments Ltd
http://www.mgateway.comTwitter: @rtweed
A bit about me• 1980s:
– Lead Mumps developer at Royal MarsdenHospital, London: Hospital Information System
• Early 1990s: Management consultant at
Touche Ross, London• Major NHS Networking project for Dept of Health
• Mid-1990s – present:
– Caché/Mumps web app framework developer
– Javascript expert
– Internet technology industry watcher
A bit about me
• Pioneer of web development for Mumps
& Caché
– WebLink & WebLink Developer
• Sold to InterSystems: 1996
– Consultant for early CSP design
– Assisted most of InterSystems' major
customers worldwide on Web Application
design & development
A bit about me
• Author of EWD
– 2004 – present
– Advanced web/ajax development framework
• EWD:
– Supports the largest Internet-facing Caché-based
web applications
• World's largest Caché-based web app:
– Quest Diagnostics Care360 "EziOrder"
» 30,000+ concurrent users
» 500,000 test requests/day
Why Do I Despair?
• This is the industry-standard way it's done, right?
• So what's the problem?
Why Do I Despair?
• This is the industry-standard way it's done,
right?
• Yes, but nobody seems to ask the basic
question:
– How on earth did something so simple (the web) end
up being such a complex mess of stacks and layers?
– Web apps were supposed to make things simpler!
What the "Experts" miss
• The technology on which VistA was built (Mumps) turns out to also be a technology that could have been designed specifically as a web application platform– Almost no layers or moving parts needed– The entire web application development framework,
runtime AND application(s) can run on a single technical platform
• Mumps/Caché: the world's most misunderstood database technology
So why is this kind of nonsense the
accepted norm for web apps?
When it could be as simple as this:
Browser
Web
Server
(Apache/
IIS)
Gate
way
Caché
http/
https
WebLin
kin
terfa
ce
Caché
CS
P in
terfa
ce
Caché
Gate
way
EWDRuntime
VistA
Page Scripts
ie: the "experts" propose this:
When we could have this:
Why does this matter anyway?
Why does this matter anyway?
• 1 day of coding will cost
13 days of maintenance
• Fewer moving parts and
layers means that one person can understand
the entire application
• Remember: the people
who will maintain the application are rarely the
original developers
So why is this kind of nonsense the
accepted norm for web apps?
To understand:
• We need to go back through the history of the web
Why was the web successful?
• "Good enough" at the right price
– Free!
– Simple architecture
– Simple to build web pages
• No special tools required
• Anyone could create a web site
– Simple to access web pages
– Basic graphical UI
• Designers loved it and pushed its limits
"Static" web architecture
Browser Web Server File Store
Apache
IIS
HTTP
Request
"Static" web architecture
Browser Web Server File Store
Apache
IIS
HTTP
Response
Step2: Dynamic Pages
• Instead of fixed data in my pages, I want to display the latest figures in a table
– Dynamically generate the data in the
document
How to do this?
Browser Web Server File StoreHTTP
Response
Database
Dynamically
build a page of
HTML
Enter CGI
Browser Web Server File StoreHTTP
Response
Database
CGI Interface
But databases don't have a
scripting environment, so:
Browser Web Server File StoreHTTP
Response
Database
CGI Interface
Scripting
Language
eg Perl
Step 3
• This is ~1995, the era of Client/Server
• Lots of problems emerging:
– Deployment and maintenance of thick clients
– Complexity of development:
• Logic split between client & server
• Multiple languages & layers (sounds familiar??)
Step 3
• Why not use the browser as a universally-available standard smart UI?
• Move our application logic all back to the back-end
• Simulate an interactive UI session via a succession of generated HTML pages
• Web Apps were born!
Problem 1:
• HTTP is stateless
• An interactive user application requires maintenance of state
– Persistent information available throughout
the user's session
Problem 2:
• Browser still just communicates with the web server using just HTTP requests
• So, how to deal with security?
– How can we uniquely and reliably recognise
the incoming request as being from the
person they claim to be?
– How can we protect an individual's session
from malicious access?
Combine Problems 1 & 2:
• How can we reliably determine who the user is from the incoming request?
• How can we determine what they want to do on the basis of their request?
• How can we recreate their session state before handling their request?
The layers start to appear
Browser Web Server File StoreHTTP
Response
Corporate
RDBMS
CGI Interface
Security
Management
State
Management
HTML Page
Generation
Security Management
Browser Web Server File StoreHTTP
Response
Corporate
RDBMS
CGI Interface
Security
Management
State
Management
HTML Page
Generation
User
Credentials?
State Management
• Requirement:
– High-performance, highly scalable key/value
store
– Ideally not just simple key/value store
– RDBMS too heavy, slow
– State management database requires
something more dynamic and free-form
Enter In-memory Key/Value Stores
• memcache
• Redis
For more complex session data:
• NoSQL "Document" databases:
– MongoDB
• NoSQL Columnar databases
– Hadoop
– Cassandra
More layers appear
Browser Web Server File StoreHTTP
Response
Corporate
RDBMS
CGI Interface
Security
Management
State
Management
HTML Page
Generation
User
Credentials?NoSQL
Database(s)
CGI too slow, so fast APIs
Browser Apache File StoreHTTP
Response
Corporate
RDBMS
ISAPI / FastCGI
Security
Management
State
Management
HTML Page
Generation
User
Credentials?NoSQL
Database(s)
"Proper" scripting languages
• Perl too weird and clunky
• "Professional" languages needed
– Enter .Net and Java
• While we're at it, lets encapsulate all those other layers in our own special flavours
• "Hide" all that complexity by creating our own new complexity
Let's rationalise app design
• Model/View/Controller (MVC)
• Mapping URLs to resources (dynamic pages)
• Model for the data
• Views define the returned content
• Controllers access the database and business logic
And so we end up with, eg: J2EE
A complete nightmare
• Everyone pretends to understand it
– Nobody actually does
• It's supposedly designed to be manageable
– Nobody really knows how to manage it because nobody really understands it
• If programmers can barely understand it, what
chance do designers have?
– Design, not programming, is what makes an application a success
Result
• Timescales slip
• Budgets soar
• Staff turnover:– the people who originally designed it have
gone and new people can't understand what they've inherited
• Version 1 maybe eventually gets delivered
• Don't hold your breath for versions 2 and later!
Web Applications
• They were supposed to be simpler than Client/Server, right?
• The industry has managed to turn them into as big a mess!
A better way?
Wind back to 1994
• I realised that web pages could be generated dynamically
• The browser could provide all the UI capabilities that we'd required for clinical applications at the Royal Marsden Hospital
Mumps: the World's Least
Understood database
• Databases normally need to be accessed and manipulated via a separate scripting language
• Mumps & Caché are almost unique in that they have an integrated, tightly-coupled scripting language
– Mumps database
– Mumps language
Mumps: Tailor-made for the web
• Very high peformance, highly scalable language & database
• Interpreted, "fast and loose" language
• Dynamic, schema-free hierarchical database– "Universal" NoSQL capabilities:
• Key/value store
• Document store
• Columnar store
• Graph store
• Native XML Database / DOM
• Relational & Object (aka Caché)
Mumps: the layers disappear
Browser Web Server File StoreHTTP
Response
ISAPI / FastCGI
Caché / Mumps
Web Application
Management
EnvironmentCorporate Applications
Session Storage
EWD: back to simplicity
• High-level abstraction for app design– To the developer, it's as if you're designing using
static pages
– Native XML DOM-based compilation stage
• State & Session management happens automatically
• No baffling MVC complexity
• Simple, intuitive concepts
• High-speed development
• Simple maintenance
EWD makes it as simple as this:
Browser
Web
Server
(Apache/
IIS)
Gate
way
Caché
http/
httpsW
ebLin
kin
terfa
ce
Caché
CS
P in
terfa
ce
Gate
way
EWD
Runtime
VistA
* Page Scripts
* EWD
Pages
EWD Session
Store
Caché or GT.M
What about non-Mumps
integration?
• Several alternatives:– Ensemble
• Integration engine built on top of Cache
• Adaptors for most other databases and languages
• EWD is 100% compatible with Ensemble
– Web Service access• EWD can consume external web services
– Behave as an HTTP client
– Node.js (server-side Javascript) opens up new alternatives
• ewdGateway.js: Node.js gateway module for EWD
• Use other Node.js modules for integrating other DBMS
Ensemble Example:
Browser
Web
Server
(Apache/
IIS)
Gate
way
Caché
WebLin
kin
terfa
ce
Caché
CS
P in
terfa
ce
Gate
way
EWDRuntime
VistA
Page Scripts
Ensemble
Adaptor DoD
EWDPages
EWD SessionStore
iEHR
• You don't need to go down the industry-standard route
• EWD is tried-and-tested technology
• Direct integration with ExtJS v4 and Sencha Touch (for mobile)
• It is demonstrably orders of magnitude faster to develop with and cheaper and simpler to maintain
Which would you feel safest in?