The Real World - Plugging the Enterprise Into It (nodejs)
description
Transcript of The Real World - Plugging the Enterprise Into It (nodejs)
THE REAL WORLDPlugging The Enterprise Into It.
MOBILE ARCHITECT ‣Love Distributed Systems ‣Entropy Reducer ‣Payment systems ‣R&D Work ‣B2E and Commercial Banking Apps
Experience ‣ Front Office Trading Systems ‣ Messaging Middleware Integration ‣ Big Systems ‣ C/C++/C#/Java
MORE ‣ @akohli https://slideshare.net/akohli
series 2, episode 22, “Daddy Pig’s Office” http://www.channel5.com/shows/peppa-pig/episodes/daddy-pigs-office
TODAYWhy Node
What we want to do
Node as the underpinning of real world or electronic asset interaction
Backing our interactions, eventing services
Not so much about monolith deconstruction
What we did
Initial proxy and protocol
Our performance and scalability testing
NODE IS MAINSTREAM
WHY NODE?✔ Node
• Asynchronous Eventing Model • We live in an async nonblocking
world • Ideal for mobile and sensor
applications • Everyone knows Javascript, right? • Community • Diverse protocol and lots of
modules • Rapid development and
Expediency
HOMOLOGATEDor how we can use it in a big company
• Node is approved for internal usage
• Less Yak Shaving than other solutions
• different at least
• good internal communitybeware of dog, staff only
- @adam_baldwin
“Walmart has had good success with HAPI and Node”
- @ eoinbrazil
“Node is good. I’ve heard good things about HAPI”
ENTERPRISE MOBILE APPLICATIONS
ENTERPRISE MOBILE APPLICATIONS
• Plurality of systems, services • web resources • web sites
• Connectivity challenges • direct • mediated
• Security • AuthN • AuthZ • Data Encryption at rest
THE REFLEKTOR
Security Pass
Employee DevicesSensors
The Physical World
Security Services
AuthZAuthN
…
Eventing Engine Bridge
Payment Services
Access Services
Printing Services
App Services and Resources
the ReflektorBridge and New Services
thePROXY First Release
IT AIN’T EASYbut we gotta try
PROXYING IS EASY
DETAILS
THE FLOW
• The Protocol
• Security - Gateway Access
• Federated Identity, my foot • NTLM, has it’s own approach
PROTOCOLRequest
json body
target
headers
body/post-data
loginfo
request = { URL = "http://www.citigroup.net/", method = "GET", timeout = 19500, clientInfo = { identifier = “…E”, model = "iPad Simulator", systemName = "iPhone OS", systemVersion = "7.1", }, headers = { Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", Cookie = "CGPLNG=ENG; JSESSIONID_CGNR3=..”, "User-Agent" = "Mozilla/5.0 (iPad; CPU OS 7_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Mobile/11D167" }, logEntries = [ { URL = “https://cinternal.site/target/fooa”, downstreamDuration = 656, httpMethod = "GET", roundtripDuration = 3461, statusCode = 200 } ] }
RESPONSE body = “<base64>", code = 200, duration = 31, headers = { "Accept-Ranges" = [ "bytes" ], "Content-Length" = [ 225 ], "Content-Type" = [ "text/html" ], Date = [ "Thu, 29 May 2014 15:28:29 GMT" ], Etag = [ "\"e1-4e50c74f\"" ], "Last-Modified" = [ "Sun, 21 Aug 2011 08:52:31 GMT" ] }, message = "OK"}
NTLM AUTHENTICATIONEnterprise authentication protocol (Microsoft). !NTLM requires all phases to take place across a single HTTP connection. !NTLM messages are sent and received as request headers. !The server’s response from the NTLM type 3 message is the requested content. !This authentication process must be completed for every requested resource, unless an open connection is maintained.
WORKINGImplementation Challenges • Storage of password on mobile device is prohibited,
but is required in the authentication process. • Persistent connection not available. • Latency issues – 3 requests for every web resource.
Solution • Ported from Apache Java implementation to Node.js.
• Hashed username / password pair stored on device, transmitted to server for authentication rather than raw password.
• hmac_md5(username, md4(password))
• NTLM message calculation split between client app and proxy server.
• Defaults used and optional parameters omitted – simplified messages.
• Observed desktop browsers wait for a 401 before beginning the authentication process. Pre-emptively sending the username / password hash eliminates the initial 401 response.
Process is reduced from 3 direct requests to a single client request, mapped to 2 proxy requests.
SCALABILITY AND PERFORMANCE TESTING
GITHUB.COM/SPUMKO/FLOD
$ flod -n 2000 -t 1500 -c 100..1000 -v http://target-place!!
GITHUB.COM/SPUMKO/FLOD
$ flod -n 2000 -t 1500 -c 100..1000 -v http://target-place!!
num req per batch
timeout
range of concurrent request per batch - “rate”
FLOD OUTPUT## 6k page results ec2-user@ip-10-199-51-233 node-hapi]$ flod -n 2000 -t 1500 -c 100..1000 -v http://localhost/loremipsum-6k-ish.htmlThis is Flod, version 0.2.2Copyright 2013 Walmart, http://github.com/spumko/flod!Benchmarking (hold on)...!Server Requests/sec Latency (ms) --------------------------------------- ------------ ---------------http://localhost/loremipsum-6k-ish.html 100 96.48 ± 18.54 http://localhost/loremipsum-6k-ish.html 200 164.24 ± 17.03 http://localhost/loremipsum-6k-ish.html 300 263.80 ± 62.44 http://localhost/loremipsum-6k-ish.html 400 359.61 ± 49.20 http://localhost/loremipsum-6k-ish.html 500 437.66 ± 58.69 http://localhost/loremipsum-6k-ish.html 600 481.29 ± 120.04http://localhost/loremipsum-6k-ish.html 700 606.74 ± 114.45http://localhost/loremipsum-6k-ish.html 800 555.08 ± 133.74http://localhost/loremipsum-6k-ish.html 900 674.08 ± 190.91http://localhost/loremipsum-6k-ish.html 1000 763.27 ± 69.25
## running with high timeout - doubling responses times vs nginx direct [ec2-user@ip-10-199-51-233 node-hapi]$ ../node_modules/flod/bin/flod -n 2000 -t 4500 -c 100..1000 -v http://localhost:8000This is Flod, version 0.2.2Copyright 2013 Walmart, http://github.com/spumko/flod!Benchmarking (hold on)...!Server Requests/sec Latency (ms) --------------------- ------------ ----------------http://localhost:8000 100 200.55 ± 39.40 http://localhost:8000 200 389.54 ± 67.39 http://localhost:8000 300 558.14 ± 112.57 http://localhost:8000 400 777.09 ± 160.01 http://localhost:8000 500 970.61 ± 305.76 http://localhost:8000 600 1032.37 ± 274.44http://localhost:8000 700 1216.49 ± 249.94http://localhost:8000 800 1483.31 ± 690.64http://localhost:8000 900 1559.54 ± 805.31http://localhost:8000 1000 1909.23 ± 845.81
MODIFYING FLOD
• modified server to pull our decorated response timing information
• modified reporting/logging to include this information
• hope to contribute back to mainline
ENVIRONMENT
• HTTP 1.1 no Keep-Alive, request payload is json • Client iOS ObjectiveC;Server is Node + Hapijs (with Some Good Monitoring)
Machine OS Type Processor Cores Memory
Int Server RHEL 6.4 VM Xeon 2.6GHz 2 4GB
Prod ServerWindows
Server 2k8r2
VM Xeon 1.8 Ghz 4 6GB
Dev Mac Mini Full i5 2.5 GHz 2 8 GB
SCENARIOS• Closed network, direct connection,
Mac to Mac
• Client server on a redhat VM, loopback. Redhat VM
• Redhat client to Windows Server via network, Redhat to Windows
• via Mobile network/wifi could only support 100 transactions/s because of latency
Req/s Response (ms)
Mac to Mac 1000 2000
Redhat VM 1000 8500
RH to Windows 1000 30, 000
External 100 17, 000
RESULTS• Consistent proxied service response
• ~20ms Mac ➔ Mac
• ~250ms RHEL ➔ Windows Server
• Gateway service < 50 ms
• We need better concurrency, request servicing
• Infrastructure adds significant overhead
RESULTS
100# 200# 300# 400# 500# 600# 700# 800# 900# 1000#0#
10000#
20000#
30000#
40000#
50000#
60000#
70000#
Requests'per'second'
Millisecon
ds(m
s)'
99th'Percen7les'
Mac#To#Mac# RedhatVM# Redhat#to#Windows#
OFF NETWORK
ON NETWORK
TABULAR Request Mac To Mac RHEL to Windows100/s 483 17,235200/s 783 21,209300/s 1127 17,479400/s 1493 22,937500/s 1859 28,872600/s 2171 34,253700/s 2487 39,878800/s 2878 46,970900/s 3285 51,0831000/s 3555 57,189
EXCELLENT
EXPERIENCE
• Enterprise and Legal approvals hard
• We are ahead of Ops, so waiting for VMs and infrastructure to catch up - software, machines, and network
• Some bits of node need tightening - especially around security and password storage
• Still learning and it is fun!
SCALABILITY PACKETS
• Pile of VMs to auto-scale
• Need elastic environment with a smart load balancer and configuration management
• Great Details on Best practice
• https://gist.github.com/hueniverse/7686452
THANKS!
QUESTIONS? COMMENTS?
NOUN PROJECTS THANKS
Smartphone designed by James Fenton from the Noun Project ! Creative Commons – Attribution (CC BY 3.0) Identification designed by Mark Shorter from the Noun Project Ibeacon designed by Stéphanie Rusch from the Nount Project ! Creative Commons – Attribution (CC BY 3.0) Arduino designed by uizin from the Noun Project !