Node Summit 2016: Building your DevOps for Node.js
-
Upload
chetan-desai -
Category
Software
-
view
222 -
download
0
Transcript of Node Summit 2016: Building your DevOps for Node.js
3
1. TurboTax’s Services Journey
2. Node.js in the TurboTax Stack
3. Enterprise Principles for Node Services
4. Best Practices for Publishing Node Modules
Topics
8
30+ million users filed with TurboTax • ~5 million used desktop • ~25 million used online
Mobile is the biggest growing segment
TurboTax is 25 years old • Roots as a Desktop App
12
Node.js in the TurboTax Stack
Application Services
Capability Services
Utility Services
User Experience Multi-device user experiences (cares about layout)
Application specific services (data interaction for UX)
Re-usable capabilities across applications (tax engine)
Data platform type services (login & identity)
13
My First Reaction I come from deploying in containers like JBoss & Tomcat
What do you mean the whole thing dies if there is an exception?! Run 30 million+ customers through it.
Consult some experts: NodeSource
Our DevOps practices was integral to our success
15
Enterprise Principles for Services
1. Isolation: Shared build farms
2. Reproducibility: Build once / deploy multiple times
3. Reliability: Minimize failure points during deployment
4. Availability: Keep server running
16
Build & Deploy Flow at Intuit
Build Farm
GitHub Enterprise
npm on-site
Artifact Repository
Intuit Data Center
17
Isolation: Shared Build Farms
• Problem: global npm dependencies
npm WARN prefer global <pkgName>@<ver> should be installed with –g
• No root access or sudo privileges
• Other node services with different versions
18
Isolation: Shared Build Farms • Solution: package.json npm
scripts!
• Add global dependencies to devDependencies section
• Add your CLI calls as npm scripts
• Execute Scripts • npm run <script>
19
The power of the package.json
Other Benefits
1. Local setup documentation: - scripts for building, testing, running.
2. Obtain consistency between developer workstations - at least compatible
3. Works in shared build environments! - dependencies localized
20
Reproducibility: Build Once / Deploy Multiple
Problem: Transitive Dependencies in package.json
• semver major.minor.patch • ~1.1.1 "Approximately equivalent to version”
• ^1.1.1 "Compatible with version”
• 1.1.1 "Specific version”
• Did you install version 2.0.0 or version 2.0.0? • Transitive dependencies still had ~ or ^ notation.
21
Solution: npm shrinkwrap –dev • Full transitive dependency list and versions installed
Landed somewhere in the middle • Don’t check in shrinkwrap.json
• Generate it once at build time
How to balance developer speed and compliance?
Reproducibility: Build Once / Deploy Multiple
Developer Speed Reproducibility / Compliance
22
Reliability: Fast and Reliable Deployments
Problem: when deploying, choose between 1. massive node_modules folder compared to the code size 2. dependency on an npm registry to download modules – potential failure point
Solution: • Reduce size of node_modules with npm prune –production
• Zip up the remaining contents with service code
• Never run npm at deployment time
• Our build OS (RHEL) is same as runtime OS
23
Availability: Keep Server Running
Problem: Process Management • “Let it terminate and restart”
• Lots of options: pm2, forever, strongloop pm, cluster
http://strong-pm.io/compare/
• Build & Deploy
• Clustering & Management
• Profiling
• Metrics
24
Availability: Keep Server Running
Enterprises have heterogeneous set of languages • Deploy, monitor, & existing features need to work across the stack
Who can manage processes better than the OS itself? • RHEL6 upstart & RHEL7 systemd
Upstart Approach • Multiple stateless processes, 1:1 with CPUs
• Load balanced & SSL termination with nginx
• If the process terminates, upstart restarts it
– Configurable respawn window
– Splunk for log monitoring
25
Recap: Node Services
1. Isolation: Shared build farms - localize dependencies through npm scripts
2. Reproducibility: Build once / deploy multiple times - shrinkwrap for transitive dependencies
3. Reliability: Minimize failure points during deployment - only run npm at build time - prune out dev dependencies
4. Availability: Keep server running - upstart (RHEL6), systemd (RHEL7)
27
Best Practices for Node Modules
1. Only publish what’s needed
2. Reproducibility: Tagging codebase
3. Traceability: Server side publishing
28
Publish only what’s needed
We’re all familiar with .gitignore - used to exclude: • generated files
• secrets
Use .npmignore to only publish what’s needed. Exclude: • Same items in .gitignore
• Test data and files
Understand .npmignore precedence There are default items ignored during a publication
.gitignore will be used if .npmignore is missing
.npmignore won’t combine with .gitignore
29
Tagging Codebase
Reproducibility: version bumps and tagging • maven-release-plugin for maven pom.xml
• bumpversion library for python setup.py
Solution: npm version [<newversion> | major | minor | patch …] for your package.json
Add git repository type for tagging support
Push commits back to origin
30
Server Side Publishing
Server builds for traceability and reproducibility
Local builds have too much risk • Forget to check-in a file
• Out of sync with other changes
Same applies for publishing node modules
How to provide publishing credentials from the server?
31
Server Side Publishing
.npmrc for npm configuration
Developers had to set a dummy value set for NPM_TOKEN
Error: Failed to replace env in config: ${NPM_TOKEN}
Solution: Split them up • Project home .npmrc specifies internal registry (line 2)
• Build user home ~/.npmrc contains credentials (line 1)
32
Recap: Best Practices for Node Modules
1. Only publish what’s needed - Use of .npmignore file
2. Reproducibility: Tagging codebase - npm versions to tag codebase - postversion script to push
3. Traceability: Server side publishing - Two .npmrc files for registry and credentials