Post on 18-Oct-2020
Aaron Lopez, Jeff DeWeese
ArcGIS Enterprise:Tuning and Scaling
Session Overview
• We will cover all the components of ArcGIS Enterprise:
- ArcGIS Server- Authoring and Tuning Services- Optimizing a specific service or many services
- Portal for ArcGIS- Operational tasks such as backups
- ArcGIS Data Store(s)- Three types for storing information
- ArcGIS Web Adaptor- An application for forwarding requests
- Enterprise Geodatabase- Outside the scope of this session
Infrastructure: A Key Component
• Environments can vary:- Bare metal (physical)- VMs (virtual)- Cloud and related services- Containers/Kubernetes: Up and coming…
• Placement of Storage Components- Server config store- Server directories
- Input, output, cache, jobs- Portal content store- Data Store data content directory
Service Fundamentals
Map Authoring: Tuning Individual Services
• Desktop maps do not always scale for the web• Focus your map
- Scale dependencies- Generalization- Definition queries- Remove unneeded layers- Hide fields you are not using- Annotations instead of labeling- Use same coordinate system for data frame and data
• Modify the ArcGIS Server cacheControlMaxAge property
General Data Considerations
• Match resolution of feature class to the accuracy of the data- If data is only at a meter, there is no need for millimeter resolution
• Index the file and enterprise geodatabase data appropriately
• For enterprise geodatabases- Maintain statistics- Use the recommended spatial type for the RDBMS
- st_geometry, native spatial type, etc…
Enterprise Geodatabases
• Enterprise geodatabase data- Fast
- Keep statistics up-to-date- Indexes on queried fields
- Current data availability- Traditional Versioning
- Fine for desktop editing, more complicated for the web- Versioned queries are expensive….access non-versioned layers when possible
- Branch Versioning- ArcGIS Pro and Enterprise- Designed for many concurrent users and the web - Simpler table structure
File-Based Data
• File geodatabases- Optimization
- Performs best when kept local to server- Best with static data- Make read-only
- Better than shapefiles- Performance- Scalability
- Greater storage limits
Service Models
Tuning ArcGIS Server – How Instances Handle Requests
• Request lifecycle for traditional, dedicated service
A single service instance can only handle one request at a time!
Tuning ArcGIS Server – Services Models
• Three models that services run under:- Dedicated (Traditional)
- Fine tune service settings- Has concept of service instances (also known as SOCs or ArcSOCs)
- Hosted- Utilizes the Relational Data Store of ArcGIS Enterprise
- Shared- Provides a group of instances always available for various services- ArcSOC based
Any service that references user-managed data (e.g. file geodatabase or enterprise geodatabase) -- is run under the traditional model
Tuning ArcGIS Server – Dedicated Services
• The number of service instances impacts:- Memory usage at rest (min setting)- Concurrent requests to be serviced (max setting)
- Wait time for getting access to an instance
• A common administrator goal:- Find balance between min/max
• Tools can help determine service popularity and performance- System Log Parser- ArcGIS Server statistics
Tuning ArcGIS Server – Dedicated Services
• For predictable performance use min=max- Default setting: min = 1, max = 2
- Review and modify min/max based on popularity- Cached service: set max = 1 to conserve memory
- Tile requests not serviced by ArcSOC processes- Be aware of swap space and page file
- Could be factor for popular services if min and max delta is very large
Tuning ArcGIS Server – Dedicated Services
• Avoid resource overload with traditional services- Avoid over allocation
• Concurrent users is not the same as concurrent requests
• Which will the deployment have? - Many services with low-to-moderate popularity - Few services with high popularity- Mixture of both
• Min=0 can be an option for seldom used services- Publish as a Shared service type instead
Tuning ArcGIS Server – Hosted Services
• Hosted services- Does not utilize service instances (i.e., ArcSOCs)
- Highly scalable number of services
- Data returned as JSON/PBF- Client-side rendering
- Data resides in the Relational Data Store- ArcGIS managed
Tuning ArcGIS Server – Shared Services
• Shared services – a game changer!- Utilizes a shared instance pool- Introduced at ArcGIS Server 10.7- Requires services published via ArcGIS Pro- Not all service types are supported
• Provides a group of instances always available for various services- Default is 4- Reduces memory footprint
• Ideal for “seldom” used services- Better option for “Min=0” configuration
Tuning ArcGIS Server – Shared Services
• Dedicated- # Instances = 8
• Shared - # Instances = 6
Map Service Min Instances Max Instances
LosAngeles 0 1Orange 0 1Venture 0 1Riverside 0 1SanBernardino 0 1PalmSprings 0 1NetworkAnalysis 2 2
Map Service Min Instances Max Instances
LosAngeles
4
OrangeVentureRiversideSanBernardinoPalmSpringsNetworkAnalysis 2 2
Tuning
Tuning a Web Map
• Hosted feature service considerations:- Read-only feature services are smaller, faster- If some editors and viewers are needed, use views
• Reduce clicks- The default extent should perform well
• Cache (tiles) can reduce amount of traffic
• Large amounts of feature service data can be slow and overwhelming- Aggregate data using smart mapping- New releases of Enterprise utilize on-the-fly quantization
Tuning a Web Map
• A typical setup:
Enterprise portal
Web Layer
Web Layer
Web Layer
GIS Server
GIS Service
GIS Service
GIS Service
Data Store
Dataset
Tuning a Web Map
• A more efficient setup: consolidate layers with like security into a single service
Enterprise portal
Active Wells
Proposed Wells
Wells by Status
GIS Server
Wells
Data Store
Wells
Tuning Portal for ArcGIS
• Login settings:- Identity and group stores can greatly affect login performance
- Active Directory with users in many groups- Impact greater on older releases
• Backups- Choose right combination of full and incremental- After first full backup, Portal will allow database to grow unbounded- Perform backups on regular basis to keep size under control
- Backup cadence (off hours)
Tuning ArcGIS Data Store
• Understand the three types of data stores available:
- Relational data store- Stores the data for hosted feature layers- PostgreSQL
- Tile Cache data store- Supports 3D Scene Layers and related services- CouchDB
- Spatiotemporal Big Data Store- High volume archiving from GeoEvent Server- Large result data from GeoAnalytics Server
Managing ArcGIS Data Store
• Many command-line tools for managing ArcGIS Data Store:- ArcGIS Data Store command utility reference
• -changedatastoremode (relational data store)- Toggle read-only mode
• -changedbproperties (all types)- Disk space thresholds, RAM heap size for spatiotemporal big data store
• -changebackuplocation (all types)• -updatebackupretaindays (all types)• -updatebackupschedule (all types)
• The describedatastore command gives insight into the current machine’s configuration
Scaling
Scaling Direction – Scale Up or Scale Out?
• Scaling Up - Adding resources to existing machines- Increases throughput without adding machines (+CPU)- Supports increased services instances (+Memory)
• Scaling Out - Adding servers that work as a unit- Manage capacity by adding more servers
• Both strategies can be used together- Scale Up to a reasonable point…then Scale Out
4 cores16 GB
8 Cores32 GB
16 Cores64 GB
8 Cores32 GB
8 Cores32 GB
8 Cores32 GB
8 Cores32 GB
Scaling Constraints – Scaling Up
• Scaling Up- Cannot scale up forever due to practical CPU
and Memory limits…talk with your IT Dept.- Remember, you need at least two servers to
support high-availability- Large servers can lead to large capacity
reduction if a server is unavailable
16 Cores64 GB
16 Cores64 GB
8 Cores32 GB
8 Cores32 GB
8 Cores32 GB
8 Cores32 GB
50% Capacity Reduction
25% Capacity Reduction
Scaling Constraints – Scaling Out
• Scaling Out- Portal High-Availability is limited to 2 servers (active/active)- Relational Data Store is limited to 2 servers (active/passive)- GIS Server sites are highly scalable but there are practical limits- GeoAnalytics Server supports one and three-server site configurations only- GeoEvent Server can be challenging to scale out- More servers = more Windows Server licensing- More servers = more administration
Tiered Deployments: Single-Tier
• Use Case- Small deployments (10 or less users)- Proof of Concept (POC)- Supporting another targeted function (e.g., a
Raster Analytics deployment)
• Components- Portal for ArcGIS + Web Adaptors + Hosting GIS
Server + Relational Data Store
Tiered Deployments: Two-Tier
• Use Case- Deployments where the hosting function is not
dominant and the primary purpose is enabling a broad set of GIS capabilities.
• Components- Web Tier: Portal for ArcGIS + Web Adaptors- App Tier: Hosting GIS Server + Relational Data
Store
Tiered Deployments: Three-Tier
• Use Case- Deployments where the hosting function is
dominant.- Large number of hosted services- Using spatial analysis tools (e.g., Insights for
ArcGIS)
• Components- Web Tier: Portal for ArcGIS + Web Adaptors- App Tier: Hosting GIS Server- Data Tier: Relational Data Store
Workload Separation: Drivers
• Manage scalability for different workloads- Data Management- Analysis- Visualization
• Maximize performance/scalability
• Reduce the risk of service interruption
• Align with licensing strategies
• Allow for effective monitoring/troubleshooting
Workload Separation: Separate GIS Server Role Deployments
• Licensing Roles- GIS Server (e.g., Mapping Site)- Image Server- GeoEvent Server- GeoAnalytics Server- Business Analyst Server- Notebook Server
• Other Roles- Hosting Server- Geoprocessing/Printing Server- Utility Network
Image ServerGIS Server GeoEvent Server
GeoAnalytics Server
Business Analyst Server
NotebookServer
ArcGIS Server Licensing Roles
Workload Separation: Example Architecture
• Four GIS Server Sites- Hosting Site- Mapping Site- Utility Network Site- Imagery Site
Workload Separation: Example “Large” Architecture
• Eleven GIS Server Sites- Hosting Site- Imagery Site- Viewer Site 1- Viewer Site 2- Viewer Site 3- Geoprocessing Site- ArcFM Web Site- ArcFM Mobile Site- Mapping Site- StreetMap Site- Utility Network Site
Thank you!