ERDAS APOLLO PERFORMANCE BENCHMARK ... APOLLO Performance Benchmark April 14, 2014 2 Contents...

ERDAS APOLLO PERFORMANCE BENCHMARK ECW DELIVERY PERFORMANCE OF ERDAS APOLLO VERSUS ESRI

® ARCGIS FOR SERVER

White Paper April 14, 2014

ERDAS APOLLO Performance Benchmark

April 14, 2014 2

Contents

Introduction .............................................................................................................................. 3

Sample Dataset ................................................................................................................... 4

Test Hardware ..................................................................................................................... 5

Summary ................................................................................................................................... 6

Methodology ............................................................................................................................. 7

Server Configuration ........................................................................................................... 7

Benchmark Configuration .................................................................................................... 9

Results ...................................................................................................................................... 12

Response Time ................................................................................................................... 12

Hardware Performance ....................................................................................................... 13

ArcGIS for Server during load test ................................................................................. 13

APOLLO Essentials during load test ............................................................................. 14

Data Transfer ...................................................................................................................... 15

Response Time Distribution ................................................................................................ 15

Response time vs Users ..................................................................................................... 16


April 14, 2014 3

Introduction This whitepaper aims to contrast the performance characteristics serving the Enhanced Compressed Wavelet (ECW) file format from ERDAS APOLLO against Esri® ArcGIS for Server. Even when the storage format and hardware is identical, there are substantial differences in how geospatial server applications deliver this data from a performance point of view. Critical enterprise deployment questions include:

Response time under low, consistent user load

Response time under increasing user load

Response time consistency

Response time across various view sizes

CPU & memory hardware utilization

Bandwidth requirements

Cost effectiveness

All these concepts are addressed in the following sections by performance testing the OGC Web Map Service (WMS) interface common to both server platforms.


April 14, 2014 4

Sample Dataset

Australia_Topographic_Map.ecw

File type: ECW Image

Dimensions: 146,105 x 122,280 px

Gigapixels: 18

Projection: EPSG:4326

Cell size: 0.000286 degrees

Structure: 3 Band, RGB UINT8

File size: 1,999,168 KB

Figure 1 Dataset extents

Figure 2 Native resolution sample


April 14, 2014 5

Test Hardware All performance metrics found in this whitepaper were performed on the following hardware.

Data drive throughput, reported by ATTO Disk Benchmark (4QD)

Warning: The intent of this whitepaper is to highlight peak performance of ERDAS APOLLO on given hardware, under specific, selected test plans. It should only be used as a relative guide when estimating any deployment, as characteristics will vary.


April 14, 2014 6

Summary

ERDAS APOLLO 2014 is 3.05x faster and more consistent at delivering the ECW file format than Esri® ArcGIS for Server 10.2 with Hexagon Geospatial’s “ECW for ArcGIS Server” installed.

50.34

2.54

170.33

150.57

5.96

649.01

0 100 200 300 400 500 600 700

600

2048

256

Requests / Second

Request size (px)

ERDAS APOLLO Essen:als 2014 vs

Esri ArcGIS for Server 10.2

APOLLO Essen5als

ArcGIS for Server


April 14, 2014 7

Methodology

Server Configuration 1. Configure entry level 4-core test server, “PALE” with the latest software releases

a. ArcGIS for Server 10.2

i. With ECW for ArcGIS Server 2014

b. APOLLO Essentials 2014

2. PALE represents:

a. Windows Server 2008 R2

b. IIS 7.5

3. Configure test dataset for ArcGIS for Server

a. Add ECW data to ArcGIS for Desktop, add ECW > Publish to Server

b. Enable WMS service capability

i. http://pale:6080/arcgis/services/largeECW/MapServer/WMSServer?

Figure 3 - Publish ECW to ArcGIS for Server from ArcGIS for Desktop

4. Tune ArcGIS for Server to obtain highest throughput possible

a. Test case proved the highlighted configuration to be preferred

i. High isolation, 1->2 instances = 121ms average (default setting)

ii. Low isolation, 1 instance, 8 processes = 157ms average

iii. High isolation, 1->4 instances = 79ms average


April 14, 2014 8

5. Tune ECW for ArcGIS Server ECW cache

a. A value of 10% yielded the highest throughput

i. <SDKCacheSize>0.01</SDKCacheSize>

b. 10% of 24GB total system memory adds ~240MB to each ArcSOC.exe. So we can estimate under load, each ArcSOC should not exceed 500MB

6. Configure test dataset for ERDAS APOLLO Essentials

a. Connect to Server > Add Image

b. Enable WMS Protocol > Press apply

i. http://pale:20140/erdas‐iws/ogc/wms?


April 14, 2014 9

Figure 4 – ERDAS APOLLO Essentials administration console

Benchmark Configuration 1. Three different test cases were identified.

a. Random 600 x 600px WMS views with stepped user load up to 20 users. Time bound to 6 minutes

b. Random 2048 x 2048px WMS views with consistent 2 user load. Request bound to 1000 requests

c. Random 256 x 256px WMS views with consistent 10 user load. Request bound to

2. Generate random WMS extents across the dataset, maintaining aspect ratio and using all scales via the RandomWMS command line tool. Save output to CSV for use within test plans


April 14, 2014 10

Figure 5 - RandomWMS generating extents for 256 image size

3. Configure JMeter test plan to capture the three tests. See appendix for full JMX

a. The stepping thread group used in the 600px test is defined in increments of 2 users, up to max of 20.

b. Basic thread groups for the other two tests, with 2 threads and 10 threads respectively

c. Configure HTTP Request Samplers to point to the relevant WMS end points to issue GetMap requests

i. Using BBOX defined in the CSV Variables $(BOX)

d. Configure relevant listeners to measure throughput and hardware utilization


April 14, 2014 11

4. Execute JMX test plan from remote load generator machine, connected via Gigabit Ethernet to the PALE server


April 14, 2014 12

Results

Overview

Label Requests Avg Min Max Std Dev Errors Throughput

(req/sec) KB/sec

Avg. Bytes

ArcGIS Server

600px 18126 229 33 845 125.3 0 50.34 5,792 117,815

ArcGIS Server

2048px 1000 785 262 1681 322.7 0 2.53 3,561 1,438,220

ArcGIS Server

256px 5000 53 15 175 18.8 0 170.33 3,439 20,675

APOLLO Essentials

600px 54208 75 12 819 42.3 0 150.57 12,283 83,536

APOLLO Essentials

2048px 1000 333 123 1285 159.1 0 5.96 5,729 983,796

APOLLO Essentials

256px 5000 10 3 199 7.5 0 649.01 9,444 14,901

Response Time


April 14, 2014 13

Hardware Performance

Both servers successfully met the expected 100% CPU utilization at 4 concurrent users

ERDAS APOLLO Essentials substantial 256px test throughput can be clearly seen by the Disk IO counters

ArcGIS for Server during load test

Figure 6 ArcGIS for Server during load test with 4 processes


April 14, 2014 14

ERDAS APOLLO Essentials during load test

Figure 7 – ERDAS APOLLO Essentials during load test with 1 process


April 14, 2014 15

Data Transfer

Response Time Distribution

As highlighted by the result summary, ERDAS APOLLO Essentials shares a far smaller standard deviation which is clearly seen on the response time distribution chart above. Consistency is very important for baselining and defining performance criteria to more accurately plan and scale server deployments


April 14, 2014 16

ArcGIS Server response time increases sharply once CPU Utilization reaches 100% at 8 concurrent users. While expected, the rate at which performance declines is far worse than ERDAS APOLLO Essentials which maintains a relatively linear, consistent trend ensuring more reliable performance can be assured under heavy load.

Conclusion The substantial 300% performance advantage seen with ERDAS APOLLO can be attributed to:

1. ERDAS APOLLO architecture

ERDAS APOLLO Essentials is a heavily optimized and multithreaded server. Unlike ArcGIS for Server that requires multiple processes to scale across CPU resources, ERDAS APOLLO utilizes just a single multi-threaded process to scale from 4 cores up to 32 cores which yields significant advantages including:

Higher memory cache hits

Lower memory requirements

Lower disk I/O requirements

No process duplication

Shared resources across map services

2. ERDAS APOLLO hardware acceleration

ERDAS APOLLO leverages additional CPU SIMD hardware acceleration above what’s available within the core ECW JP2 SDK shared by both servers. These optimizations ensure ERDAS APOLLO can generate imagery quicker than ArcGIS for Server, or indeed any other image server platform, across the whole image processing pipeline. This includes JPEG generation, to reprojection warping and resampling.

0

50

100

150

200

250

300

350

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Response :mes (m

s)

Users

Average Response Time vs Users

ArcGIS Server

600px

APOLLO

Essen5als

600px


April 14, 2014 17

3. ERDAS APOLLO Native C++ application

ERDAS APOLLO is a native C++ application that provides direct communication with the IIS ISAPI handler on Windows, or Apache FastCGI on Linux. This direct access path removes unnecessary latency or complexity that is typically required within other server applications. For example within ArcGIS for Server, where there are multiple bridges between Web Adaptors, Java application servers, inter-process communications and process management required to manage child processes (ArcSOC.exe).

4. Hexagon Geospatial are the image compression and serving experts

For over 15 years the ECW format and what is now the ERDAS APOLLO server platform have gone through continual improvement, and we constantly leverage our experience in this area to refine these solutions even further. Because we design and build both, we understand exactly how to architect a high performance image server while leveraging the core capabilities of the file formats including ECW.

Despite the performance difference, the availability of ECW for ArcGIS Server continues to address the demand from existing users of ArcGIS for Server. While it will not represent the full performance capabilities of the format, it still offers improved performance for many Hexagon Geospatial customers.


April 14, 2014 18

About Hexagon Geospatial Hexagon Geospatial helps you make sense of the dynamically changing world. Known globally as a maker of leading-edge technology, we enable our customers to easily transform their data into actionable information, shortening the lifecycle from the moment of change to action. Hexagon Geospatial provides the software products and platforms to a large variety of customers through direct sales, channel partners, and Hexagon businesses, including the underlying geospatial technology to drive Intergraph

® Security, Government & Infrastructure (SG&I)

industry solutions. Hexagon Geospatial is a division of Intergraph® Corporation. For more information, visit

www.hexagongeospatial.com.

Intergraph® Corporation is part of Hexagon (Nordic exchange: HEXA B). Hexagon is a leading global provider of

design, measurement and visualisation technologies that enable customers to design, measure and position objects, and process and present data.

Learn more at www.hexagon.com.

© 2014 Intergraph® Corporation. All rights reserved. Intergraph is part of Hexagon. Intergraph and the Intergraph logo are registered

trademarks of Intergraph Corporation or its subsidiaries. Hexagon and the Hexagon logo are registered trademarks of Hexagon AB or its subsidiaries. All other trademarks or servicemarks used herein are property of their respective owners. Intergraph believes the information in this publication is accurate as of its publication date. Such information is subject to change without notice. GEO – US – 0253A - ENG

ERDAS APOLLO PERFORMANCE BENCHMARK ... APOLLO Performance Benchmark April 14, 2014 2 Contents...

Documents

Transcript of ERDAS APOLLO PERFORMANCE BENCHMARK ... APOLLO Performance Benchmark April 14, 2014 2 Contents...