Setting Up an Arc Hydro Based Geoprocessing

ESRI 380 New York St., Redlands, CA 92373-8100, USA TEL 909-793-2853 FAX 909-793-5953 E-MAIL [email protected] WEB www.esri.com

SettingupanArcHydrobasedGeoprocessingService

October 2011

Prepared by Esri Water Resources Team [email protected] Dean Djokic Zichuan Ye Christine Dartiguenave

Setting up an Arc Hydro based geoprocessing service

October 2011 i

Copyright 2011 ESRI All rights reserved. Printed in the United States of America. The information contained in this document is the exclusive property of ESRI. This work is protected under United States copyright law and other international copyright treaties and conventions. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storage or retrieval system, except as expressly permitted in writing by ESRI. All requests should be sent to Attention: Contracts Manager, ESRI, 380 New York Street, Redlands, CA 92373-8100, USA.

The information contained in this document is subject to change without notice.


October 2011 ii

Table of Contents

Prerequisites 3Setting up the data used by the geoprocessing service 4Setting up the delineation service 6

1. Setting up the model to publish in ArcMap 62. Publishing the service using ArcGIS Server 21

Testing the geoprocessing service in ArcMap 28Testing the geoprocessing service using the REST API 30

1. REST API in ArcGIS Server 302. Testing the service using REST 32

Testing the geoprocessing service using a Silverlight Client 411. Installing Silverlight 412. Setting up the virtual directory 423. Setting up the solution 42

Consideration for ArcSDE data 48


October 2011 3

1. Prerequisites

ArcGIS 10 with Spatial Analyst extension.

ArcGIS Server 10.

Arc Hydro Tools 2.0 with Water Utilities Framework: install the Arc Hydro Tools 2.0 for ArcGIS 10 by running the msi file. This will install the correct version of ApFramework as well.

Copy the Arc Hydro Tools toolbox installed by the setup under the desktop

toolboxes folder into the server toolbox folder so that the server can have permissions to access the tools from the Arc Hydro toolbox used in the published service. (If you have a prior version of Arc Hydro installed, make sure that the tool boxes are replaced by the installation program with the latest toolboxes. Some times during uninstall/install, the tool boxes are not deleted and not updated).


October 2011 4

2. Setting up the data used by the geoprocessing service

Create unc paths to the location where the input data used by the service will

be stored. Create the share(s) as \\servername\streamstatsdata for example.

Give read access to ArcGISSOC user to the share(s) and to the content of the folders on the disk as well.

Give yourself full control to the shared folder and to the content of the folder as

well.

Give ArcGISSOC user and yourself full security permission.


October 2011 5

In the ArcCatalog window, add a folder connection to the share and add the

input data using this connection.

To read more about how to share and connect to UNC paths, click the following link. http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Sharing_and_connecting_to_UNC_paths/00570000000m000000/


October 2011 6

3. Setting up the delineation service

3.1. Setting up the model to publish in ArcMap

Open a new ArcMap document and add the input data required by the model you want to create and then save the map in the shared folder. To test the local delineation, add the following input layers:

- Fdr - Str - Str100 (optional): used for snapping - Catchment - AdjointCatchment - Batchpoint

Note The hucpoly layer is just added to help visualizing the data used in this example and should not be added in the map as it is not a required input to the service.


October 2011 7

Create a new toolbox in the shared folder by right-clicking the connection in the Catalog window and selecting New > Toolbox. Rename this toolbox AHServer for example.

Create a new model in the new toolbox (e.g. AHDelineation): this model can

contain tools from any toolbox located in the server toolbox directory (i.e. you need to copy the Arc Hydro Tools toolbox to the server toolbox directory if not already done).

Create a folder connection to the ArcGIS Server toolbox directory.


October 2011 8


October 2011 9

Set the inputs using the data added into the Table of Contents and save the Model.


October 2011 10

Right-click the model to edit it. Right-click the model window and select Create Variable. Select Feature Set as variable data type and click OK. Right-click the new variable and select Properties. Activate the Data Type tab and set Import schema and symbology from: to BatchPoint using the full path. Click OK.

Warning If you use the layer name (BatchPoint) instead of the full path (by browsing to the layer), the service will fail when called using REST and you will see the following error message:


October 2011 11

Rename the FeatureSet variable PourPoint.

Note This is the variable name that will be used by the REST API.

Right-click the Batch Watershed Delineation tool and select Open:

- Set the Input Batch Point to PourPoint - Check the box next to Clear Existing Features (optional) - Make sure that the output path to Watershed/WatershedPoint are set to a file

geodatabase using the shared path. - Click OK to save the changes.

Right-click PourPoint and make it a Model Parameter.

Delete the Batch Point input and save the model.


October 2011 12

The output to the model must be saved to a scratch workspace before the model can be published. There is a lock that prevents currently from saving directly the output to the scratch workspace. This is why the model creates first the output in a shared workspace, and then copies the new features into the scratch workspace.

Add 2 Feature Class to Feature Class tools from the To Geodatabase toolset located in the Conversion Tools toolbox into the model.


October 2011 13

Right-click and open the first tool. Set Input Features to the Watershed feature class and Output Location to the geoprocessing variable %scratchworkspace%. Enter OutWatershed as Output Feature Class. Click OK.

Right-click and open the second Feature Class to Feature Class tool. Set Input Features to the WatershedPoint feature class and set Output location to the variable %scratchworkspace% as well. Enter OutWatershedPoint as Output Feature Class. Click OK.


October 2011 14

Rename OutWatershed.shp OutWatershed and OutWatershedPoint.shp OutWatershedPoint.

Right-click OutWatershed and OutWatershedPoint and make them Model

Parameters.

Right-click Watershed and WatershedPoint and make sure they are not set as Intermediate. Save the model and close it.


October 2011 15

Note The %scratchworkspace% will use the value set for the geoprocessing scratch worskpace when using the model in the desktop. The value used by the model published on the server will be the scratch workspace defined for the server. You now need to setup the scratch workspace in the desktop environment. You are going to set this variable to mimic the server environment.

In the Catalog window, right-click your shared folder and select New > Folder. Rename this folder scratch.

Open the ArcToolbox window by selecting Geoprocessing > ArcToolbox in

the main menu. Right-click the top ArcToolbox node and select Environments to open the Environment Settings window.

Click on the top Workspace node to expand it and set the Scratch workspace to

the new scratch directory under your shared folder. Click OK to save the setting.


October 2011 16

Notes Make sure that the Scratch Workspace Environment Settings is not set for the model or for the tools within the model or those settings will overwrite the environment you just set at the ArcToolbox level. Instead of storing the final output as shape files in the scratchworkspace, you can also store them as feature classes in the scratch geodatabase. To store the output data into feature classes in a scratch file geodatabase (scratch.gdb), you still need to point the Scratch workspace to the scratch directory, but you need to point the Output Location for the Feature Class to Feature Class tool to %scratchworkspace%\scratch.gdb instead of %scratchworkspace% and remove .shp from the Output feature class.


October 2011 17


October 2011 18


October 2011 19

You are now going to test the model to make sure it is properly setup.

Drag the AHDelineation model into the Table of Contents of ArcMap. Right-click the model in the Table of Contents and select Open to run it. Note

that the structure of the input PourPoint is imported from the Batch Point feature class as requested. You can leave the attributes blank for testing. By default, Batch Done is assumed to be 0 (not processed), SnapOn 1 (snap) and SrcType is ignored.


October 2011 20

Click on Add feature and then click on the map at the point where you want to perform the delineation.

One record is added in the table.

Enter a value for Name and Descript if desired then click OK to delineate this point.

The model is executed and the outputs are added into the Table of Contents below the AhDelineation group layer. Note that the outputs are stored in the specified scratch workspace.


October 2011 21

Save the map document.

3.2. Publishing the service using ArcGIS Server

Open the previous map document if needed.

Setup the symbology for the OutWatershed and OutWatershedPoint layers and save the map.

In the Catalog window, right-click the map document under the shared folder

connection ad selectPublish to ArcGIS Server. This will create both a map service used as Result Map Service and a geoprocessing services.


October 2011 22

Select the name of the ArcGIS Server and the name of the service you want to

create (map and geoprocessing). You can keep the name of the map proposed by default. Keep the defaults setting and click OK.


October 2011 23

Keep the default in the next window and click OK.


October 2011 24

Click Finish after taking note of the services that will be created to generate the services.


October 2011 25

Create an administrative connection to your ArcGIS Server in the Catalog window if it does not already exist. 2 new services have been created, a geoprocessing service and a map service.

Right-click the geoprocessing service and select Service Properties


October 2011 26

Note that the geoprocessing service has been created using a map (by adding the model in the Table of Contents) and that the map is set as Result Map Service. The service is created Asynchronous.


October 2011 27

Click OK to close the window. Stop the geoprocessing service, then click

Service Properties and check the box next to Show Messages. Click Apply, then OK and restart the service. This will give you more feedback messages when testing the services.

Save and close the map document.

You are now going to test the geoprocessing service in ArcMap, using a Rest API and using a Silverlight client.


October 2011 28

4. Testing the geoprocessing service in ArcMap

Open a new map document.

Add a non administrative connection to the ArcGIS Server publishing the

service if it does not already exists as follows: - In the Catalog window, under GIS Servers, double-click Add ArcGIS Server. - Select Use GIS Services in the first window and click next. - Specify the type of connection (Internet) or Local and enter the relevant

information. Click Finish.

You will now see the published model AHDelineation under the geoprocessing service you created.


October 2011 29

Add a basemap to the map (e.g. topomap) by clicking File > Add Data > Add Basemap and selecting USA Topo Maps (or another relevant basemap). Click Add to add the basemap into the Table of Contents.

Add any other layer that will help you find a location to delineate (e.g.

str100_all).

Double-click the published model AHDelineation in the Catalog window.

Click on Add Features then click on the map at the location where you want to perform the delineation. Then click OK.

The published model performs the delineation and generates the output OutWatershed and OutWatershedPoint as shapefiles in the scratch directory associated to the current job. The results are also added in the Table of Contents of ArcMap.


October 2011 30

5. Testing the geoprocessing service using the REST API

5.1. REST API in ArcGIS Server The ArcGIS Server REST API, short for Representational State Transfer, provides a simple, open Web interface to services hosted by ArcGIS Server. All resources and operations exposed by the REST API are accessible through a hierarchy of endpoints or Uniform Resource Locators (URLs) for each GIS service published with ArcGIS Server. When using the REST API, you typically start from a well-known endpoint, which represents the server catalog. The table of contents for this Help system mimics the hierarchy of resources in the REST API. The default start URL for an ArcGIS Server installation is:

Java: http://:8399/argis/rest

.NET:


October 2011 31

http:///arcgis/rest For example, to get to the root directory of ArcGIS Online services, the URL is: http://server.arcgisonline.com/arcgis/rest/services From this base URL, you can traverse to any of the listed service and resources and operations associated with the service. The REST API is stateless, which means that REST does not keep track of transactions from one request to the next. Each request must contain all the information necessary for successful processing. Services Directory Each ArcGIS Server comes with a Service Directory, which is a view of the ArcGIS Server REST API in HTML format. You can browse the contents of an ArcGIS Server and obtain information such as service metadata and supported operations that can be useful in developing applications. Resources and Operations The REST API has some level of support for each of the ArcGIS Server service types. Each service is a resource and has a unique URL. Although a REST system always returns only representations of resources to the clients, for the sake of simplicity, the resources of the ArcGIS REST API are divided into two types: resources and operations. The supported service types that include operations are map, geocode, geoprocessing, geometry, and image. Another set of service types that are supported as resources but have no associated operations include network, geodata, and globe services. One requirement of the REST API is that ArcGIS Server services must be configured as Pooled. Non-pooled services are not listed in Services Directory. Output Formats The REST API supports responses in several formats. The format is specified by using the query parameter f in the URL. The list of supported formats includes html (HyperText Markup Language), json (JavaScript Object Notation), image, kmz (compressed KML, or Keyhole Markup Language), help, lyr (layer file), nmf (ArcGIS Explorer map file), jsapi (JavaScript), ve (Virtual Earth), and gmaps (Google Maps). REST API Administration In addition to being a browser for your published data, Services Directory also provides access to an administrative console. The administration page is needed to refresh the site after adding, deleting, and updating services or after updating to a new version of the REST API. Services Directory can also be disabled through the console. For more information, see REST API Admin. REST API Admin URL http:////admin Description

The REST API supports an admin console. Assuming a default installation, it is available at the following URLs

Java Server: http://:8399/arcgis/rest/admin .NET Server: http:///arcgis/rest/admin The REST Admin is secured so that only users of the agsadmin group have permissions to access the admin.


October 2011 32

5.2. Testing the service using REST

Open a browser and enter the start URL for the ArcGIS Server hosting the published geoprocessing service (e.g. http://chrid/ArcGIS/rest, where chrid is the name of the server).

This url allows accessing the Service Directory window listing the available services. Note If you do not see the service you just published, you may need to clear the cache by connecting to the REST Admin console as follows:

Open a browser and enter the REST Admin URL for the ArcGIS Server hosting the published geoprocessing service (e.g. http://chrid/ArcGIS/rest/admin, where chrid is the name of the server).

Enter your login/password with administrative right when prompted (e.g. windows login if you have the appropriate rights).

In the Admin window, click Clear Cache Options.


October 2011 33

Click Clear Cache Now and reenter the start URL (e.g. http://chrid/ArcGIS/rest). You should now see the newly published services (e.g. CA_ss10).

Click on the new GPServer service (CA_ss10).

The geoprocessing service is asynchronous and is associated to the result Map Server CA_ss10. It supports both the REST and the SOAP interface. It has one associated task, AHDelineation, which is the model you published.


October 2011 34

Click on the AHDelineation task.

The window describes the directory the service and its parameters, as well as the supported API and operation (at the end).


October 2011 35


October 2011 36

Click Submit Job. This is the operation associated to asynchronous task.

The Submit Job: AHDelineation window allows entering the parameter(s) required to run the published geoprocessing service and to execute it.


October 2011 37

Enter the input PourPoint parameter as follows:

{ "features" : [{ "geometry" : {"x" : -2016360, "y" : 2332860}, "attributes" : {"BatchDone" : 0, "SnapOn" : 1}}]} Where x and y are the coordinates of the input point in the same coordinates as the underlying data supporting the delineation (e.g. Albers).


October 2011 38

Click Check Job Details Again to refresh the window with the current execution status.


October 2011 39

You can look on the server in the arcgisjobs directory for the folder associated to the job submitted (e.g.jfe3a9354e1ac42d6b0959cdca6b20f0c).


October 2011 40

Note You can access directly the job submission page for the service by entering the following url http://chrid/ArcGIS/rest/services/CA_ss10/GPServer/AHDelineation/submitjob where: AHDelineation is the name of the model in the published geoprocessing service CA_ss10 Submit job is appended because the service is asynchronous.


October 2011 41

6. Testing the geoprocessing service using a Silverlight Client

6.1. Installing Silverlight

Install ArcGIS Server GIS Services + post install (you may need to log out to refresh the permissions set during the installation process.)

Install ArcGIS_Server_NET_GISServices (Note: if you are experiencing some

issues when creating the services, you need to double-check that the permissions have been correctly set).

Install Silverlight by following the steps below:

http://www.silverlight.net/getstarted/

Or, if you already have Visual Studio 2010 installed, and prefer not to use the Web Platform installer, you can still get

everything you need by downloading the Silverlight 4 Tools for Visual Studio 2010.

System Requirements

Supported Operating Systems:Windows 7;Windows Vista;Windows XP Service Pack 3 o Windows 7

o Windows Vista Service Pack 2

o Windows XP Service Pack 3

Visual Studio Requirements: o Microsoft Visual Studio 2010 with the Visual Web Developer feature

o Microsoft Visual Web Developer 2010 Express

- Install Visual Web Development feature in Visual Studio if missing - Install Silverlight SDK - Install ArcGIS Silverlight API (2.0)

(http://help.arcgis.com/en/webapi/silverlight/) - Install Silverlight runtime (if missing)


October 2011 42

Copy the sample project SilverApp (Visual Studio 2010)

Open the solution file SilverApp.sln. If you are getting a message stating that the project is not supported by this installation, you may need to uninstall and reinstall the Silverlight runtime.

6.2. Setting up the virtual directory

Open IIS Create a new virtual directory (e.g. DelineationGP) and point it to

SilverApp\SilverApp.Web Give all permissions except write Set target to 3.5 Set IIS target asp.net to 4.0

6.3. Setting up the solution You need to edit the solution file to point to your service. This sample is using a synchronous service so you are first going to edit the service you published to make it synchronous.

In the Catalog window, stop the geoprocessing service, right-click it and select Service Properties.

Change it Execution Type to Synchronous (Note that the Result Map Service becomes disabled).

Click Apply, then OK and restart the service. Open the REST API Admin window (e.g. http://chrid/ArcGIS/rest/admin) and

clear the cache. Double-check the updated service in the service (e.g.

http://chrid/ArcGIS/rest/services/CA_ss10/GPServer/AHDelineation). Note that the supported operation is now ExecuteTask instead of SubmitJob.


October 2011 43

Open the solution file if not already done. The map services are defined in the file App.xaml.cs in the SilverApp directory. This sample uses:

- one basemap map service (tiled): http://services.arcgisonline.com/ArcGIS/rest/services/ESRI_StreetMap_World_2D/MapServer

- one dynamic map service layer used to display the HUC and stream grid used for snapping (you can use the map server created at the same time as your service in this example, but you should really create a specific map with all the layers that you want to display.)

- graphics layer to display the results from the delineation.

The application uses the synchronous geoprocessing service to perform the delineation. This is setup in the file MainPage.xaml.cs located in the SilverApp directory.

Point the dynamic map service to your published map and the geoprocessing

service to your published synchronous service and save the solution.

Compile the solution and run it to test it or enter the url to access your new application (e.g. http://chrid/delineationgp/default.aspx).


October 2011 44

Zoom in and out and pan using your mouse, then click on the map at the location where you want to perform the delineation.

The watershed is delineated and added to the map as well as its associated watershed point.


October 2011 45

Note:

A common mistake is to use the result map service to display a basemap for

your geoprocessing service. For example, if your user identifies a parcel by point-

and-click, and your task draws the identified parcel symbolized by some attribute of

the parcel, your initial thought would be to use the result map service to display the

input parcel data (the basemap) as well as the identified parcel. There are two

reasons why you should not use the result map service as a basemap:

When a result map service is added to the application, all layers in the map service are available for display. These layers include geoprocessing tool layers used to

draw output, layers that may contain sensitive data, or layers used by your

geoprocessing service but that make no sense to the user (like tool layers).

Basemaps are multiscale and multiresolution. As you zoom in and out, the basemap changes, showing details at large scales and aggregating details into

generalizations at small scales (for example, rivers change from line features at

small scale to polygon features at large scales). Constructing a multiscale and

multiresolution basemap that draws quickly is not something your result map

service needs to be dealing withits job is to draw outputs. You need to keep the

design and implementation of basemap map services separate from the design and

implementation of result map services.


October 2011 46

7. Considerations for ArcSDE data

7.1 Connection file ArcSDE connection files can be created as needed using the Create ArcSDE Connection File tool or using Add Spatial Database Connection option from Catalog->Database Connections. If the connection file is created using Database Connection option, it is stored in users temp location (e.g. C:\Documents and Settings\user\Application Data\ESRI\Desktop10.0\ArcCatalog\Connection to CUAHSI.sde). The required ArcSDE connection file is copied or created in the folder along with any other data required for the tool to successfully run.

Model and script tools you develop should be written to access the ArcSDE connection file found in the ToolData folder using a relative path to the connection file.

You may have noticed this is really no different than how you would share other geodatabase data, such as file geodatabase data. For file geodatabases, you also have to provide the file geodatabase folder (equivalent to providing the ArcSDE connection file) along with the tools or provide access to the file geodatabase on the network and have your tools access the data using UNC paths.


October 2011 47

7.2 SDE Access ArcGISSOC needs to be given permission to view/edit ArcSDE database. You can grant the permission by running Change Privileges tool from Data Management Tools. Browse to the datasets, from the database connection, for which the privileges need to be granted. Specify the SOC user account on the server and select GRANT for view/edit options.


October 2011 48

To read more about geoprocessing considerations for ArcSDE data, checkout this link.

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//002t0000000v000000.htm

1. ClientAccessPolicy.xml file needs to be created and put in web server root directory (e.g. c:\initpub\wwwroot) for cross domain permission. http://help.arcgis.com/en/webapps/silverlightviewer/help/index.html#//01770000000r000000.htm

To access data from a remote server, the remote server needs to have a clientaccesspolicy.xml file in the web servers root directory (for example, c:\inetpub\wwwroot). The ArcGIS Viewer for Microsoft Silverlight is built on the Microsoft Silverlight platform. For security reasons, a Web browser is not allowed to access data that resides outside the exact Web domain where Silverlight application (Map Web Part) originated. However, a Silverlight application can load data across domains if permission is granted from the server. This is accomplished by having a small clientaccesspolicy.xml file on the remote server that permits Silverlight to connect to services on that server. For additional information, read the topic Making a Service Available Across Domain Boundaries. For more information on the structure and function of a clientaccesspolicy.xml file, read the topic Network Security Access Restrictions in Silverlight. As an example, you can also view the policy file used by ArcGIS Online:http://services.arcgisonline.com/clientaccesspolicy.xml.


October 2011 49

ArcGIS Online clientaccesspolicy.xml content

2. ArcGISSOC account must be given permission to access SDE database. http://resources.arcgis.com/content/kbase?fa=articleShow&d=33688

HowTo: Setup ArcGISSOC account permissions for a SQL Server Enterprise SDE geodatabase

Article ID:

33688

Software: ArcGIS Server 9.1, 9.2, 9.3, 9.3.1, 10

Platforms: Windows XP, 2003Server

Summary When publishing data residing in a SQL Server ArcSDE geodatabase that uses operating system authentication, an ArcGISSOC account needs to be added to the SQL Server SDE geodatabase with at lease db_datareader permission. Procedure The permission for the ArcGISSOC account should be explicitly setup in SQL server Enterprise Manager. Instructions provided describe how to add an ArcGISSOC account in SQL Server Enterprise Manager. The following steps are written for SQL Server 2000, similiar steps apply to SQL Server 2005:


October 2011 50

1. Open SQL Server Enterprise manager. Go to the Security folder

2. Right-click on Logins > New Login

3. Use Windows Authentication and select the domain of the ArcGIS server SOC machine or

the server machine, if ArcGIS server was installed on one machine. Add the ArcGISSOC

account as a new login to the SQL server. Select Grant access.

4. Go to Database Access tab, check the database under 'Specify which databases can be

accessed by this login' and check db_datareader under 'Database Roles'.

5. Click Ok. Close the SQL Server Enterprise Manager.

Setting Up an Arc Hydro Based Geoprocessing

Documents

Transcript of Setting Up an Arc Hydro Based Geoprocessing