ArcPy and ArcGIS – Geospatial Analysis with Python · Table of Contents ArcPy and ArcGIS –...
Transcript of ArcPy and ArcGIS – Geospatial Analysis with Python · Table of Contents ArcPy and ArcGIS –...
TableofContents
ArcPyandArcGIS–GeospatialAnalysiswithPython
Credits
AbouttheAuthor
AbouttheReviewers
www.PacktPub.com
Supportfiles,eBooks,discountoffers,andmore
Whysubscribe?
FreeaccessforPacktaccountholders
Preface
Whatthisbookcovers
Whatyouneedforthisbook
Whothisbookisfor
Conventions
Readerfeedback
Customersupport
Downloadingtheexamplecode
Downloadingthecolorimagesofthisbook
Errata
Piracy
Questions
1.IntroductiontoPythonforArcGIS
OverviewofPython
Pythonasaprogramminglanguage
Interpretedlanguage
Standard(built-in)library
Thegluelanguage
Wrappermodules
ThebasicsofPython
Importstatements
Variables
Forloops
If/Elif/Elsestatements
Whilestatements
Comments
Datatypes
Strings
Integers
Floats
Lists
Tuples
Dictionaries
Iterabledatatypes
Otherimportantconcepts
Indentation
Functions
Keywords
Namespaces
Zero-basedindexing
ImportantPythonModulesforGISAnalysis
TheArcPymodule
TheOperatingSystem(OS)module
ThePythonSystem(SYS)module
TheXLRDandXLWTmodules
Commonlyusedbuilt-infunctions
Commonlyusedstandardlibrarymodules
Summary
2.ConfiguringthePythonEnvironment
WhatisaPythonscript?
HowPythonexecutesascript
WhatisthePythoninterpreter?
WhereisthePythoninterpreterlocated?
WhichPythoninterpretershouldbeused?
Howdoesthecomputerknowwheretheinterpreteris?
MakePythonscriptsexecutablewhenclickedon
IntegratedDevelopmentEnvironments(IDEs)
IDLE
PythonWin
AptanaStudio3
IDEsummary
Pythonfolderstructure
Wheremodulesreside
UsingPython’ssysmoduletoaddamodule
Thesys.path.append()method
Summary
3.CreatingtheFirstPythonScript
Prerequisites
ModelBuilder
CreatingamodelandexportingtoPython
ModelingtheSelectandBuffertools
AddingtheIntersecttool
Tallyingtheanalysisresults
Exportingthemodelandadjustingthescript
Theautomaticallygeneratedscript
FilepathsinPython
Continuingthescriptanalysis:theArcPytools
TheIntersecttoolandstringmanipulation
Thestringmanipulationmethod1–stringaddition
Thestringmanipulationmethod2–stringformatting#1
Thestringmanipulationmethod3–stringformatting#2
AdjustingtheScript
AddingtheCSVmoduletothescript
Accessingthedata:Usingacursor
Thefinalscript
Summary
4.ComplexArcPyScriptsandGeneralizingFunctions
Pythonfunctions–Avoidrepeatingcode
Technicaldefinitionoffunctions
Afirstfunction
Functionswithparameters
Usingfunctionstoreplacerepetitivecode
Moregeneralizationofthefunctions
Summary
5.ArcPyCursors–Search,Insert,andUpdate
Thedataaccessmodule
Attributefieldinteractions
Updatecursors
Updatingtheshapefield
Adjustingapointlocation
DeletingarowusinganUpdateCursor
UsinganInsertCursor
Insertingapolylinegeometry
Insertingapolygongeometry
Summary
6.WorkingwithArcPyGeometryObjects
ArcPygeometryobjectclasses
ArcPyPointobjects
ArcPyArrayobjects
ArcPyPolylineobjects
ArcPyPolygonobjects
Polygonobjectbuffers
OtherPolygonobjectmethods
ArcPygeometryobjects
ArcPyPointGeometryobjects
Summary
7.CreatingaScriptTool
Addingdynamicparameterstoascript
Displayingscriptmessagesusingarcpy.AddMessage
Addingdynamiccomponentstothescript
CreatingaScripttool
Labellinganddefiningparameters
Addingdatatypes
AddingtheBusStopfeatureclassasaparameter
AddingtheCensusBlockfeatureclassasaparameter
AddingtheCensusBlockfieldasaparameter
Addingtheoutputspreadsheetasaparameter
Addingthespreadsheetfieldnamesasaparameter
AddingtheSQLStatementasaparameter
Addingthebusstopfieldsasaparameter
Inspectingthefinalscript
RunningtheScriptTool
Summary
8.IntroductiontoArcPy.Mapping
UsingArcPywithmapdocuments
Inspectingandreplacinglayersources
Fixingthebrokenlinks
Fixingthelinksofindividuallayers
ExportingtoPDFfromanMXD
Adjustingmapdocumentelements
Automatedmapdocumentadjustment
Thevariables
Themapdocumentobjectandthetextelements
Thelayerobjects
Replacingthedatasources
Adjustinglayervisibility
Generatingabufferfromthebusstopsfeatureclass
Intersectingthebusstopbufferandcensusblocks
Populatingtheselectedbusstopandbufferfeatureclasses
Updatingthetextelements
ExportingtheadjustedmaptoPDF
RunningthescriptinthePythonWindow
Summary
9.MoreArcPy.MappingTechniques
Usingarcpy.mappingtocontrolLayerobjects
Layerobjectmethodsandproperties
Definitionqueries
Controllingthedataframewindowextentandscale
AddingaLayerobject
Exportingthemaps
Summary
10.AdvancedGeometryObjectMethods
CreatingaPythonmodule
The__init__.pyfile
Addingadvancedanalysiscomponents
AdvancedPolygonobjectmethods
Generatingrandompointstorepresentpopulation
Usingthefunctionswithinascript
CreatinganXLSusingXLWT
Summary
11.NetworkAnalystandSpatialAnalystwithArcPy
TheNetworkAnalystextension
UsingNetworkAnalyst
CreatingaFeatureDataset
Importingthedatasets
CreatingtheNetworkDataset
AccessingtheNetworkDatasetusingArcPy
Breakingdownthescript
TheNetworkAnalystmodule
AccessingtheSpatialAnalystExtension
Addingelevationtothebusstops
UsingMapAlgebratogenerateelevationinfeet
Addinginthebusstopsandgettingelevationvalues
Thefinalresult
Summary
12.TheEndoftheBeginning
Gettingfieldinformationfromfeatureclasses
AccessingtheListFields’properties
Listcomprehensions
Creatingthefieldinformationfunctions
Queryingfeatureclassinformation
GeneratingFileGeodatabasesandfeatureclasses
Generatingafeatureclass
Settingupthescripttoolparameters
Environmentalsettings
Resolutionandtolerancesettings
Summary
Index
ArcPyandArcGIS–GeospatialAnalysiswithPythonCopyright©2015PacktPublishing
Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.
Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthor,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.
PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.
Firstpublished:February2015
Productionreference:1210215
PublishedbyPacktPublishingLtd.
LiveryPlace
35LiveryStreet
BirminghamB32PB,UK.
ISBN978-1-78398-866-2
www.packtpub.com
CreditsAuthor
SilasToms
Reviewers
AlessioDiLorenzo
DaraO’Beirne
MarkPazolli
MarjorieRoswell
CommissioningEditor
AshwinNair
AcquisitionEditor
HarshaBharwani
ContentDevelopmentEditor
AkashdeepKundu
TechnicalEditor
DeeptiTuscano
CopyEditors
AartiSaldanha
AdithiShetty
ProjectCoordinator
MiltonDsouza
Proofreaders
SimranBhogal
JoannaMcMahon
BernadetteWatkins
Indexer
PriyaSane
ProductionCoordinator
AlwinRoy
CoverWork
AlwinRoy
AbouttheAuthorSilasTomsisageospatialprogrammerandanalystwithaloveofgeography,history,food,andsports.HeresidesintheSanFranciscoBayAreaandcan’tdecidewhichsideoftheBayismorebeautiful.Hereceivedabachelor’sdegreeinGeographyfromHumboldtStateUniversityandiscurrentlypursuingamaster’sdegreeinGISatSanFranciscoStateUniversity.WithabackgroundinGISanalysisforcitygovernmentsandenvironmentalconsulting,SilaslovesthecombinationofGISandPythonforanalysisautomationanddatamanipulation.
WorkingforAriniGeographics,SilasishelpinggovernmentsunderstandhowGIScanorganizeandsimplifythemanagementofinfrastructureandtheenvironment.ThisdualroleasaprogrammerandanalystallowshimtousePythonandGIStoquicklyproducegeospatialdataandtools.Combinedwithwebmapping,thesetoolsaretransforminghowgovernmentsworktoservethepublic.HealsoteachesworkshopsonArcPyandwebmappingattheCityCollegeofSanFrancisco,whilehopingtoonedayfinishhismaster’sthesis.
SilashasworkedasarevieweronthebookPythonGeospatialAnalysis,PacktPublishingandisworkingonthebookPythonGeospatialDevelopment,PacktPublishingtobepublishedin2015.
Iwouldliketothankmygirlfriend,Christine,forherencouragementandpatience.Iwouldliketothankmyboss,GabrielPaun,forhisinspirationandforpushingmetobecomeatrueGISprofessional.IwouldliketothankthefacultyatHSUandSFSUfortheirhelpalongtheway,andIwouldliketothankmyfamilyfortheirbeliefinmeandforneveraskingmeifIwasgoingtobecomeateacherwithmygeographydegree(eventhoughIhaveandIloveit!).
AbouttheReviewersAlessioDiLorenzoisamarinebiologistandhasanMScinGeographicalInformationSystems(GIS)andRemoteSensing.Since2006,hehasbeendealingwiththeanalysisanddevelopmentofGISapplicationsdedicatedtothestudyandspreadofenvironmentalandepidemiologicaldata.HeisexperiencedintheuseofthemainproprietaryandopensourceGISsoftwareandprogramminglanguages.
DaraO’BeirneisacertifiedGISProfessional(GISP)withovereightyearsofGISandPythonexperience.DaraearnedbothhisBachelorsandMastersofArtsdegreesingeographyfromSanFranciscoStateUniversity.DaraiscurrentlyaGISAnalystworkingatAriniGeographicsinSantaClara,CA.BeforejoiningAriniGeographics,DarawasaGISAnalystandtechnicalleadatTowillInc.,aGISandLandSurveyingcompanyinNorthernCalifornia.AtTowill,DaraplayedacentralroleindevelopingandimplementingproceduresrelatedtothecollectionandanalysisofLiDARdataforenvironmentalandengineeringapplications.PriortoTowill,DaragainedhisprofessionalGISexperienceworkingfortheGoldenGateNationalRecreationAreamanagedbytheNationalParkService,oneofthelargesturbanparksystemsintheworld,whichincludesNationaltreasures,suchasAlcatraz,MuirWoods,andtheMarinHeadlands.HisMaster’sThesisexaminedtheerrorsassociatedwithmeasuringtreeheightsinanurbanenvironmentwithbothtraditionalfieldmethodsandairborneLiDARdata.
Iwouldliketothankmywife,Kate,anddaughter,AnyaO’Beirne,fortheirpatienceandassistanceduringthereviewofthisbook.
MarjorieRoswellisawebdeveloperandmapmakerfromBaltimore,MD.ShepurchasedherfirstGISin1991,andbuiltanapplicationtoassistcitizencallerstotheBaltimoreOfficeofRecycling.Recentprojectsincludeinteractivemapsoflegislativescores,politicalendorsements,committees,electiondata,andadvocacyinterests.
Hersitehttp://committeemaps.org/detailsCongressionalcommitteemembership,whilethesitehttp://farmbillprimer.org/isdevotedtomappingandchartingfederalfoodandfarmpolicy.
MarjorieistheauthorofDrupal5ViewsRecipes,PacktPublishing.ShewasthetechnicalreviewerofjQueryUI1.10,TheUserInterfaceLibraryforjQuery,PacktPublishing.
MarkPazolliisanengineeranddatascientistwhousesArcGISandPythontohelphisemployersdecipherthemountainsofdatatheykeepontheassetsoftheWesternAustralianelectricalnetwork.HehasqualificationsinElectricalEngineering,ComputerScience,andAppliedMathematics.Heappreciatesexcellentdesignandenjoysbuildinginterestingthings.
Supportfiles,eBooks,discountoffers,andmoreForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.
DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusat<[email protected]>formoredetails.
Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.
https://www2.packtpub.com/books/subscription/packtlib
DoyouneedinstantsolutionstoyourITquestions?PacktLibisPackt’sonlinedigitalbooklibrary.Here,youcansearch,access,andreadPackt’sentirelibraryofbooks.
Whysubscribe?FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser
FreeaccessforPacktaccountholdersIfyouhaveanaccountwithPacktatwww.PacktPub.com,youcanusethistoaccessPacktLibtodayandview9entirelyfreebooks.Simplyuseyourlogincredentialsforimmediateaccess.
PrefaceArcGIS,theGISsoftwarefromindustryleaderESRI,allowsfortheanalysisandpresentationofgeospatialdata.
TheintegrationofPythonintoArcGIShasmadetheArcPymoduleanimportanttoolforGISstudentsandprofessionals.TheArcPymoduleprovidesapowerfulwaytoimproveproductivitywhenperforminggeospatialanalysis.FrombasicPythonscriptingthroughadvancedArcPymethodsandproperties,ArcPyandotherPythonmoduleswillimprovethespeedandrepeatabilityofanyGISworkflow.
ThisbookwillguideyoufrombasicPythonscriptingtoadvancedscriptingtools.Itfocusesongeospatialanalysisscriptingandtouchesonautomatingcartographicoutput.Bytheendofthisbook,youwillbeabletocreatereusablemodules,addrepeatableanalysesasscripttoolsinArcToolbox,andexportmapsautomatically.Byreducingthetime-consumingnatureofGISfromdaystohours,oneGISprofessionalcanbecomeaspowerfulasawholeteam.
WhatthisbookcoversChapter1,IntroductiontoPythonforArcGIS,offersaquickintroductiontothebasicsofPython,includingotherusesfortheprogramminglanguage.ItcoversPythondatatypesandimportantmodulesusedthroughoutthebook.
Chapter2,ConfiguringthePythonEnvironment,isanintroductiontohowPythonworks:itsfolderstructure,executables,andmodules.Italsoexplainsimportingmodulesintoscripts,thebuilt-inmodules,andcoversIntegratedDevelopmentEnvironments(IDEs),whicharepowerfulprogrammingaids.
Chapter3,CreatingtheFirstPythonScript,demonstrateshowtouseArcGISModelBuildertomodelthefirstanalysisandthenexportitasaPythonscript.StringmanipulationsandhowtousefilepathsinPythonarealsointroduced.
Chapter4,ComplexArcPyScriptsandGeneralizingFunctions,examineshowtoperformanalysesandproduceoutputsthatarenotpossibleusingModelBuilder.Byusingfunctions,orreusablecodeblocks,repeatingcodeisavoided.
Chapter5,ArcPyCursors–Search,Insert,andUpdate,coversArcPydataaccesscursorsandhowtheyareusedtosearch,update,orinsertrecordsinfeatureclassesandtables.Itexplainsthequirksofiteratingusingcursors,andhowtoonlyselectorupdatetherecordsofinterest.
Chapter6,WorkingwithArcPyGeometryObjects,exploresArcPyGeometryobjectsandhowtheyarecombinedwithcursorstoperformspatialanalysis.Itdemonstrateshowtobuffer,clip,reproject,andmoreusingthedatacursorsandtheArcpygeometrytypeswithoutusingArcToolbox.
Chapter7,CreatingaScriptTool,explainshowtomakescriptsintotoolsthatappearinArcToolboxandaredynamicinnature.ItexplainshowthetoolsandscriptscommunicateandhowtosetuptheArcTooldialogtocorrectlypassparameterstothescript.
Chapter8,IntroductiontoArcPy.Mapping,exploresthepowerfulArcpy.Mappingmoduleandhowtofixbrokenlayerlinks,turnlayersonandoff,anddynamicallyadjusttitlesandtext.Itshowshowtocreatedynamicmapoutputbasedonageospatialanalysis.
Chapter9,MoreArcPy.MappingTechniques,introducesLayerobjects,andtheirmethodsandproperties.Itdemonstrateshowtocontrolmapscalesandextentsfordataframes,andcoversautomatedmapexport.
Chapter10,AdvancedGeometryObjectMethods,expandsontheArcPyGeometryobjectmethodsandproperties.Italsoexplainshowtocreateamoduletosavecodeforreuseinsubsequentscripts,anddemonstrateshowtocreateExcelspreadsheetscontainingresultsfromageospatialanalysis.
Chapter11,NetworkAnalystandSpatialAnalystwithArcPy,introducesthebasicsofusingArcPyforadvancedgeospatialanalysisusingtheArcGISforDesktopNetworkAnalystandSpatialAnalystExtensions.
Chapter12,TheEndoftheBeginning,coversotherimportanttopicsthatneedtobeunderstoodtohaveafullgraspofArcPy.ThesetopicsincludetheEnvironmentSettings,XYvaluesandZandMresolutions,SpatialReferenceSystems(Projections),theDescribefunctions,andmore.
WhatyouneedforthisbookYouwillneedtheproprietaryorfreeversionofArcGIS10.1/10.2/10.3.Tosupportyourenvironment,youwillneed2GBRAM,32-bitor64bitmachinehardwareconfiguration,andWindows7/8.Python2.7isrequiredtodotheprogrammingandisinstalledalongwithArcGIS.
WhothisbookisforThisbookisintendedforGISstudentsandprofessionalswhoneedanunderstandingofhowtouseArcPytoreducerepetitivetasksandperformanalysisfaster.ItisalsoavaluablebookforPythonprogrammerswhowouldliketounderstandhowtoautomategeospatialanalysisusingtheindustrystandardArcGISsoftwareanditsArcPymodule.
ConventionsInthisbook,youwillfindanumberofstylesoftextthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestyles,andanexplanationoftheirmeaning.
Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummyURLs,userinput,andTwitterhandlesareshownasfollows:“Thetwodatapieces,theBusStopIDandtheaveratePopvariablearethenaddedtoalist.”
Ablockofcodeissetasfollows:
witharcpy.da.SearchCursor(Intersect71Census,["STOPID","POP10"])as
cursor:
forrowincursor:
busStopID=row[0]
pop10=row[1]
ifbusStopIDnotindataDictionary.keys():
dataDictionary[busStopID]=[pop10]
else:
dataDictionary[busStopID].append(pop10)
Anycommand-lineinputoroutputiswrittenasfollows:
>>>aString="Thisisastring"
>>>bString="andthisisanotherstring"
>>>aString+bString
Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,inmenusordialogboxesforexample,appearinthetextlikethis:“Selectitbyclickingonit,andthenclickingontheEditbutton.”
NoteWarningsorimportantnotesappearinaboxlikethis.
TipTipsandtricksappearlikethis.
ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook—whatyoulikedormayhavedisliked.Readerfeedbackisimportantforustodeveloptitlesthatyoureallygetthemostoutof.
Tosendusgeneralfeedback,simplysendane-mailto<[email protected]>,andmentionthebooktitleviathesubjectofyourmessage.
Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideonwww.packtpub.com/authors.
CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.
DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesforallPacktbooksyouhavepurchasedfromyouraccountathttp://www.packtpub.com.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.
DownloadingthecolorimagesofthisbookWealsoprovideyouwithaPDFfilethathascolorimagesofthescreenshots/diagramsusedinthisbook.Thecolorimageswillhelpyoubetterunderstandthechangesintheoutput.Youcandownloadthisfilefromhttp://www.packtpub.com/sites/default/files/downloads/8662OS_ColorImages.pdf.
ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks—maybeamistakeinthetextorthecode—wewouldbegratefulifyoucouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheErrataSubmissionFormlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedtoourwebsiteoraddedtoanylistofexistingerrataundertheErratasectionofthattitle.
Toviewthepreviouslysubmittederrata,gotohttps://www.packtpub.com/books/content/supportandenterthenameofthebookinthesearchfield.TherequiredinformationwillappearundertheErratasection.
PiracyPiracyofcopyrightmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.Ifyoucomeacrossanyillegalcopiesofourworks,inanyform,ontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.
Pleasecontactusat<[email protected]>withalinktothesuspectedpiratedmaterial.
Weappreciateyourhelpinprotectingourauthors,andourabilitytobringyouvaluablecontent.
QuestionsYoucancontactusat<[email protected]>ifyouarehavingaproblemwithanyaspectofthebook,andwewilldoourbesttoaddressit.
Chapter1.IntroductiontoPythonforArcGISInthischapter,wewilldiscussthedevelopmentofPythonasaprogramminglanguage,fromitsbeginninginthelate1980stoitscurrentstate.Wewilldiscussthephilosophyofdesignthatspurreditsdevelopment,andtouchonimportantmodulesthatwillbeusedthroughoutthebook,especiallyfocusingonthemodulesbuiltintothePythonstandardlibrary.ThisoverviewofthelanguageanditsfeatureswillhelpexplainwhatmakesPythonagreatlanguageforArcGISautomation.
Thischapterwillcover:
AquickoverviewofPython:Whatitisanddoes,whocreatedit,andwhereitisnowTheArcPymoduleandotherimportantmodulesPythonasageneralpurposeprogramminglanguage
OverviewofPythonPython,createdbyGuidovanRossumin1989,wasnamedafterhisfavoritecomedytroupe,MontyPython.HisworkgroupatthetimehadatraditionofnamingprogramsafterTVshows,andhewantedsomethingirreverentanddifferentfromitspredecessors-ABC,Pascal,Ada,Eiffel,FORTRAN,andothers.SohesettledonPython,feelingitwasabitedgyandcatchyaswell.It’scertainlymorefuntosaythanC,thelanguageonwhichPythonisbased.
Today,Pythonisamajorprogramminglanguage.Itisusedinwebdevelopment,databaseadministration,andeventoprogramrobots.MostimportantlytoGISAnalysts,PythoncanbeusedtocontrolArcGIStoolsandMapDocumentstoproducegeospatialdataandmapsinanorganizedandspeedymannerusingtheexcellentArcPymodule.
ArcPyisinstalledwithArcGISfordesktopandArcGISforserver.ArcPyhasbeentheofficialArcGISscriptinglanguagesinceArcGIS10.0andhassteadilyimprovedinfunctionalityandimplementation.ThisbookwilltargetArcGISforDesktop10.1andlater,andwilldemonstratehowtomakeuseofPythonanditspowerfulprogramminglibraries(ormodules)whencraftingcomplexgeospatialanalyses.
PythonasaprogramminglanguageOverthepast40years,programminglanguageshavedevelopedfromassemblyandmachinecodetowardshigh-levelabstractedlanguagesthataremuchclosertoEnglish.ThePythonprogramminglanguagewasdesignedtoovercomemanyissuesthatprogrammerswerecomplainingaboutinthe1980s:slowdevelopmenttime,overlycomplicatedsyntax,andhorriblereadability.VanRossumwantedtodevelopalanguagethatcouldenablerapidcodedevelopmentandtesting,havesimpleoratleastreadable)syntax,andproduceresultswithfewerlinesofcode,inlesstime.ThefirstversionofPython(0.9.0)wasreleasedin1991andwasfreelyobtainablefromthestart;Pythonwasopensourcebeforethetermopensourcewasinvented.
InterpretedlanguagePythonisaninterpretedlanguage.ItiswritteninC,acompiledlanguage,andthecodeisinterpretedfromPythonintoCbeforeitisexecuted.Practically,thismeansthatthecodeisexecutedassoonasitisconvertedandcompiled.WhilecodeinterpretationcanhavespeedimplicationsfortheexecutionofPython-basedprograms,thefasterdevelopmenttimeallowedbyPythonmakesthisdrawbackeasytoignore.Testingofcodesnippetsismuchfasterinaninterpretiveenvironment,anditisperfecttocreatescriptstoautomatebasic,repeatablecomputingtasks.Pythonscriptshavethe.pyextentions.Oncethecodehasbeeninterpreted,asecondPythonscript(withthe.pycextentions)isgeneratedtosavethecompiledcode.The.pycscriptwillbeautomaticallyrecompiledwhenchangesaremadeintheoriginal.pyscript.
Standard(built-in)libraryPython,wheninstalled,hasabasicsetoffunctionalitythatisreferredtoasthestandardlibrary.ThesetoolsallowPythontoperformstringmanipulations,mathcomputations,andHTTPcallsandURLparsing,alongwithmanyotherfunctions.Someofthetoollibraries,knowntoPythonprogrammersasmodules,arebuilt-inandavailableassoonasPythonisstarted,whileothersmustbeexplicitlycalledusingtheimportkeywordtomaketheirfunctionsandclassesavailable.OthermoduleshavebeendevelopedbythirdpartiesandcanbedownloadedandinstalledontothePythoninstallationasneeded.
ManynewprogrammerswonderifPythonisarealprogramminglanguage,whichisaloadedquestion.Theanswerisyes;Pythoncanbeusedtocreatecompleteprograms,buildwebsites,runcomputernetworks,andmuchmore.Thebuilt-inmodulesandadd-onmodulesmakePythonverypowerful,anditcanbe(andhasbeen)usedfornearlyanypartofacomputer—operatingsystems,databases,webservers,desktopapplications,andsoon.Itisnotalwaysthebestchoiceforthedevelopmentofthesetools,butthathasnotstoppedprogrammersfromtryingandevensucceeding.
ThegluelanguagePythonisatitsbestwhenitisusedasagluelanguage.ThistermdescribestheuseofPythontocontrolotherprograms,sendinginputstothemandcollectingoutputs,whicharethensenttoanotherprogramorwrittentodisk.AnArcGISexamplewouldbetousePythontodownloadzippedshapefilesfromawebsite,unzippingthefiles,processingthefilesusingArcToolbox,andcompilingtheresultsintoanExcelspreadsheet.AllofthisisaccomplishedusingfreelyavailablemodulesthatareeitherincludedinPython’sstandardlibrary,oraddedwhenArcGISisinstalled.
WrappermodulesTheArcPymoduleisawrappermodule.WrappermodulesarecommoninPython,andaresonamedbecausetheywrapPythonontothetoolswewillneed.TheyallowustousePythontointerfacewithotherprogramswritteninCorotherprogramminglanguages,usingtheApplicationProgrammingInterface(API)ofthoseprograms.Forexample,wrappersmakeitpossibletoextractdatafromanExcelspreadsheetandtransformorloadthedataintoanotherprogram,suchasArcGIS.Notallmodulesarewrappers;somemodulesarewritteninpurePythonandperformtheiranalysisandcomputationsusingthePythonsyntax.Eitherway,theendresultisthatacomputeranditsprogramsareavailabletobemanipulatedandcontrolledusingPython.
TheZenofPythonwascreatedtobestraightforward,readable,andsimplified,comparedtootherlanguagesthatexistedpreviously.ThisgoverningphilosophywasorganizedintoapoembyTimPeters,anearlyPythondevelopercalledtheZenofPython;itisanEasteregg(ahiddenfeature)includedineveryPythoninstallationandisshownwhenimportthisistypedinthePythoninterpreter:
TheZenofPython,byTimPeters:
Beautifulisbetterthanugly.
Explicitisbetterthanimplicit.
Simpleisbetterthancomplex.
Complexisbetterthancomplicated.
Flatisbetterthannested.
Sparseisbetterthandense.
Readabilitycounts.
Specialcasesaren'tspecialenoughtobreaktherules.
Althoughpracticalitybeatspurity.
Errorsshouldneverpasssilently.
Unlessexplicitlysilenced.
Inthefaceofambiguity,refusethetemptationtoguess.
Thereshouldbeone--andpreferablyonlyone--obviouswaytodo
Althoughthatwaymaynotbeobviousatfirstunlessyou'reDutch.
Nowisbetterthannever.
Althoughneverisoftenbetterthan*right*now.
Iftheimplementationishardtoexplain,it'sabadidea.
Iftheimplementationiseasytoexplain,itmaybeagoodidea.
Namespacesareonehonkinggreatidea—let'sdomoreofthose!
NoteGotohttps://www.python.org/doc/humor/formoreinformation.
ThebasicsofPythonPythonhasanumberoflanguagerequirementsandconventionsthatallowforthecontrolofmodulesandstructuringofcode.Thefollowingareanumberofimportantbasicconcepts,whichwillbeusedthroughoutthisbookandwhencraftingscriptsforusewithgeospatialanalyses.
ImportstatementsImportstatementsareusedtoaugmentthepowerofPythonbycallingothermodulesforuseinthescript.ThesemodulescanbepartofthestandardPythonlibraryofmodules,suchasthemathmodule(usedtodohighermathematicalcalculations)or,importantly,ArcPy,whichwillallowustointeractwithArcGIS.
NoteImportstatementscanbelocatedanywherebeforethemoduleisused,butbyconvention,theyarelocatedatthetopofascript.
Therearethreewaystocreateanimportstatement.Thefirst,andmoststandard,istoimportthewholemoduleasfollows:
importarcpy
Usingthismethod,wecanevenimportmorethanonemoduleonthesameline.Thefollowingimportsthreemodules:arcpy,os(theoperatingsystemmodule),andsys(thePythonsystemmodule):
importarcpy,os,sys
Thenextmethodofimportingascriptistoimportaspecificportionofamodule,insteadofimportingtheentiremodule,usingthefrom<module>import<submodule>syntax:
fromarcpyimportmapping
ThismethodisusedwhenonlyaportionofthecodefromArcPywillbeneeded;ithasthepracticaleffectoflimitingtheamountofmemoryusedbythemodulewhenitiscalled.Wecanalsoimportmultipleportionsofthemoduleinthesamefashion:
fromarcpyimportmapping,da
Thethirdwaytoimportamoduleisthefrom<module>import<submodule>syntax,butbyusinganasterisktoimportallpartsofthemodule:
fromarcpyimport*
Thislastmethodisstillusedbutitisdiscouragedasitcanhaveunforeseenconsequences.Forinstance,thenamesofthevariablesinthemodulemightconflictwithanothervariableinanothermoduleiftheyarenotexplicitlyimported.Forthisreason,itisbesttoavoidthisthirdmethod.However,lotsofexistingscriptsincludeimportstatementsofthistypesobeawareoftheseconsequences.
VariablesVariablesareapartofallprogramminglanguages.Theyareusedtoreferencedataandstoreitinmemoryforuselaterinascript.Therearealotofargumentsoverthebestmethodtonamevariables.NostandardhasbeendevelopedforPythonscriptingforArcGIS.Thefollowingaresomebestpracticestousewhennamingvariables.
Makethemdescriptive:Don’tjustnameavariablex;thatvariablewillbeuselesslaterwhenthescriptisreviewedandthereisnowaytoknowwhatitisusedfor,orwhy.Theyshouldbelongerratherthanshorter,andshouldhintatthedatatheyreferenceoreventhedatatypeoftheobjecttheyreference:
shapefilePath='C:/Data/shapefile.shp'
Usecamelcasetomakethevariablereadable:Camelcaseisatermusedforvariablesthatstartwithalowercaseletterbuthaveuppercaselettersinthemiddle,resemblingacamel’shump:
camelCase='thisisastring'
Includethedatatypeinthevariablename:Ifthevariablecontainsastring,callitvariableString.Thisisnotrequired,andwillnotbeuseddogmaticallyinthisbook,butitcanhelporganizethescriptandishelpfulforotherswhowillreadthesescripts.Pythonisdynamicallytypedinsteadofstatically.Aprogramminglanguagedistinctionmeansthatavariabledoesnothavetobedeclaredbeforeitcanbeused,unlikeVisualBasicorotherstaticallytypedlanguages.Thisimprovesthespeedofwritingascript,butitcanbeproblematicinlongscriptsasthedatatypeofavariablewillnotbeobvious.
NoteTheArcGISdoesnotusecamelcasewhenitexportsPythonscripts,andmanyexampleswillnotincludeit;nevertheless,itisrecommendedwhenwritingnewscripts.Also,variablescannotstartwithanumber.
ForloopsBuiltintoprogramminglanguagesistheabilitytoiterate,orperformarepeatingprocess,overadatasettotransformorextractdatathatmeetsspecificcriteria.Python’smainiterationtoolisknownasaforloop.Thetermforloopmeansthatanoperationwillloop,oriterate,overtheitemsinadatasettoperformtheoperationoneachitem.Thedatasetmustbeiterabletobeusedinaforloop,adistinctiondiscussedfurtherahead.
Wewillbeusingforloopsthroughoutthisbook.HereisasimpleexamplethatusesthePythonInterpretertotakestringvaluesandprinttheminanuppercaseformat,usingaforloop:
>>>newlist=['a','b','c','d']
>>>foriteminnewlist:
printitem.upper()
Theoutputisshownasfollows:
A
B
C
D
Thevariableitemisagenericvariableassignedtoeachobjectasitisenteredintotheforloop,andnotatermrequiredbyPython.Itcouldhavebeenxorvalueinstead.Withintheloop,thefirstobject(a)isassignedtothegenericvariableitemandhastheupperstringfunctionappliedtoittoproducetheoutputA.Oncethisactionhasbeenperformed,thenextobject(b)isassignedtothegenericvariabletoproduceanoutput.Thisloopisrepeatedforallmembersofthedatasetnewlist;oncecompleted,thevariableitemwillstillcarrythevalueofthelastmemberofthedataset(dinthiscase).
TipDownloadingtheexamplecode
YoucandownloadtheexamplecodefilesforallPacktbooksyouhavepurchasedfromyouraccountathttp://www.packtpub.com.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.
If/Elif/ElsestatementsConditionalstatements,calledif/elsestatementsinPython,arealsostandardinprogramminglanguages.Theyareusedwhenevaluatingdata;whencertainconditionsaremet,oneactionwillbetaken(theinitialifstatement;ifanotherconditionismet,anotheractionistaken;thisisanelifstatement),andifthedatadoesnotmeetthecondition,afinalactionisassignedtodealwiththosecases(theelsestatement).ThesearesimilartoawhereconditionalinaSQLstatementusedwiththeSelecttoolinArcToolbox.Hereisanexampleofhowtouseanif/elsestatementtoevaluatedatainalist(adatatypediscussedfurtherahead)andfindtheremainderwhendividedusingthemodulusoperator(%)andPython’sisequaltooperator(==):
>>>data=[1,2,4,5,6,7,10]
>>>forvalindata:
ifval%2==0:
printval,"noremainder"
elifval%3==2:
printval,"remainderoftwo"
else:
print"finalcase"
Theoutputisshownasfollows:
finalcase
2noremainder
4noremainder
5remainderoftwo
6noremainder
finalcase
10noremainder
WhilestatementsAnotherimportantevaluationtoolisthewhilestatement.Itisusedtoperformanactionwhileaconditionistrue;whentheconditionisfalse,theevaluationwillstop.Notethattheconditionmustbecomefalse,ortheactionwillbealwaysperformed,creatinganinfiniteloopthatwillnotstopuntilthePythoninterpreterisshutoffexternally.Hereisanexampleofusingawhilelooptoperformanactionuntilatrueconditionbecomesfalse:
>>>x=0
>>>whilex<5:
printx
x+=1
Theoutputisshownasfollows:
0
1
2
3
4
CommentsCommentsinPythonareusedtoaddnoteswithinascript.Theyaremarkedbyapoundsign,andareignoredbythePythoninterpreterwhenthescriptisrun.Commentsareusefultoexplainwhatacodeblockdoeswhenitisexecuted,ortoaddhelpfulnotesthatscriptauthorswouldlikefuturescriptuserstoread:
#Thisisacomment
Whileitisaprogrammingtruismthatgoodcodeiswell-commentedcode,manyprogrammersskipthisvaluablestep.Also,toomanycommentscanreducetheirusefulnessandthescript’sreadability.Ifvariablesaredescriptiveenough,andcodeiswell-organized,commentsarelessnecessary;writingthecodeasverboseandaswell-organizedaspossiblewillrequirelesstimetobespentoncomments.
DatatypesGISusespoints,lines,polygons,coverages,andrasterstostoredata.EachoftheseGISdatatypescanbeusedindifferentwayswhenperformingananalysisandhavedifferentattributesandtraits.Python,similartoGIS,hasdatatypesthatorganizedata.ThemaindatatypesinPythonarestrings,integers,floats,lists,tuples,anddictionaries.Theyeachhavetheirownattributesandtraits(orproperties),andareusedforspecificpartsofcodeautomation.Therearealsobuilt-infunctionsthatallowfordatatypestobeconverted(orcasted)fromonetypetoanother;forinstance,theinteger1canbeconvertedtothestring1usingthestr()function:
>>>variable=1
>>>newvar=str(variable)
>>>newvar
Theoutputisshownasfollows:
1
StringsStringsareusedtocontainanykindofcharacter.Theybeginandendwithquotationmarks,witheithersingleordoublequotesused,thoughthestringmustbeginandendwiththesametypeofquotationmarks.Withinastring,quotedtextcanappear;itmustusetheoppositequotationmarkstoavoidconflictingwiththestring.Checkthefollowingexample:
>>>quote='Thisstringcontainsaquote:"Hereisthequote"'
Athirdtypeofstringisalsoemployed,amultiplelinestringthatstartsandendswiththreesinglequotemarks:
>>>multiString='''Thisstringhas
multiplelinesandcangofor
aslongasIwantittoo'''
IntegersIntegersarewholenumbersthatdonothaveanydecimalplaces.Thereisaspecialconsequencetotheuseofintegersinmathematicaloperations;ifintegersareusedfordivision,anintegerresultwillbereturned.Checkoutthiscodesnippetbelowtoseeanexampleofthis:
>>>5/2
Theoutputisshownasfollows:
2
Insteadofanaccurateresultof2.5,Pythonwillreturnthefloorvalue,orthelowestwholeintegerforanyintegerdivisioncalculation.Thiscanobviouslybeproblematicandcancausesmallbugsinscriptsthatcanhavemajorconsequences.
TipPleasebeawareofthisissuewhenwritingscriptsandusefloatstoavoiditasdescribedinthefollowingsection.
FloatsFloatingpointvalues,orfloats,areusedbyPythontorepresentdecimalvalues.Theuseoffloatswhenperformingdivisionisrecommended:
>>>5.0/2
Theoutputisshownasfollows:
2.5
Becausecomputersstorevaluesinabase2binarysystem,therecanbeissuesrepresentingafloatingvaluethatwouldnormallyberepresentedinabase10system.Readdocs.python.org/2/tutorial/floatingpoint.htmlforafurtherdiscussionoftheramificationsofthislimitation.
ListsListsareorderedsetsofdatathatarecontainedinsquarebrackets([]).Listscancontainanyothertypeofdata,includingotherlists.Datatypescanbemixedwithinasinglelist.Listsalsohaveasetofmethodsthatallowthemtobeextended,reversed,sorted,summed,orextractthemaximumorminimumvalue,alongwithmanyothermethods.Datapieceswithinalistareseparatedbycommas.
Listmembersarereferencedbytheirindex,orpositioninthelist,andtheindexalwaysstartsatzero.Lookatthefollowingexampletounderstandthisbetter:
>>>alist=['a','b','c','d']
>>>alist[0]
Theoutputisshownasfollows:
'a'
Thisexampleshowsushowtoextractthefirstvalue(attheindex0)fromthelistcalledalist.Oncealisthasbeenpopulated,thedatawithinitisreferencedbyitsindex,whichispassedtothelistinsquarebrackets.Togetthesecondvalueinalist(thevalueatindex1),thesamemethodisused:
>>>alist[1]
Theoutputisshownasfollows:
'b'
Tomergetwolists,theextendmethodisused:
>>>blist=[2,5,6]
>>>alist.extend(blist)
>>>alist
Theoutputisshownasfollows:
['a','b','c','d',2,5,6]
TuplesTuplesarerelatedtolistsandaredenotedbyparentheses(()).Unlikelists,tuplesareimmutable—theycannotbeadjustedorextendedoncetheyhavebeencreated.Datawithinatupleisreferencedinthesamewayasalist,usingindexreferencesstartingatzero:
>>>atuple=('e','d','k')
>>>atuple[0]
Theoutputisshownasfollows:
'e'
DictionariesDictionariesaredenotedbycurlybrackets({})andareusedtocreatekey:valuepairs.Thisallowsustomapvaluesfromakeytoavalue,sothatthevaluecanreplacethekeyanddatafromthevaluecanbeusedinprocessing.Hereisasimpleexample:
>>>adic={'key':'value'}
>>>adic['key']
Theoutputisshownasfollows:
'value'
Notethatinsteadofreferringtoanindexposition,suchaslistsortuples,thevaluesarereferencedusingakey.Also,keyscanbeanyothertypeofdataexceptlists(becauselistsaremutable).
Thiscanbeveryvaluablewhenreadingashapefileorfeatureclass.UsinganObjectIDasakey,thevaluewouldbealistofrowattributesassociatedwithObjectID.Lookatthefollowingexampletobetterunderstandthisbehavior:
>>>objectIDdic={1:['100','Main','St']}
>>>objectIDdic[1]
Theoutputisshownasfollows:
['100','Main','St']
Dictionariesareveryvaluableforreadinginfeatureclassesandeasilyparsingthroughthedatabycallingonlytherowsofinterest,amongotheroperations.Theyaregreatfororderingandreorderingdataforuselaterinascript,sobesuretopayattentiontothemmovingforward.
IterabledatatypesLists,tuples,andstringsarealliterabledatatypesthatcanbeusedinforloops.Whenenteredintoaforloop,thesedatatypesareoperatedoninorder,unlessotherwisespecified.Forlistsandtuples,thisiseasytounderstand,astheyhaveanobviousorder:
>>>aList=[1,3,5,7]
>>>forvalueinaList:
printvalue*2
Theoutputisshownasfollows:
2
6
10
14
Forstrings,eachcharacterislooped:
>>>aString="esri"
>>>forvalueinaString:
printvalue.upper()
Theoutputisshownasfollows:
E
S
R
I
Dictionariesarealsoiterable,butwithaspecificimplementationthatwillonlyallowdirectaccesstothekeysofthedictionary(whichcanthenbeusedtoaccessthevalues).Also,thekeysarenotreturnedinaspecificorder:
>>>aDict={"key1":"value1",
"key2":"value2"}
>>>forvalueinaDict:
printvalue,aDict[value]
Theoutputisshownasfollows:
key2value2
key1value1
OtherimportantconceptsTheuseofPythonforprogrammingrequiresanintroductiontoanumberofconceptsthatareeitheruniquetoPythonbutrequiredorcommonprogrammingconceptsthatwillbeinvokedrepeatedlywhencreatingscripts.IncludedfollowingareanumberoftheseconceptsthatmustbecoveredtobefluentinPython.
IndentationPython,unlikemostotherprogramminglanguages,enforcesstrictrulesonindentinglinesofcode.ThisconceptisderivedagainfromGuido’sdesiretoproduceclean,readablecode.Whencreatingfunctionsorusingforloops,orif/elsestatements,indentationisrequiredonthesucceedinglinesofcode.Ifaforloopisincludedinsideanif/elsestatement,therewillbetwolevelsofindentation.VeteranprogrammersofotherlanguageshavecomplainedaboutthestrictnatureofPython’sindentation.Newprogrammersgenerallyfindittobehelpfulasitmakesiteasytoorganizecode.NotethatalotofprogrammersnewtoPythonwillcreateanindentationerroratsomepoint,somakesuretopayattentiontotheindentationlevels.
FunctionsFunctionsareusedtotakecodethatisrepeatedoverandoverwithinascript,oracrossscripts,andmakeformaltoolsoutofthem.Usingthekeyworddef,shortforthedefinefunction,functionsarecreatedwithdefinedinputsandoutputs.Theideaofafunctionincomputingisthatittakesdatainonestateandconvertsitintodatainanotherstate,withoutaffectinganyotherpartofthescript.ThiscanbeveryvaluabletoautomateaGISanalysis.
Hereisanexampleofafunctionthatreturnsthesquareofanynumbersupplied:
defsquare(inVal):
returninVal**2
>>>square(3)
Theoutputisshownasfollows:
9
Whilethisofcourseduplicatesasimilarfunctionbuiltintothemathmodule,itshowsthebasicsofafunction.Afunction(generally)acceptsdata,transformsitasneeded,andthenreturnsthenewstateofthedatausingthereturnkeyword.
KeywordsThereareanumberofkeywordsbuiltintoPythonthatshouldbeavoidedwhennamingvariables.Theseincludemax,min,sum,return,list,tuple,def,del,from,not,in,as,if,else,elif,or,while,and,with,amongmanyothers.Usingthesekeywordswillresultinanerror.
NamespacesNamespacesarealogicalwaytoorganizevariablenameswhenavariableinsidea
function(alocalvariable)sharesthesamenameasavariableoutsideofthefunction(aglobalvariable).Localvariablescontainedwithinafunction(eitherinthescriptorwithinanimportedmodule)andglobalvariablescanshareanameaslongastheydonotshareanamespace.
Thisissueoftenariseswhenavariablewithinanimportedmoduleunexpectedlyhasthesamenameofavariableinthescript.PythonInterpreterwillusenamespacerulestodecidewhichvariablehasbeencalled,whichcanleadtoundesirableresults.
Zero-basedindexingAsmentionedintheprecedingsectionthatdescribeslistsandtuples,Pythonindexingandcountingstartsatzero,insteadofone.Thismeansthatthefirstmemberofagroupofdataisatthezeroposition,andthesecondmemberisatthefirstposition,andsoontillthelastposition.
Thisrulealsoapplieswhenthereisaforloopiterationwithinascript.Whentheiterationstarts,thefirstmemberofthedatabeingiteratedisinthezeroposition.
Also,indexingcanbeperformedwhencountingfromthelastmemberofaniterableobject.Inthiscase,theindexofthelastmemberis-1,andthesecondtolastis-2,andsoonbacktothefirstmemberoftheobject.
ImportantPythonModulesforGISAnalysisModules,orcodelibrariesthatcanbecalledbyascripttoincreaseitsprogrammingpotential,areeitherbuiltintoPythonorarecreatedbythirdpartiesandaddedlatertoPython.MostofthesearewritteninPython,butanumberofthemarealsowritteninotherprogramminglanguagesandthenwrappedinPythontomakethemavailablewithinPythonscripts.ModulesarealsousedtomakeotherprogramsavailabletoPython,suchasthetoolsbuiltinMicrosoftWord.
TheArcPymoduleTheArcPymoduleisbothawrappermoduleusedtointeractwiththeArcGIStools,whicharethenexecutedbyArcGISinitsinternalcodeformat,andacodebasethatallowsforadditionalcontrolofgeospatialanalysesandmapproduction.ArcPyisusedtocontrolthetoolsinArcToolbox,butthetoolshavenotbeenrewritteninPython;instead,weareabletousetheArcGIStoolsusingArcPy.ArcPyalsogivesustheabilitytocontrolArcGISMapDocuments(MXDs)andtheobjectsthatMXDsinclude:legends,titles,images,layers,andthemapviewitself.ArcPyalsohastoolsthatarenotavailableinArcToolbox.Themostpowerfulofthesearethedatacursors,especiallythenewDataAnalysisCursorsthatcreateamorePythonicinterfacewithGISdata.Thedatacursors,coveredextensivelyinChapters5,ArcPyCursors:Search,InsertandUpdateandChapter6,WorkingwithArcPyGeometryObjectsareveryusefultoextractrowsofdatafromdatasourcesforanalysis.
TheabilitytocontrolgeospatialanalysesusingArcPyallowsfortheintegrationofArcGIStoolsintoworkflowsthatcontainotherpowerfulPythonmodules.Python’sgluelanguageabilitiesincreasetheusefulnessofArcGISbyreducingtheneedtotreatgeospatialdatainaspecialmanner.
TheOperatingSystem(OS)moduleTheOSmodule,partofthestandardlibrary,allowsPythontoaccessoperatingsystemfunctionality.Acommonuseofthemoduleistousetheos.pathmethodtocontrolfilepathsbydividingthemintodirectorypaths(thatis,folders)andbasepaths(thatis,files).Thereisalsoausefulmethod,os.walk,whichwillwalk-throughadirectoryandreturnallfileswithinthefoldersandsubfolders.TheOSmoduleisaccessedconstantlywhenperformingGISanalysis.
ThePythonSystem(SYS)moduleThesysmodule,partofthestandardlibrary,referstothePythoninstallationitself.IthasanumberofmethodsthatwillgetinformationabouttheversionofPythoninstalled,aswellasinformationaboutthescriptandanyarguments(orparameters)suppliedtothescript,usingthesys.argvmethod.Thesys.pathmethodisveryusefultoappendthePythonfilepath;practically,thismeansthatfolderscontainingscriptscanbereferencedbyotherscriptstomakethefunctionstheycontainimportabletootherscripts.
TheXLRDandXLWTmodulesTheXLRDandXLWTmodulesareusedtoreadandwriteExcelspreadsheets,respectively.ThemodulescanbeveryusefultoextractdatafromlegacyspreadsheetsandconvertthemintousabledataforGISanalysis,ortowriteanalysisresultswhenageospatialanalysisiscompleted.TheyarenotpartofthePythonstandardlibrary,butareinstalledalongwithArcGIS10.2andPython2.7.
Commonlyusedbuilt-infunctionsThereareanumberofbuilt-infunctionsthatwewillusethroughoutthebook.Themainonesarelistedasfollows:
str:Thestringfunctionisusedtoconvertanyothertypeofdataintoastringint:Theintegerfunctionisusedtoconvertastringorfloatintoaninteger.Tonotcreateanerror,anystringpassedtotheintegerfunctionmustbeanumbersuchas1.float:Thefloatfunctionisusedtoconvertastringoranintegerintoafloat,muchliketheintegerfunction.
CommonlyusedstandardlibrarymodulesThefollowingstandardlibrarymodulesmustbeimported:
datetime:Thedatetimemoduleisusedtogetinformationaboutthedateandtime,andconvertstringdatesintoPythondates.math:Themathmoduleisusedforhigherlevelmathfunctionsthatarenecessaryattimes,suchasgettingavalueforPiorcalculatingthesquareofanumber.string:Thestringmoduleisusedforstringmanipulations.csv:TheCSVmoduleisusedtocreateandeditcomma-separatedvaluetypefiles.
Checkouthttps://docs.python.org/2/libraryforacompletelistofthebuilt-inmodulesinthestandardlibrary.
SummaryInthischapter,wediscussedabouttheZenofPythonandcoveredthebasicsofprogrammingusingPython.WebeganourexplorationofArcPyandhowitcanbeintegratedwithotherPythonmodulestoproducecompleteworkflows.WealsodiscussedthePythonstandardlibraryandthebasicdatatypesofPython.
Next,wewilldiscusshowtoconfigurePythonforusewithArcGIS,andexplorehowtouseIntegratedDevelopmentEnvironments(IDEs)towritescripts.
Chapter2.ConfiguringthePythonEnvironmentInthischapter,wewillconfigurebothPythonandourcomputertoworktogethertoexecutePythonscripts.Pathvariablesandenvironmentvariableswillbeconfiguredtoensurethatimportstatementsworkasexpected,andthatscriptsrunwhentheyareclickedon.ThestructureofthePythonfolderwillbediscussed,aswillthelocationoftheArcPymodulewithintheArcGISfolderstructure.WewillalsodiscussIntegratedDevelopmentEnvironments(IDEs),programsdesignedtoassistincodecreationandcodeexecution,andcompareandcontrastexistingIDEstodeterminewhatbenefitseachIDEcanofferwhenscriptingPythoncode.
Thischapterwillcover:
ThelocationofthePythoninterpreter,andhowitiscalledtoexecuteascriptAdjustingthecomputer’senvironmentvariablestoensurecorrectcodeexecutionIntegratedDevelopmentEnvironmentsPython’sfolderstructure,withafocusonwheremodulesarestored
WhatisaPythonscript?Let’sstartwiththeverybasicsofwritingandexecutingaPythonscript.WhatisaPythonscript?Itisasimpletextfilethatcontainsaseriesoforganizedcommandswritteninaformalizedlanguage.Thetextfilehastheextension.py,butotherthanthat,thereisnothingtodistinguishitfromanyothertextfile.ItcanbeopenedusingatexteditorsuchasNotepadorWordpad,butthemagicthatisPythondoesnotresideinaPythonscript.WithoutthePythoninterpreter,aPythonscriptcannotberunandthecommandsitcontainscannotbeexecuted.
HowPythonexecutesascriptUnderstandinghowPythonworkstointerpretascriptandthenexecutethecommandswithinisasimportantasunderstandingthePythonlanguageitself.HoursofdebugginganderrorcheckingcanbeavoidedbytakingthetimetosetupPythoncorrectly.TheinterpretivenatureofPythonmeansthatascriptwillhavetobefirstconvertedintobytecodebeforeitcanbeexecuted.WewillcoverthestepsthatPythontakestoachieveourgoalofautomatingGISanalysis.
WhatisthePythoninterpreter?ThePythoninterpreter,onaWindowsenvironment,isaprogramthathasbeencompiledintoaWindowsexecutable,whichhastheextension.exe.ThePythoninterpreter,python.exe,hasbeenwritteninC,anolderandextensivelyusedprogramminglanguagewithamoredifficultsyntax.
ProgramswritteninC,whicharealsoinitiallywrittenastextfiles,mustbeconvertedintoexecutablesbyacompiler,aspecializedprogramthatconvertsthetextcommandsintomachinecodetocreateexecutableprograms.ThisisaslowprocessthatcanmakeproducingsimpleprogramsinCalaboriousprocess.Thebenefitisthattheprogramsproducedarestandaloneprogramscapableofrunningwithoutanydependencies.Python,ontheotherhand,interpretsandexecutesthePythoncommandsquickly,whichmakesitagreatscriptinglanguage,butthescriptsmustberunthroughaninterpreterandcannotbeexecutedbythemselves.
ThePythoninterpreter,asitsnameimplies,interpretscommandscontainedwithinaPythonscript.WhenaPythonscriptisrun,orexecuted,thesyntaxisfirstcheckedtomakesurethatitconformstotherulesofPython(forexample,indentationrulesarefollowedandthevariablesfollownamingconventions).Then,ifthescriptisvalid,thecommandscontainedwithinareconvertedintobytecode,aspecializedcodethatisexecutedbythebytecodeinterpreter,avirtualmachinewritteninC.Thebytecodeinterpreterfurtherconvertsthebytecode(whichiscontainedwithinfilesthatendwiththeextension.pyc)intothecorrectmachinecodeforthecomputerbeingused,andthentheCPUexecutesthescript.Thisisacomplexprocess,whichallowsPythontomaintainasemblanceofsimplicity.
ThereareotherversionsofthePythoninterpreterthathavebeenwritteninJava(knownasJython)andin.NET(knownasIronPython);thesevariantsareusedtowritePythonscriptsinothercomputingenvironmentsandwillnotbeaddressedinthisbook.TheArcGISinstallerincludesthestandardimplementationofPython,whichisalsocalledCPythontodistinguishitfromthesevariants.
WhereisthePythoninterpreterlocated?ThelocationofthePythoninterpreterwithinthefolderstructureofacomputerisanimportantdetailtomaster.Pythonisoftendownloadeddirectlyfromwww.python.organdinstalledseparatelyfromArcGIS.However,eachArcGISversionwillrequireaspecificversionofPython;giventhisrequirement,theinclusionofPythonwithintheArcGISinstallationpackageishelpful.Forthisbook,wewillbeusingArcGIS10.2,andthiswillrequirePython2.7.
OnaWindowsmachine,thePythonfolderstructureisplaceddirectlyontheC:drive,unlessitisexplicitlyloadedonanotherdrive.TheinstallationprocessforArcGIS10.2willcreateafolderatC:\Python27,whichwillcontainanotherfoldercalledeitherArcGIS10.2orArcGIS10.2x64,dependingontheoperatingsystemandtheversionofArcGISthathasbeeninstalled.Forthisbook,Iwillbeusingthe32-bitversionofArcGIS,sothefinalfolderpathwillbeatC:\Python27\ArcGIS10.2.
Withinthisfolderareanumberofsubfolders,aswellaspython.exe(thePythoninterpreter).Alsoincludedisasecondversionoftheinterpretercalledpythonw.exe.Pythonw.exewillexecuteascriptwithoutaterminalwindowwithprogramfeedbackappearing.Bothpython.exeandpythonw.execontaincompletecopiesofallPythoncommandsandcanbeusedtoexecuteascript.
WhichPythoninterpretershouldbeused?ThegeneralruletoexecuteascriptdirectlyusingthePythoninterpretersistousepythonw.exe,asnoterminalwindowwillappear.Whenthereisaneedtotestcodesnippets,ortoseetheoutputwithinaterminalwindow,startpython.exebydouble-clickingontheexecutable.
Whenpython.exeisstarted,aPythoninterpreterconsolewillappear:
Notethedistinctivethreechevrons(>>>)thatappearbelowtheheaderexplainingversioninformation.ThatisthePythonprompt,wherecodeisenteredtobeexecutedlinebyline,insteadofinacompletedscript.Thisdirectaccesstotheinterpreterisusefultotestcodesnippetsandunderstandsyntax.Aversionofthisinterpreter,thePythonWindow,hasbeenbuiltintoArcMapandArcCatalogsinceArcGIS10.Itwillbediscussedmoreinlaterchapters.
Howdoesthecomputerknowwheretheinterpreteris?TobeabletoexecutePythonscriptsdirectly(thatis,tomakethescriptsrunbydouble-clickingonthem),thecomputerwillalsoneedtoknowwheretheinterpretersitswithinitsfolderstructure.ToaccomplishthisrequiresbothadministrativeaccountaccessandadvancedknowledgeofhowWindowssearchesforaprogram.Wewillhavetoadjustanenvironmentvariablewithintheadvancedsystemsettingsdialoguetoregistertheinterpreterwiththesystempath.
OnaWindows7machine,clickonthestartmenuandright-clickonComputer,thenselectPropertiesfromthemenu.OnaWindows8machine,clickonWindowsexplorerandrightclickonThisPC,andselectPropertiesfromthemenu.ThesecommandsareshortcutstogettotheControlPanel’sSystemandSecurity/Systemmenus.SelectAdvancedsystemsettingsfromthepanelontheleft.ClickontheEnvironmentVariablesbuttonatthebottomoftheSystemPropertiesmenuthatappears.InthelowerportionoftheEnvironmentVariablesmenu,scrollthroughtheSystemvariableswindowuntilthePathvariableappears.Selectitbyclickingonit,andthenclickingontheEditbutton.Thefollowingwindowwillappear:
Thisvariablehastwocomponents:Variablename(path)andVariablevalue.Thevalueisaseriesoffolderpathsseparatedbysemicolons.ThisisthepaththatissearchedwhenWindowslooksforspecificexecutablesthathavebeenassociatedwithafileextension.Inourcase,wewillbeaddingthefolderpaththatcontainsthePythoninterpreter.TypeC:\Python27\ArcGIS10.2(ortheequivalentonyourmachine)intotheVariablevaluefield,makingsuretoseparateitfromthevaluebeforeitwithasemicolon.ClickonOKtoexittheEditdialogue,andOKtoexittheEnvironmentVariablesmenu,andOKtoexittheSystemPropertiesmenu.ThemachinewillnowknowwherethePythoninterpreteris,asitwillsearchallfolderscontainedwithinthePathvariabletolookforanexecutablecalledPython.Totestthatthepathadjustmentworkedcorrectly,openupacommandwindow(Startmenu/runcmd)andtypepython.Theinterpretershoulddirectlyruninthecommandwindow:
IfthePythonheaderwithversioninformationandthetriplechevronappears,thepathadjustmenthasworkedcorrectly.
NoteIfthereisnoadminaccessavailable,thereisaworkaround.Inacommand-linewindow,passtheentirepathtothePythoninterpreter(C:\Python27\ArcGIS10.2\python.exe)tostarttheinterpreter.
MakePythonscriptsexecutablewhenclickedonThefinalstepinmakingthescriptsrunwhendouble-clicked(whichalsomeanstheycanrunoutsideoftheArcGISenvironment,savinglotsofmemoryoverhead)istoassociatefileswiththe.pyextensionwiththePythoninterpreter.Ifthescriptshavenotalreadybeenassociatedwiththeinterpreter,theywillappearasfilesofanunknowntypeorasatextfile.
Tochangethis,right-clickonaPythonscript.SelectOpenWith,andthenselectChooseDefaultProgram.Ifpython.exeorpythonw.exedoesnotappearasachoice,navigatetothefolderthatholdsthem(C:\Python27\ArcGIS10.2,inthiscase)andselecteitherpython.exeorpythonw.exe.Again,thedifferencebetweenthetwoistheappearanceofaterminalwindowwhenthescriptsarerunusingpython.exe,whichwillcontainanyoutputfromthescript(butthiswindowwilldisappearwhenthescriptisdone).Irecommendusingpythonw.exewhenexecutingscripts,andpython.exetotestcode.
NotePythonscriptscanalsoexplicitlycallpythonw.exebyadjustingtheextensionto.pywinsteadof.py.
IntegratedDevelopmentEnvironments(IDEs)ThePythoninterpretercontainseverythingthatisneededtoexecuteaPythonscriptortotestPythoncodebyinteractingwiththePythoninterpreter.However,writingscriptsrequiresatexteditor.ThereareusuallyatleasttwosimpletexteditorsincludedonaWindowsmachine(NotepadandWordpad)andtheyworkinanemergencytoeditascriptorevenwriteawholescript.Unfortunately,theyareverysimpleanddonotallowtheuserfunctionalitythatwouldmakeiteasiertowritemultiplescriptsorverylongscripts.
Tobridgethegap,aseriesofprogramscollectivelyknownasIntegratedDevelopmentEnvironmentshavebeendeveloped.IDEsexistforallprogramminglanguages,andincludefunctionssuchasvariablelisting,codeassist,andmore,thatmakethemidealtocraftprogrammingscripts.WewillreviewafewofthemtoassesstheirusefulnesstowritePythonscripts.Thethreediscussedasfollowsareallfreeandwell-establishedwithindifferentPythoncommunities.
IDLEPythonincludesanIDEwhenitisinstalled.TheIDEiscalledIDLE,whichisawordplayonbothIDEandthenameofaprominentmemberofMontyPython,EricIdle.ItcanbestartedinWindows7bygoingtotheStartmenuandfindingtheArcGISfolderwithintheProgramsmenu.WithinthePythonfolder,IDLEwillbeoneofthechoiceswithinthatfolder.SelectittostartIDLE.
IDLEcontainsaninteractiveinterpreter(i.e.thetriplechevron)andtheabilitytoruncompletePythonscripts.ItisalsowrittenusingPython’sbuilt-inGUImodule,calledTkinter,soithastheadvantageofbeingwritteninthesamelanguagethatitexecutes.
AnotheradvantageofusingIDLEoverthePythonconsole(python.exe)isthatanyprintstatementsorotherscriptoutputisdirectedtotheIDLEinteractivewindow,whichdoesnotdisappearafterexecutingthescript.IDLEisalsolightweightwithrespecttomemoryuse.ScriptsareopenedusingafiledialoguecontainedwithintheFilemenu,andrecentlyrunscriptsarelistedwithintheFilemenu’s,RecentFiles.
DisadvantagesofIDLEincludealimitedcodeassist(orcodeauto-complete),ausefulIDEtool,andhavingnowaytoorganizescriptsintologicalprojects.Thereisnowaytofindallvariablescontainedwithinascript,anotherusefulfeatureofotherIDEs.Also,theRecentFilesmenuhasalimitonthenumberofscriptsthatitwilllist,makingithardertofindascriptthathasnotbeenruninmonths(whichisacommonoccurrence,believeme!).IDLEisapassableIDEthatisusefulifnootherprogramscanbeinstalledonthemachine.Itisalsoveryusefulforrapidtestingofcodesnippets.WhileitisnotmymainIDE,IfindmyselfusingIDLEalmostdaily.
PythonWinPythonWin(shortforPythonforWindows)isavailableathttp://sourceforge.net/projects/pywin32/files/pywin32,andincludesbothanIDEandhelpfulmodulestousePythoninaWindowsenvironment.SelectthenewestbuildofPythonWin,andthenselectthecorrectversion32modulebasedontheinstalledversionofPython(formymachine,Iselectedpywin32-218.win32-py2.7.exe,thecorrectversionformy32-bitPython2.7installation).Runtheexecutable,andifthecorrectversionhasbeendownloaded,theinstallationGUIwillrecognizePython2.7inthesystemregistryandwillinstallitself.
PythonWinincludesanInteractiveWindowwheretheusercandirectlyinteractwiththePythoninterpreter.ScriptscanalsobeopenedwithinPythonWin,anditincludesasetoftilingcommandsintheWindowsmenuthatallowstheusertoorganizethedisplayofallopenscriptsandtheInteractiveWindow.
AnotherniceadvantagethatPythonWinhasoverIDLEistheabilitytodisplaydifferentportionsofascriptwithinthesamescriptwindow.Ifascripthasgrowntoolong,itcanbeapaintoscrollupanddownthescriptwhenediting.PythonWinallowstheusertopulldownfromthetopofthescripttocreateasecondscriptwindow,whichcanfocusonaseparatepartofthescript.Also,ontheleftside,anotherwindowcanbeopenedthatwilllistPythonclassesandvariables,makingiteasiertonavigatetoaparticularsectionofthescript.
OnesmallbuthelpfulfeaturebuiltintoPythonWin’sInteractiveWindowistheabilityto
searchthroughpreviouslyenteredcodestatements.Atthetriplechevronprompt,holddowntheCtrlkeyandusetheupanddownarrowkeystonavigatethroughthelinestofindoneofinterest.Thissavesalotoftimewhentestingaparticularsnippetofcode.
Allinall,PythonWinisausefulandeasy-to-useIDE,andmostArcGISprofessionalswhocreatePythonscriptsusePythonWin.ThedrawbacksIfindwithPythonWinincludeitslackofabilitytoorganizescriptsintoprojects,anditslackofalistofvariablesthatexistwithinthescript,whichcanbeveryhelpfulwhennavigatinglargerscripts.
AptanaStudio3Sometimesthetoolsofthegreaterprogrammingcommunitycanseemdauntingtonewscripters,whoaremorefocusedonsimplycreatingascriptthatwillsavetimeonaGISanalysisthanusingthecorrecttoolforprogrammingdaily.Itremindsmeofinexperiencedcomputerusers,whodon’tfeelliketheyneedthefullpowerofatop-of-the-linecomputerbecausetheyonlywanttobrowsetheinternetandsende-mails.
However,theexactoppositeistrue:thecomputeradverseisbetteroffhavinganeasiertousetop-of-the-linecomputer,whileanexperiencedcomputerusercouldmakedowithanetbook.
Thesamecanbesaidforprogrammersandscripters.Sometimes,it’sbettertohaveanover-the-topIDEthatwillactuallymakeascriptermoreproductive,whileanexperiencedprogrammercouldmakedowithNotepad.AllofthebellsandwhistlesincludedinanIDEsuchasAptanaStudio3willsavescripterstimeandtakeremarkablylittletimetolearn.
AptanaStudio3isavailableathttp://aptana.com.Downloadandruntheinstallerprovidedtoinstallit.Chooseadefaultmainprojectfolderthatcancontainallofthescriptsprojects;forthisbook,IcreatedafoldercalledC:\Projects.Foreachprojectcreated,Aptanawillcreateaprojectfileholdinginformationabouteachproject.WhenusingAptanaStudioatwork,usinganetworkfoldercanbeusefulasotherscanthenaccesstheprojectswiththeirrespectiveAptanainstallations.
Onceithasbeeninstalled,thenextstepistocreateaPyDevproject.GototheFilemenuandselectNew,andthenselectPyDevproject.Whencreatingthisfirstproject,PythonInterpreterwillhavetobeaddedtoAptana’sPythonpath.Aptanacansupportmorethanoneinterpreter;forourpurposes,onewilldo.GotothebottomofthePyDevprojectmenuandclickonClickheretoconfigureaninterpreter.WhenthePreferences/PythonInterpretersmenuappears,makesuretoselectInterpreter-Pythonontheleft,andthenclickonNewinthetop-rightmenu.
OnceNewhasbeenselected,asmalldialogwillappearaskingforanamefortheinterpreterandthepathtotheexecutable.Clickonbrowseandnavigatetothefolderwithpython.exe.NoterminalwindowwillbegeneratedwhenrunningaPythonscriptusingAptanaStudioasalloutputisredirectedtotheAptanaStudioconsole.Selectpython.exeandclickonOK.Next,clickonOKintheSelectInterpretermenu,andthenclickonOKinthePreferencesmenu.BackinthePyDevProjectmenu,givetheprojectaname,andeitherusethedefaultworkspacelocationoracustomone(forexample,C:\Projects).
Allofthisconfigurationonlyhastohappenthefirsttime;oncethatisdone,creatingaPyDevprojectwillonlyrequiregivinganameandlocation.Now,allofthescriptsassociatedwiththatprojectwillalwaysbelistedintheleftmenu(PyDevPackageExplorer),whichisaverypowerfulwaytoorganizeprojectsandscripts.
MakingsurethatAptanaStudioisinthePyDevperspective(intheWindows/OpenPerspective/Othermenu,choosePyDev)willgivethreemainwindows–PackageExplorerontheleft,Scriptwindowinthemiddle,andOutlinewindowontheright,wherevariablescontainedwithinascriptarelisted.Clickingononeofthevariablesontherightwillmovethescriptwindowtothatsectionofthecode,makingscriptnavigation
fast.Also,IliketoaddtheConsolewindowinthemiddlebelowtheScriptwindow,wheretheoutputofthescriptcanbedisplayed.
OpenscriptseachhaveatabwithintheScriptwindow,makingiteasytoswitchbetweenthescripts.Also,thewindowscanbeclosedtogivemoreroomtotheScriptwindowasneeded.Hoveringoveravariablewithinascriptwillcallupapop-upmenuthatdescribeswherethevariablewasfirstcreated,whichcanbealifesaverasitiseasytoforgetattimeswhichvariableiswhich(unless,ofcourse,theyareclearlynamedaccordingtotherulesdescribedinthepreviouschapter;eventhen,itcanbeapainattimes).
IDEsummaryTherearemanyotherIDEs,bothcommercialandfree,availableforcodinginPython.Intheend,eachGISanalystmustchoosethetoolthatmakesthemfeelproductiveandcomfortable.Thismaychangeasprogrammingbecomesabiggerpartoftheirdailyworkflow.BesuretotestoutafewdifferentIDEstofindonethatiseasytouseandintuitive.
PythonfolderstructurePython’sfolderstructureholdsmorethanjustthePythonInterpreter.Withinthesubfoldersresideanumberofimportantscripts,digitallinklibraries,andevenClanguagemodules.Notallofthescriptsareusedallthetime,buteachhasaroleinmakingthePythonprogrammingenvironmentpossible.Themostimportantfoldertoknowaboutisthesite-packagesfolder,wheremostmodulesthatwillbeimportedinPythonscriptsarecontained.
WheremodulesresideWithineveryPythonfolderisafoldercalledLib,andwithinthatfolderisafoldercalledsite-packages.Onmymachine,thefoldersitsatC:\Python27\ArcGIS10.2\Lib\site-packages.Almostallthird-partymodulesarecopiedintothisfoldertobeimportedasneeded.Themainexceptiontothisrule,forourpurposes,istheArcPymodule,whichisstoredwithintheArcGISfolderintheProgramFilesfolder(forexample,C:\ProgramFiles(x86)\ArcGIS\Desktop10.2\arcpy).Tomakethatpossible,theArcGISinstalleradjuststhePythonsystempath(usingthesysmodule)tomakethearcPymoduleimportable.
UsingPython’ssysmoduletoaddamodulePython’ssysmoduleisamodulethatallowstheusertotakeadvantageofsystemtoolsbuiltintothePythonInterpreter.Oneofthemostusefulofthefunctionsinthesysmoduleissys.path.Itisalistoffilepaths,whichtheusercanmodifytoadjustwherePythonwilllookforamoduletoimport,withoutneedingadministrativeaccess.
WhenPython2.7isinstalledbytheArcGIS10.2installer,theinstallertakesadvantageofthesys.pathfunctionstoaddC:\ProgramFiles(x86)\ArcGIS\Desktop10.2\arcpytothesystempath.Totestthis,startthePythonInterpreteroranIDEandtypethefollowing:
>>>importsys
>>>printsys.path
Theoutputisasfollows:
['','C:\\WINDOWS\\SYSTEM32\\python27.zip',
'C:\\Python27\\ArcGIS10.2\\Dlls','C:\\Python27\\ArcGIS10.2\\lib',
'C:\\Python27\\ArcGIS10.2\\lib\\plat-win',
'C:\\Python27\\ArcGIS10.2\\lib\\lib-tk',
'C:\\Python27\\ArcGIS10.2\\Lib\\site-packages\\pythonwin',
'C:\\Python27\\ArcGIS10.2','C:\\Python27\\ArcGIS10.2\\lib\\site-packages',
'C:\\ProgramFiles(x86)\\ArcGIS\\Desktop10.2\\bin','C:\\ProgramFiles
(x86)\\ArcGIS\\Desktop10.2\\arcpy','C:\\ProgramFiles
(x86)\\ArcGIS\\Desktop10.2\\ArcToolbox\\Scripts',
'C:\\Python27\\ArcGIS10.2\\lib\\site-packages\\win32',
'C:\\Python27\\ArcGIS10.2\\lib\\site-packages\\win32\\lib']
Thesystempath(storedinthevariablesys.path)includesallofthefoldersthatArcPyrequirestoautomateArcGIS.ThesystempathincorporatesalldirectorieslistedinthePYTHONPATHenvironmentvariable(ifonehasbeencreated);thisisseparatefromtheWindowspathenvironmentvariablediscussedpreviously.ThetwoseparatepathvariablesworktogethertohelpPythonlocatemodules.
Thesys.path.append()methodThesys.pathfunctionisalist(didyounoticethesquarebracketsintheprecedingcodeoutput?)andassuchcanbeappendedorextendedtoincludenewfilepathsthatwillpointtomodulestheuserwantstoimport.Toavoidtheneedtoadjustsys.path,copythemoduleintothesite-packagesfolder.Whenthisisnotpossible,usethesys.path.append()methodinstead:
>>>sys.path.append("C:\\Projects\\Requests")
>>>sys.path
['','C:\\WINDOWS\\SYSTEM32\\python27.zip',
'C:\\Python27\\ArcGIS10.2\\Dells',
'C:\\Python27\\ArcGIS10.2\\lib',
..'C:\\Python27\\ArcGIS10.2\\lib\\plat-win',
..'C:\\Python27\\ArcGIS10.2\\lib\\lib-tk',
..'C:\\Python27\\ArcGIS10.2\\Lib\\site-packages\\pythonwin',
..'C:\\Python27\\ArcGIS10.2',..'C:\\Python27\\ArcGIS10.2\\lib\\site-
packages','C:\\Program
..Files(x86)\\ArcGIS\\Desktop10.2\\bin','C:\\ProgramFiles
..(x86)\\ArcGIS\\Desktop10.2\\arcpy','C:\\ProgramFiles
..(x86)\\ArcGIS\\Desktop10.2\\ArcToolbox\\Scripts',
..'C:\\Python27\\ArcGIS10.2\\lib\\site-packages\\win32',
..'C:\\Python27\\ArcGIS10.2\\lib\\site-packages\\win32\\lib',
..'C:\\Projects\\Requests']
Whenthesys.path.append()methodisused,theadjustmentistemporary.AdjustthePYTHONPATHenvironmentvariableintheWindowsSystemPropertiesmenu(discussedinthepathenvironmentvariablesection)tomakeapermanentchange(andcreatethePYTHONPATHifithasnotbeencreated).
Onelastnoteisthattoimportamodulewithoutadjustingthesystempathorcopyingthemoduleintothesite-packagesfolder,placethemoduleinthefolderwiththescriptthatisimportingit.Aslongasthemoduleisconfiguredcorrectly,itwillworknormally.Thisisusefulwhenthereisnoadministrativeaccessavailabletoamachine.
SummaryInthischapter,wecoveredalotabouthowPythonworkstoexecutescriptsandcommands,andaboutdevelopmentenvironmentsusedtocraftscripts.Inparticular,wediscussedhowaPythonscriptisreadandexecutedbythePythonInterpreter,wherethePythonInterpreterislocatedwithinthePythonfolderstructure,andwhatthedifferentPythonscriptextensionsmean(.py,.pyc,.pyw).WealsocoveredIntegratedDevelopmentEnvironmentsandhowtheycompareandcontrast.
Inthenextchapter,wewillcoverhowtouseModelBuildertoconvertamodeledanalysisintoaPythonscript,andhowtomakeitmorepowerfulthantheexportedversion.
Chapter3.CreatingtheFirstPythonScriptNowthatwehavePythonconfiguredtofitourneeds,wecancreatePythonscripts.ThischapterwillexplorehowtouseArcGISModelBuildertomodelasimpleanalysisasthebasisforourscript.ModelBuilderisveryusefulonitsownandforcreatingPythonscriptsasithasanoperationalandavisualcomponent,andallmodelscanbeoutputtedasPythonscripts.ThiswillallowustocomparehowthemorefamiliarModelBuilderutilizestoolsintheArcToolboxtohowPythonhandlesthesametools.WewillalsodiscussiterationandwhenitisbesttousePythonoverModelBuilder.
Inthischapter,wewillcoverthefollowingtopics:
ModelingasimpleanalysisusingModelBuilderExportingthemodelouttoaPythonscript
Prerequisites“AlongwithArcGISModelBuilder,adatasetandscriptsarerequired.”
Forthischapter,theaccompanyingdataandscriptsshouldbedownloadedfromPacktPublishing’swebsite.Thecompletedscriptsareavailableforcomparisonpurposesandthedatawillbeusedforthischapter’sanalysis.
ModelBuilderArcGIShasbeenindevelopmentsincethe1970s.Duringthattime,itincludedavarietyofprogramminglanguagesandtoolstohelpGISanalystsautomateanalysesandmapproduction.TheseincludetheAvenuescriptinglanguageintheArcGIS3xseriesandtheARCMacroLanguage(AML)intheARC/Infoworkstationdays,aswellasVBScriptupuntilArcGIS10xwhenPythonwasintroduced.AnotherusefultoolintroducedinArcGIS9xwasModelBuilder,avisualprogrammingenvironmentusedforbothmodelinganalysisandcreatingtoolsthatcanbeusedrepeatedlywithdifferentinputfeatureclasses.
AnotherusefulfeatureofModelBuilderisanexportfunctionthatallowsmodelerstocreatePythonscriptsdirectlyfromamodel.ThiswillmakeiteasiertocomparehowinputsinaModelBuildertoolareacceptedversushowaPythonscriptcallsthesametoolandsuppliestheinputstoit,orhowthefeatureclassesthatarecreatedarenamedandplacedwithinthefilestructure.ModelBuilderisafantastictoolthatwillmakeiteasyforaGISanalysttobridgethegapfromnormalGISworkflowstoautomatedPython-basedworkflows.
CreatingamodelandexportingtoPythonThischapterwilldependonthedownloadableSanFrancisco.gdbfilegeodatabase,availablefromthePacktPublishingwebsite.TheSanFranciscoGDBcontainsdatadownloadedfromdata.sfgov.organdtheUSCensus’AmericanFactfinderwebsiteavailableatfactfinder2.census.gov.Allcensusandgeographicdataincludedinthegeodatabaseisfromthe2010census.ThedataiscontainedwithinafeaturedatasetcalledSanFrancisco.ThedatainthisfeaturedatasetisinNAD83CaliforniaStatePlaneZone3andthelinearunitofmeasureistheUSFoot(thiscorrespondstoSRID2227intheEuropeanPetroleumSurveyGroup,orEPSG,format).
Theanalysiswewillcreatewiththemodel,andeventuallyexporttoPythonforfurtherrefinement,willusebusstopsalongaspecificlineinSanFrancisco.Thesebusstopswillbebufferedtocreatearepresentativeregionaroundeachbusstop.Thebufferedareaswillthenbeintersectedwithcensusblockstofindouthowmanypeoplearewithineachrepresentativeregionaroundthebusstops.
ModelingtheSelectandBuffertoolsUsingModelBuilder,wewillfirstmodelthebasisofthebusstopanalysis.Onceithasbeenmodeled,itwillbeexportedasanautomaticallygeneratedPythonscript.Followthesestepstobegintheanalysis:
1. OpenupArcCatalogandcreateafolderconnectiontothefoldercontainingSanFrancisco.gdb.Right-clickongeodatabaseandaddanewtoolboxcalledChapter3Tools.
2. Next,openModelBuilderandcreateaModel,savingitintheChapter3ToolstoolboxasChapter3Model1.
3. DragtheBus_StopsfeatureclassandtheSelecttoolfromtheAnalysis/ExtracttoolsetinArcToolbox.
4. OpentheSelecttoolandnametheoutputfeatureclassInbound71.MakesurethatthefeatureclassiswrittentotheChapter3Resultsfeaturedatasetintothemodel.
5. OpentheExpressionSQLQueryBuilderandcreatethefollowingSQLexpression:NAME=‘71IB’ANDBUS_SIGNAG=‘FerryPlaza’.
6. ThenextstepistoaddaBuffertoolfromtheAnalysis/Proximitytoolset.TheBuffertoolwillbeusedtocreatebuffersaroundeachbusstop.Thebufferedbusstopsallowustointersectwithcensusdataintheformofcensusblocks,creatingtherepresentativeregionsaroundeachbusstop.
7. ConnecttheoutputoftheSelecttool(Inbound71)totheBuffertool.OpenuptheBuffertoolandadd400totheDistancefield,andmaketheunitsFeet.Leavetherestoftheoptionsblank.ClickonOKandreturntothemodel.
AddingtheIntersecttoolNowthatwehaveselectedthebuslineofinterest,andbufferedthestopstocreaterepresentativeregions,wewillneedtointersecttheregionswiththecensusblockstofindthepopulationofeachrepresentativeregion:
1. First,addtheCensusBlocks2010featureclassfromtheSanFranciscofeaturedatasettothemodel.
2. Next,addtheIntersecttool,locatedintheAnalysis/OverlaytoolsetinArcToolbox.WhilewecoulduseSpatialJointoachieveasimilarresult,IamusingtheIntersecttooltocapturetheareaofintersectforuselaterinthemodelandscript.
Atthispoint,ourmodelshouldlooklikethis:
TallyingtheanalysisresultsAfterwecreatedthissimpleanalysis,thenextstepistodeterminetheresultsforeachbusstop.Findingthenumberofpeoplethatliveincensusblockstouchedbythe400feetbufferofeachbusstopinvolvesexaminingeachrowofdatainthefinalfeatureclassandselectingrowsthatcorrespondtothebusstop.Oncetheseareselected,asumoftheselectedrowswouldbecalculatedeitherusingtheFieldCalculatorortheSummarizetool.Allofthesemethodswillwork,andyetnoneareperfect.Theytaketoolong,andworse,arenotrepeatableautomaticallyifanassumptioninthemodelisadjusted(ifthebufferisadjustedfrom400feetto500feet,forinstance).
ThisiswherethetraditionalusesofModelBuilderbegintofailanalysts.Itshouldbeeasytoinstructthemodeltoselectallrowsassociatedwitheachbusstop,andthengenerateasummedpopulationfigureforeachbusstop’srepresentativeregion.Itwouldbeevenbettertohavethemodelcreateaspreadsheettocontainthefinalresultsoftheanalysis.It’stimetousePythontotakethisanalysistothenextlevel.
ExportingthemodelandadjustingthescriptWhilemodelinganalysisinModelBuilderhasitsdrawbacks,thereisonefantasticoptionbuiltintoModelBuilder;theabilitytocreateamodelandthenexportthemodeltoPython.AlongwiththeArcGIShelpdocumentation,itisthebestwaytodiscoverthecorrectPythonsyntaxtousewhenwritingArcPyscripts.
CreateafolderthatcanholdtheexportedscriptsnexttotheSanFranciscogeodatabase(forexample,C:\Projects\Scripts).ThiswillholdboththeexportedscriptsthatArcGISautomaticallygenerates,andtheversionsthatwewillbuildfromthosegeneratedscripts.
OpenthemodelcalledChapter3Model1andclickontheModelmenuintheupperleft.SelectExportfromthemenu,andthenselectToPythonScript.SavethescriptinthescriptfolderasChapter3Model1.py.
NoteNotethatthereisalsotheoptiontoexportthemodelasagraphic.Creatingagraphicofthemodelisagoodwaytosharewhatthemodelisdoingwithotheranalystswithouttheneedtosharethemodelandthedata,andcanalsobeusefulwhensharingPythonscriptsaswell.
TheautomaticallygeneratedscriptOpentheautomaticallygeneratedscriptinanIDE.Itshouldlooklikethis:
#-*-coding:utf-8-*-
#-------------------------------------------------------------------------
--
#8662_Chapter3Model1.py
#Createdon:2014-04-2221:59:31.00000
#(generatedbyArcGIS/ModelBuilder)
#Description:
#-------------------------------------------------------------------------
--
#Importarcpymodule
importarcpy
#Localvariables:
Bus_Stops="C:\\Projects\\PacktDB.gdb\\SanFrancisco\\Bus_Stops"
CensusBlocks2010=
"C:\\Projects\\PacktDB.gdb\\SanFrancisco\\CensusBlocks2010"
Inbound71="C:\\Projects\\PacktDB.gdb\\Chapter3Results\\Inbound71"
Inbound71_400ft_buffer=
"C:\\Projects\\PacktDB.gdb\\Chapter3Results\\Inbound71_400ft_buffer"
Intersect71Census=
"C:\\Projects\\PacktDB.gdb\\Chapter3Results\\Intersect71Census"
#Process:Select
arcpy.Select_analysis(Bus_Stops,
Inbound71,
"NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'")
#Process:Buffer
arcpy.Buffer_analysis(Inbound71,
Inbound71_400ft_buffer,
"400Feet","FULL","ROUND","NONE","")
#Process:Intersect
arcpy.Intersect_analysis("C:\\Projects\\PacktDB.gdb\\Chapter3Results\\Inbou
nd71_400ft_buffer
#;C:\\Projects\\PacktDB.gdb\\SanFrancisco\\CensusBlocks2010#",
Intersect71Census,"ALL","","INPUT")
Let’sexaminethisscriptlinebyline.Thefirstlineisprecededbyapoundsign(#),whichagainmeansthatthislineisacomment;however,itisnotignoredbythePythoninterpreterwhenthescriptisexecutedasusualbutisusedtohelpPythoninterprettheencodingofthescriptasdescribedhere:http://legacy.python.org/dev/peps/pep-0263.
Thesecondcommentedlineandthethirdlineareincludedfordecorativepurposes.Thenextfourlines,allcommented,areusedtoprovidereaderswithinformationaboutthescript,whatitiscalledandwhenitwascreated,alongwithadescriptionthatispulledfromthemodel’sproperties.Anotherdecorativelineisincludedtoseparateouttheinformativeheaderfromthebodyofthescriptvisually.Whilethecommentedinformationsectionisnicetoincludeinascriptforotherusersofthescript,itisnotnecessary.
Thebodyofthescript,ortheexecutableportionofthescript,startswiththeimportarcpyline.Importstatementsare,byconvention,includedatthetopofthebodyofthescript.Inthisinstance,theonlymodulethatisbeingimportedisArcPy.
ModelBuilder’sexportfunctioncreatesnotonlyanexecutablescript,butalsocommentseachsectiontohelpmarkthedifferentsectionsofthescript.ThecommentslettheuserknowwherethevariablesarelocatedandwheretheArcToolboxtoolsarebeingexecuted.Thecommentswillgrowtobesuperfluousasthereadergrowstounderstandthecode,butitwasniceofESRItoincludethecomments.
Belowtheimportstatementsarethevariables.Inthiscase,thevariablesrepresentthefilepathstotheinputandoutputfeatureclasses.Thevariablenamesarederivedfromthenamesofthefeatureclasses(thebasenamesofthefilepaths).Thefilepathsareassignedtothevariablesusingtheassignmentoperator(=),andthepartsofthefilepathsareseparatedbytwobackslashes.
FilepathsinPythonItwouldbegoodtoreviewhowfilepathsareusedinPythoncomparedtohowtheyarerepresentedinWindows.InPython,filepathsarestrings,andstringsinPythonhavespecialcharactersusedtorepresenttabs(\t),newlines(\n),orcarriagereturns(\r),amongmanyothers.Thesespecialcharactersallincorporatesinglebackslashes,makingitveryhardtocreateafilepaththatusessinglebackslashes.Thiswouldnotbeabigdeal,exceptthatfilepathsinWindowsExplorerallusesinglebackslashes.
Thereareanumberofmethodsusedtoavoidthisissue.PythonwasdevelopedwithintheLinuxenvironment,wherefilepathshaveforwardslashes.ThismorePythonicrepresentationisalsoavailablewhenusingPythoninaWindowsenvironment,demonstratedasfollows:
WindowsExplorer:
"C:\Projects\PacktDB.gdb\Chapter3Results\Intersect71Census"
Pythonicversion:
"C:/Projects/PacktDB.gdb/Chapter3Results/Intersect71Census"
WithinaPythonscript,thefilepathwiththeforwardslasheswillwork,whiletheWindowsExplorerversionmightcausethescripttothrowanexception.
AnothermethodusedtoavoidtheissuewithspecialcharactersistheoneemployedbyModelBuilderwhenitautomaticallycreatesthePythonscriptsfromamodel.Inthiscase,thebackslashesareescapedusingasecondbackslash.Theprecedingscriptusesthissecondmethodtoproducethefollowingresults:
Pythonescapedversion:
"C:\\Projects\\PacktDB.gdb\\Chapter3Results\\Intersect71Census"
Thethirdmethod,whichIprefer,istocreatewhatisknownasarawstring.Thisisthesameasaregularstring,butitincludesanrbeforethescriptbegins.ThisralertsthePythonInterpreterthatthefollowingscriptdoesnotcontainanyspecialcharactersorescapecharacters.Hereisanexampleofhowitisused:
Pythonrawstring:
r"C:\Projects\PacktDB.gdb\Chapter3Results\Intersect71Census"
UsingrawstringswillmakeiteasiertograbafilepathfromWindowsExplorerandaddittoastringinsideascript.Itwillalsomakeiteasiertoavoidaccidentallyforgettingtoincludeasetofdoublebackslashesinafilepath,whichhappensallthetimeandisthecauseofmanyscriptbugs.
Continuingthescriptanalysis:theArcPytoolsThenext,andmostimportant,sectionofthescriptiswheretheanalysisisexecuted.Thesametoolsthatwecreatedinthemodel,theSelect,theBuffer,andtheIntersecttools,areincludedinthissection.Thesameparametersthatwesuppliedinthemodelarealsoincludedhere:theinputsandoutputs,plustheSQLstatementintheSelecttool,andthebufferdistanceintheBuffertool.
Thetoolparametersaresuppliedtothetoolsinthescriptinthesameorderastheyappearinthetoolinterfacesinthemodel.HereistheSelecttoolinthescript:
arcpy.Select_analysis(Bus_Stops,Inbound71,"NAME='71IB'ANDBUS_SIGNAG
='FerryPlaza'")
Itworkslikethis.ThearcPymodulehasamethod,oraspecialproperty,calledSelect_analysis.Thismethod,whencalled,requiresthreeparameters:theinputfeatureclass(orshapefile),theoutputfeatureclass,andtheSQLstatement.Inthisexample,theinputisrepresentedbythevariableBus_StopsandtheoutputfeatureclassisrepresentedbythevariableInbound71,bothofwhicharedefinedinthevariablesection.TheSQLstatementisincludedasthethirdparameter.Notethatitcouldalsoberepresentedbyavariable,ifthevariablewasdefinedabovethisline;theSQLstatement,asastring,couldbeassignedtoavariableandthevariablecouldreplacetheSQLstatementasthethirdparameter.Hereisanexampleofparameterreplacementusingavariable:
sqlStatement="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
arcpy.Select_analysis(Bus_Stops,Inbound71,sqlStatement)
WhileModelBuilderisgoodaboutassigninginputandoutputfeatureclassestovariables,itdoesnotassignvariablestoeveryportionoftheparameter.Thiswillbeanimportantthingtocorrectwhenweadjustandbuildourownscripts.
TheBuffertoolacceptsasimilarsetofparametersastheSelecttool.Thereisaninputfeatureclassrepresentedbyavariable,anoutputfeatureclassvariable,andthedistancethatweprovided(400feetinthiscase),alongwithaseriesofparametersthataresuppliedbydefault.Notethattheparametersrelyonkeywords,andthesekeywordscanbeadjustedwithinthetextofthescripttoadjusttheresultingbufferoutput.Forinstance,FeetcouldbeadjustedtoMetersandthebufferwouldmuchlarger.CheckthehelpsectionofthetooltobetterunderstandhowtheotherparameterswillaffectthebufferandtofindthekeywordsargumentsthatwillbeacceptedbytheBuffertoolinArcPy.Also,asnotedearlier,alloftheparameterscouldbeassignedtovariables,whichcansavetimeifthesameparametersareusedrepeatedlythroughoutascript.
Sometimesthesuppliedparameterismerelyanemptystring,asisthecaseherewiththelastparameter:
arcpy.Buffer_analysis(Inbound71,Inbound71_400ft_buffer,
"400Feet","FULL","ROUND","NONE","")
Theemptystring,whichinthiscasesignifiesthatthereisnotadissolvefieldforthisbuffer,isfoundquitefrequentlywithinArcPy.Itcouldalsoberepresentedbytwosinglequotes,butModelBuilderhasbeenbuilttousedoublequotestoencasestrings.
TheIntersecttoolandstringmanipulationThelasttool,theIntersecttool,usesadifferentmethodtorepresentthefilesthatneedtobeintersectedtogetherwhenthetoolisexecuted.Becausethetoolacceptsmultiplefilesintheinputsection(meaningthereisnolimittothenumberoffilesthatcanbeintersectedtogetherinoneoperation),itstoresallofthefilepathswithinonestring.Thestringusesthehashorpoundsign(#)toseparatethefilepathswithintheinputstring.ThisslightdeviationmustbedealtwithifwearetousetheIntersecttoolinaScripttool.Ifwearebuildingatoolfromthisscript,wewillnotknowthefilesthatwillbeintersectedbeforetheyarerun,soweneedtoknowthemethodstodealwithinsertingvariablesintostrings.
Therearethreemethodstoinsertvariablesintostrings.Eachmethodhasdifferentadvantagesanddisadvantagesofatechnicalnature.It’sgoodtoknowaboutallthreeofthemastheyhaveusesbeyondourneedshere,solet’sreviewthem.
Thestringmanipulationmethod1–stringadditionStringadditionisanoddconceptatfirstasitwouldnotseempossibletoaddstringstogether,unlikeintegersorfloats,whicharenumbers.However,withinPythonandotherprogramminglanguages,thisisanormalstep.Usingtheplussign(+),stringsareaddedtogethertomakelongerstringsorallowvariablestobeaddedtothemiddleofexistingstrings.Herearesomeexamplesofthisprocess:
>>>aString="Thisisastring"
>>>bString="andthisisanotherstring"
>>>aString+bString
Theoutputisasfollows:
'Thisisastringandthisisanotherstring'
>>>cString=aString+bString
>>>cString
Theoutputisasfollows:
'Thisisastringandthisisanotherstring'
Twoormorestringscanbeaddedtogether,andcanevenbeassignedtoathirdvariable.ThisprocesscanbeusefulforsituationssuchastheinputstringfortheIntersecttool.Thestringcanbebrokenupandvariablesrepresentingthefilepathscanbeinsertedintothemiddleofthestring:
filePath1=r"C:\Projects\Inbound71_400ft_buffer"
filePath2=r"C:\Projects\CensusBlocks2010"
arcpy.Intersect_analysis(filePath1+"#;"+filePath2+"#",
Intersect71Census,"ALL","","INPUT")
Thisisapowerfulandusefulwaytoinsertthefilepathsintotheinputstring.Aslongastheseparatorsarestillincludedinthestring,thestringwillstillbevalidandtheIntersecttoolwillrunasexpected.Hereiswhatthestringwilllooklikewhenthestringadditioniscompleted:
>>>filePath1=r"C:\Projects\Inbound71_400ft_buffer"
>>>filePath2=r"C:\Projects\CensusBlocks2010"
>>>inputString=filePath1+"#;"+filePath2+"#"
>>>printinputString
Theoutputisasfollows:
C:\Projects\Inbound71_400ft_buffer#;C:\Projects\CensusBlocks2010#
Anothersimilaroffshootofstringadditionisstringmultiplication,wherestringsaremultipliedbyanintegertoproducerepeatedversionsofthestring:
>>>"string"*3
Theoutputisasfollows:
'stringstringstring'
Thestringmanipulationmethod2–stringformatting#1Thesecondmethodofstringmanipulation,knownasstringformatting,involvesaddingplaceholdersintothestringthatwillacceptspecifickindsofdata.Thismeansthatthesespecialstringscanacceptotherstringsaswellasintegersandfloatvalues.Theseplaceholdersusethemodulo(%)andakeylettertoindicatethetypeofdatatoexpect.Stringsarerepresentedusing%s,floatsarerepresentedusing%f,andintegersarerepresentedusing%d.Thefloatscanalsobeadjustedtolimitthedigitsincludedbyaddingamodifyingnumberafterthemodulo.Ifthereismorethanoneplaceholderinastring,thevaluesarepassedtothestringinatuple.
ThismethodhasbecomelesspopularsincethethirdmethoddiscussedinthefollowingsectionwasintroducedinPython2.6,butitisstillvaluabletoknowasmanyolderscriptsuseit.Hereisanexampleofthismethod:
>>>origString="Thisstringhasasaplaceholder%s"
>>>newString=origString%"andthistextwasadded"
>>>printnewString
Theoutputisasfollows:
Thisstringhasasaplaceholderandthistextwasadded
Hereisanexamplewhenusingafloatplaceholder:
>>>floatString1="Thisstringhasafloathere:%f"
>>>newString=floatString1%1.0
>>>printnewString
Theoutputisasfollows:
Thisstringhasafloathere:1.000000
>>>floatString2="Thisstringhasafloathere:%.1f"
>>>newString2=floatString2%1.0
>>>printnewString2
Theoutputisasfollows:
Thisstringhasafloathere:1.0
Hereisanexampleusinganintegerplaceholder:
>>>intString="Hereisaninteger:%d"
>>>newString=intString%1
>>>printnewString
Theoutputisasfollows:
Hereisaninteger:1
FortheIntersecttool,the%ssymbolcanbeusedtoacceptthefilepathstringvariables:
filePath1=r"C:\Projects\Inbound71_400ft_buffer"
filePath2=r"C:\Projects\CensusBlocks2010"
arcpy.Intersect_analysis("%s#;%s#"%(filePath1,filePath2),
Intersect71Census,"ALL","","INPUT")
Thestringmanipulationmethod3–stringformatting#2Thefinalmethod,themostrecentlyintroduced,isalsoknownasstringformatting.Itissimilartothestringformattingdiscussedearlier,withtheaddedbenefitofnotrequiringaspecifictypeofplaceholder.Theplaceholders,ortokensastheyarealsoknown,areonlyrequiredtobeinordertobeaccepted.Theformatfunctionisbuiltintostrings;byadding.formattothestring,andpassinginparameters,thestringacceptsthevalues:
>>>formatString="Thisstringhas3tokens:{0},{1},{2}"
>>>newString=formatString.format("String",2.5,4)
>>>printnewString
Theoutputisasfollows:
Thisstringhas3tokens:String,2.5,4
Thetokensdon’thavetobeinorderwithinthestring,andcanevenberepeated.Theorderisderivedfromtheparameterssuppliedtothe.formatfunctionthatpassesthevaluestothestring.
FortheIntersecttool,thestringformattingwouldlooklikethis:
filePath1=r"C:\Projects\Inbound71_400ft_buffer"
filePath2=r"C:\Projects\CensusBlocks2010"
arcpy.Intersect_analysis("{0}#;{1}#".format(filePath1,filePath2),
Intersect71Census,"ALL","","INPUT")
Thethirdmethodhasbecomemygo-tomethodforstringmanipulationbecauseoftheabilitytoaddthevaluesrepeatedlyandmakeitpossibletoavoidsupplyingthewrongtypeofdatatoaspecificplaceholder,unlikethesecondmethod.
AdjustingtheScriptNowisthetimetotaketheautomaticallygeneratedscriptandadjustittofitourneeds.Wewantthescripttobothproducetheoutputdata,andtohaveitanalyzethedataandtallytheresultsintoaspreadsheet.Thisspreadsheetwillholdanaveragedpopulationvalueforeachbusstop.Theaveragewillbederivedfromeachcensusblockthatthebufferedrepresentativeregionsurroundingthestopsintersected.SavetheoriginalscriptasChapter3Model1Modified.py.
AddingtheCSVmoduletothescriptForthisscript,wewillusetheCSVmodule,ausefulmoduletocreateCommaSeparatedValuespreadsheets.Itssimplesyntaxwillmakeitausefultooltocreatescriptoutputs.ItshouldbenotedthatArcGISforDesktopalsoinstallsthexlrdandxlwtmodules,usedtoreadorgenerateExcelspreadsheetsrespectively,whenitisinstalled.
JustbelowtheimportarcPyline,addimportcsv.Thiswillallowustousethecsvmoduletocreatethespreadsheet:
#Importarcpymodule
importarcpy
importcsv
ThenextadjustmentismadetotheIntersecttool.Noticethatthetwopathsincludedintheinputstringarealsodefinedasvariablesinthevariablesection.Removethefilepathsfromtheinputstringsandreplacethemwithnumberedplaceholdertokens,andthenaddtheformatfunctionandsupplythevariablesasplaceholders:
#Process:Intersect
arcpy.Intersect_analysis("{0}#;{1}#".format(..............
Inbound71_400ft_buffer,CensusBlocks2010),
Intersect71Census,"ALL","","INPUT")
Accessingthedata:UsingacursorNowthatthescriptisinplacetogeneratetherawdataweneed,weneedawaytoaccessthedataheldintheoutputfeatureclassfromtheIntersecttool.Thisaccesswillallowustoaggregatetherowsofdatarepresentingeachbusstop.Wealsoneedsomethingtoholdtheaggregatedatainthememory,tobewrittentothespreadsheet.
Toaccomplishthesecondpart,wewilluseaPythondictionary.Toaccomplishthefirstpart,wewilluseamethodbuiltintotheArcPymodule:theDataAccessSearchCursor.
ThePythondictionarywillbeaddedbelowtheIntersecttool.AdictionaryinPythoniscreatedusingcurlybrackets.Addthefollowinglinetothescript:
dataDictionary={}
ThisscriptwillusetheBusStopIDsaskeysforthedictionary.Thevalueswillbelists,whichwillholdallofthepopulationvaluesassociatedwitheachBusStopID.AddthefollowinglinestogenerateaDataCursor:
witharcpy.da.SearchCursor(Intersect71Census,["STOPID","POP10"])as
cursor:
forrowincursor:
busStopID=row[0]
pop10=row[1]
ifbusStopIDnotindataDictionary.keys():
dataDictionary[busStopID]=[pop10]
else:
dataDictionary[busStopID].append(pop10)
ThisiterationcombinesafewideasinPythonandArcPy.Thewith…asstatementisusedtocreateavariable(cursor)thatrepresentsthearcpy.da.SearchCursorobject.Itcouldalsobewrittenlikethis:
cursor=arcpy.da.SearchCursor(Intersect71Census,["STOPID","POP10"])
NoteTheadvantageofthewith…asstructureisthatthecursorobjectiserasedfrommemorywhentheiterationiscompleted,whicheliminateslocksonthefeatureclassesbeingevaluated.
Thearcpy.da.SearchCursor()functionrequiresaninputfeatureclass,andalistoffieldstobereturned.Optionally,aSQLstatementcanlimitthenumberofrowsreturned.
Thenextline,forrowincursor,istheiterationthroughthedata.ItisnotanormalPythoniciteration,adistinctionthatwillhaveramificationsincertaininstances.Forinstance,however,itdoesallowforrow-by-rowaccesstodatacontainedwithinthesuppliedfeatureclass.NotethatwhenusingaSearchCursor,eachrowofdataisreturnedasatuple,whichcannotbemodified.Thedatacanbeaccessedusingindexes,asshownintheprecedingcode,wherethetwomembersofthetupleareassignedtovariables.
Theif/elseconditionalallowsthedatatobesorted.Asnotedearlier,theBusStopIDs,whicharethefirstmemberofthedataincludedinthetuple,willbeusedasakey.The
conditionalevaluateswhethertheBusStopIDisincludedinthedictionary’sexistingkeys(whicharecontainedinalistandaccessedusingthedictionary.keys()method).Ifitisnot,itisaddedtothekeys,andassignedavaluethatisalistcontaining(atfirst)onepieceofdata,thepopulationvaluecontainedinthatrow.Ifitdoesexistinthekeys,thelistisappendedwiththenextpopulationvalueassociatedwiththatBusStopID.Withthiscode,wehavenowsortedeachcensusblockpopulationaccordingtotheBusStopwithwhichitisassociated.
Next,weneedtoaddcodetocreatethespreadsheet.Thiscodewillusethesamewith…asstructure,andwillgenerateanaveragepopulationvaluebyusingtwobuilt-inPythonfunctions,sum,whichcreatesasumfromalistofnumbers,andlen,whichwillgetthelengthofalist,tuple,orstring:
withopen(r'C:\Projects\Output\Averages.csv','wb')ascsvfile:
csvwriter=csv.writer(csvfile,delimiter=',')
forbusStopIDindataDictionary.keys():
popList=dataDictionary[busStopID]
averagePop=sum(popList)/len(popList)
data=[busStopID,averagePop]
csvwriter.writerow(data)
TheaveragepopulationvalueisretrievedfromthedictionaryusingtheBusStopIDkey,andthenassignedtothevariableaveragePop.Thetwodatapieces,theBusStopIDandtheaveratePopvariablearethenaddedtoalist,whichissuppliedtoaCSVwriterobject,whichknowshowtoacceptthedataandwriteittoafilelocatedatthefilepathsuppliedtothebuilt-inPythontheopen()function,usedtocreatesimplefiles.
Thescriptiscomplete,althoughitisnicetoaddonemorelineattheendtogiveusvisualconfirmationthatthescripthasrun:
print"DataAnalysisComplete"
Thiswillcreateanoutputindicatingthatthescripthasrun.Onceitisdone,gotothelocationoftheoutputcsvfileandopenit,usingExcelorNotepad,andseetheresultsoftheanalysis.Ourfirstscriptiscomplete!
ThefinalscriptHereishowthescriptshouldlookintheend:
#-*-coding:utf-8-*-
#-------------------------------------------------------------------------
--
#8662_Chapter3Model1.py
#Createdon:2014-04-2221:59:31.00000
#(generatedbyArcGIS/ModelBuilder)
#Description:
#-------------------------------------------------------------------------
--
#Importarcpymodule
importarcpy
importcsv
#Localvariables:
Bus_Stops=r"C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops"
CensusBlocks2010=r"C:\Projects\PacktDB.gdb\SanFrancisco\CensusBlocks2010"
Inbound71=r"C:\Projects\PacktDB.gdb\Chapter3Results\Inbound71"
Inbound71_400ft_buffer=
r"C:\Projects\PacktDB.gdb\Chapter3Results\Inbound71_400ft_buffer"
Intersect71Census=
r"C:\Projects\PacktDB.gdb\Chapter3Results\Intersect71Census"
#Process:Select
arcpy.Select_analysis(Bus_Stops,
Inbound71,
"NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'")
#Process:Buffer
arcpy.Buffer_analysis(Inbound71,
Inbound71_400ft_buffer,
"400Feet","FULL","ROUND","NONE","")
#Process:Intersect
arcpy.Intersect_analysis("{0}#;{1}
#".format(Inbound71_400ft_buffer,CensusBlocks2010),
Intersect71Census,"ALL","","INPUT")
dataDictionary={}
witharcpy.da.SearchCursor(Intersect71Census,["STOPID","POP10"])as
cursor:
forrowincursor:
busStopID=row[0]
pop10=row[1]
ifbusStopIDnotindataDictionary.keys():
dataDictionary[busStopID]=[pop10]
else:
dataDictionary[busStopID].append(pop10)
withopen(r'C:\Projects\Output\Averages2.csv','wb')ascsvfile:
spamwriter=csv.writer(csvfile,delimiter=',')
forbusStopIDindataDictionary.keys():
popList=dataDictionary[busStopID]
averagePop=sum(popList)/len(popList)
data=[busStopID,averagePop]
spamwriter.writerow(data)
print"DataAnalysisComplete"
SummaryInthischapter,wecoveredhowtocraftamodelofananalysisandexportittoascript.Afterdiscussingthescript,weadjustedthescripttoincludearesultsanalysisandsummation,whichwasoutputtedtoaCSVfile.Inparticular,wediscussedhowtouseModelBuildertocreateananalysisandexportitasascript,andhowtoadjustthescripttobemorePythonic.WealsobrieflytouchedontheuseofSearchCursors,whichwillbecoveredingreaterdetailinChapter5,ArcPyCursors–Search,Insert,andUpdate.Also,wesawhowbuilt-inmodulessuchastheCSVmodulecanbeusedalongwithArcPytocaptureanalysisoutputinformattedspreadsheets.
Inthenextchapter,wewilldiscusshowtocreatemorecomplexscriptsandbuildfunctionstoavoidrepeatingcode.Thesefunctionswillmakeitpossibletowritecodeonceanduseitforever.ThisreuseofcodewilldemonstratehowPythongoesbeyondautomationofanalysistobecomeanewproductivitytoolset.
Chapter4.ComplexArcPyScriptsandGeneralizingFunctionsInthischapter,wewillmovefromcreatingsimplescriptsbasedonautogeneratedscriptsfromModelBuildertocomplexscriptsthatincorporateadvancedPythonandArcPyconcepts,suchasfunctions.Functionscanimprovecodeandsavetimewhenwritingscripts.Theyarealsousefulwhencreatingmodulesorotherreusablecode,allowingforstandardprogrammingoperationstobescriptedandreadyforfutureuse.
Inthischapter,willcoverthefollowingtopics:
CreatingfunctionstoavoidrepeatingcodeCreatinghelperfunctionstoworkwithArcPylimitationsGeneralizingfunctionstomakethemreusable
Pythonfunctions–AvoidrepeatingcodeProgramminglanguagesshareaconceptthathasaidedprogrammersfordecades:functions.Theideaofafunction,looselyspeaking,istocreateblocksofcodethatwillperformanactiononapieceofdata,transformingitasrequiredbytheprogrammerandreturningthetransformeddatabacktothemainbodyofcode.We’vealreadybeenintroducedtosomeofPython’sbuilt-infunctionsinthelastfewchapters,theintfunction,forinstance,willconvertastringorafloatingnumberintoaninteger;nowit’stimetowriteourown.
Functionsareusedbecausetheysolvemanydifferentneedswithinprogramming.Functionsreducetheneedtowriterepetitivecode,whichinturnreducesthetimeneededtocreateascript.Theycanbeusedtocreaterangesofnumbers(therange()function),ortodeterminethemaximumvalueofalist(themaxfunction),ortocreateaSQLstatementtoselectasetofrowsfromafeatureclass.Theycanevenbecopiedandusedinanotherscriptorincludedaspartofamodulethatcanbeimportedintoscripts.Functionreusehastheaddedbonusofmakingprogrammingmoreusefulandlessofachore.Whenascripterstartswritingfunctions,itisamajorsteptowardsmakingprogrammingpartofaGISworkflow.
TechnicaldefinitionoffunctionsFunctions,alsocalledsubroutinesorproceduresinotherprogramminglanguages,areblocksofcodethathavebeendesignedtoeitheracceptinputdataandtransformit,orprovidedatatothemainprogramwhencalledwithoutanyinputrequired.Intheory,functionswillonlytransformdatathathasbeenprovidedtothefunctionasaparameter;itshouldnotchangeanyotherpartofthescriptthathasnotbeenincludedinthefunction.Tomakethispossible,theconceptofnamespacesisinvoked.AsdiscussedinChapter1,IntroductiontoPythonforArcGIS,namespacesareusedtoisolatevariableswithinascript;variablesareeitherglobal,andavailabletobeusedinthemainbodyofascriptaswellasinafunction,orarelocalandonlyavailablewithinafunction.
Namespacesmakeitpossibletouseavariablenamewithinafunction,andallowittorepresentavalue,whilealsousingthesamevariablenameinanotherpartofthescript.Thisbecomesespeciallyimportantwhenimportingmodulesfromotherprogrammers;withinthatmoduleanditsfunctions,thevariablesthatitcontainsmighthaveavariablenamethatisthesameasavariablenamewithinthemainscript.
Inahigh-levelprogramminglanguagesuchasPython,thereisbuilt-insupportforfunctions,includingtheabilitytodefinefunctionnamesandthedatainputs(alsoknownasparameters).Functionsarecreatedusingthekeyworddefplusafunctionname,alongwithparenthesesthatmayormaynotcontainparameters.Parameterscanalsobedefinedwithdefaultvalues,soparametersonlyneedtobepassedtothefunctionwhentheydifferfromthedefault.Thevaluesthatarereturnedfromthefunctionarealsoeasilydefined.
AfirstfunctionLet’screateafunctiontogetafeelforwhatispossiblewhenwritingfunctions.First,weneedtoinvokethefunctionbyprovidingthedefkeywordandprovidinganamealongwiththeparentheses.ThefirstFunction()willreturnastringwhencalled:
deffirstFunction():
'asimplefunctionreturningastring'
return"MyFirstFunction"
>>>firstFunction()
Theoutputisasfollows:
'MyFirstFunction'
Noticethatthisfunctionhasadocumentationstringordocstring(asimplefunctionreturningastring)thatdescribeswhatthefunctiondoes;thisstringcanbecalledlatertofindoutwhatthefunctiondoes,usingthe__doc__internalfunction:
>>>printfirstFunction.__doc__
Theoutputisasfollows:
'asimplefunctionreturningastring'
Thefunctionisdefinedandgivenaname,andthentheparenthesesareaddedfollowedbyacolon.Thefollowinglinesmustthenbeindented(agoodIDEwilladdtheindentionautomatically).Thefunctiondoesnothaveanyparameters,sotheparenthesesareempty.Thefunctionthenusesthekeywordreturntoreturnavalue,inthiscaseastring,fromthefunction.
Next,thefunctioniscalledbyaddingparenthesestothefunctionname.Whenitiscalled,itwilldowhatithasbeeninstructedtodo:returnthestring.
FunctionswithparametersNowlet’screateafunctionthatacceptsparametersandtransformsthemasneeded.Thisfunctionwillacceptanumberandmultiplyitby3:
defsecondFunction(number):
'thisfunctionmultiplesnumbersby3'
returnnumber*3
>>>secondFunction(4)
Theoutputisasfollows:
12
Thefunctionhasoneflaw,however;thereisnoassurancethatthevaluepassedtothefunctionisanumber.Weneedtoaddaconditionaltothefunctiontomakesureitdoesnotthrowanexception:
defsecondFunction(number):
'thisfunctionmultiplesnumbersby3'
iftype(number)==type(1)ortype(number)==type(1.0):
returnnumber*3
>>>secondFunction(4.0)
Theoutputisasfollows:
12.0
>>>secondFunction(4)
Theoutputisasfollows:
12
>>>secondFunction("String")
>>>
Thefunctionnowacceptsaparameter,checkswhattypeofdataitis,andreturnsamultipleoftheparameterwhetheritisanintegerorafunction.Ifitisastringorsomeotherdatatype,asshowninthelastexample,novalueisreturned.
Thereisonemoreadjustmenttothesimplefunctionthatweshoulddiscuss:parameterdefaults.Byincludingdefaultvaluesinthedefinitionofthefunction,weavoidhavingtoprovideparametersthatrarelychange.If,forinstance,wewantedadifferentmultiplierthan3inthesimplefunction,wewoulddefineitlikethis:
defthirdFunction(number,multiplier=3):
'thisfunctionmultiplesnumbersby3'
iftype(number)==type(1)ortype(number)==type(1.0):
returnnumber*multiplier
>>>thirdFunction(4)
Theoutputisasfollows:
12
>>>thirdFunction(4,5)
Theoutputisasfollows:
20
Thefunctionwillworkwhenonlythenumbertobemultipliedissupplied,asthemultiplierhasadefaultvalueof3.However,ifweneedanothermultiplier,thevaluecanbeadjustedbyaddinganothervaluewhencallingthefunction.Notethatthesecondvaluedoesn’thavetobeanumberasthereisnotypecheckingonit.Also,thedefaultvalue(s)inafunctionmustfollowtheparameterswithnodefaults(orallparameterscanhaveadefaultvalueandtheparameterscanbesuppliedtothefunctioninorderorbyname).
Thesesimplefunctionscombinemanyoftheconceptsthatwediscussedinearlierchapters,includingbuilt-infunctionssuchastype,conditionals,parameters,parameterdefaults,andfunctionreturns.WecannowmoveontocreatingfunctionswithArcPy.
UsingfunctionstoreplacerepetitivecodeOneofthemainusesoffunctionsistoensurethatthesamecodedoesnothavetobewrittenoverandover.Let’sreturntoourexamplefromthelastchapterandmakeafunctionfromthescripttomakeitpossibletoperformthesameanalysisforanybuslineinSanFrancisco.
ThefirstportionofthescriptthatwecouldconvertintoafunctionisthethreeArcPyfunctions.DoingsowillallowthescripttobeapplicabletoanyofthestopsintheBusStopfeatureclassandhaveanadjustablebufferdistance:
bufferDist=400
buffDistUnit="Feet"
lineName='71IB'
busSignage='FerryPlaza'
sqlStatement="NAME='{0}'ANDBUS_SIGNAG='{1}'"
defselectBufferIntersect(selectIn,selectOut,bufferOut,intersectIn,
intersectOut,sqlStatement,bufferDist,buffDistUnit,lineName,
busSignage):
'afunctiontoperformabusstopanalysis'
arcpy.Select_analysis(selectIn,selectOut,
sqlStatement.format(lineName,busSignage))
arcpy.Buffer_analysis(selectOut,bufferOut,"{0}
{1}".format(bufferDist),"FULL","ROUND","NONE","")
arcpy.Intersect_analysis("{0}#;{1}#".format(bufferOut,intersectIn),
intersectOut,"ALL","","INPUT")
returnintersectOut
Thisfunctiondemonstrateshowtheanalysiscanbeadjustedtoaccepttheinputandoutputfeatureclassvariablesasparameters,alongwithsomenewvariables.
ThefunctionaddsavariabletoreplacetheSQLstatementandvariablestoadjustthebusstop,andalsotweaksthebufferdistancestatementsothatboththedistanceandtheunitcanbeadjusted.Thefeatureclassnamevariables,definedearlierinthescript,haveallbeenreplacedwithlocalvariablenames;whiletheglobalvariablenamescouldhavebeenretained,itreducestheportabilityofthefunction.
ThenextfunctionwillaccepttheresultoftheselectBufferIntersect()functionandsearchitusingtheSearchCursor,passingtheresultsintoadictionary.Thedictionarywillthenbereturnedfromthefunctionforlateruse:
defcreateResultDic(resultFC):
'searchresultsofanalysisandcreateresultsdictionary'
dataDictionary={}
witharcpy.da.SearchCursor(resultFC,["STOPID","POP10"])ascursor:
forrowincursor:
busStopID=row[0]
pop10=row[1]
ifbusStopIDnotindataDictionary.keys():
dataDictionary[busStopID]=[pop10]
else:
dataDictionary[busStopID].append(pop10)
returndataDictionary
Thisfunctiononlyrequiresoneparameter:thefeatureclassreturnedfromthesearchBufferIntersect()function.Theresultsholdingdictionaryisfirstcreated,thenpopulatedbythesearchcursor,withthebusStopidattributeusedasakey,andthecensusblockpopulationattributeaddedtoalistassignedtothekey.
Thedictionary,havingbeenpopulatedwithsorteddata,isreturnedfromthefunctionforuseinthefinalfunction,createCSV().ThisfunctionacceptsthedictionaryandthenameoftheoutputCSVfileasastring:
defcreateCSV(dictionary,csvname):
'afunctiontakesadictionaryandcreatesaCSVfile'
withopen(csvname,'wb')ascsvfile:
csvwriter=csv.writer(csvfile,delimiter=',')
forbusStopIDindictionary.keys():
popList=dictionary[busStopID]
averagePop=sum(popList)/len(popList)
data=[busStopID,averagePop]
csvwriter.writerow(data)
ThefinalfunctioncreatestheCSVusingthecsvmodule.Thenameofthefile,astring,isnowacustomizableparameter(meaningthescriptnamecanbeanyvalidfilepathandtextfilewiththeextension.csv).ThecsvfileparameterispassedtotheCSVmodule’swritermethodandassignedtothevariablecsvwriter,andthedictionaryisaccessedandprocessed,andpassedasalisttocsvwritertobewrittentotheCSVfile.Thecsv.writer()methodprocesseseachiteminthelistintotheCSVformatandsavesthefinalresult.OpentheCSVfilewithExceloratexteditorsuchasNotepad.
Torunthefunctions,wewillcalltheminthescriptfollowingthefunctiondefinitions:
analysisResult=selectBufferIntersect(Bus_Stops,Inbound71,
Inbound71_400ft_buffer,CensusBlocks2010,Intersect71Census,bufferDist,
lineName,busSignage)
dictionary=createResultDic(analysisResult)
createCSV(dictionary,r'C:\Projects\Output\Averages.csv')
Now,thescripthasbeendividedintothreefunctions,whichreplacethecodeofthefirstmodifiedscript.Themodifiedscriptlookslikethis:
#-*-coding:utf-8-*-
#-------------------------------------------------------------------------
--
#8662_Chapter4Modified1.py
#Createdon:2014-04-2221:59:31.00000
#(generatedbyArcGIS/ModelBuilder)
#Description:
#AdjustedbySilasToms
#20140505
#-------------------------------------------------------------------------
--
#Importarcpymodule
importarcpy
importcsv
#Localvariables:
Bus_Stops=r"C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops"
CensusBlocks2010=r"C:\Projects\PacktDB.gdb\SanFrancisco\CensusBlocks2010"
Inbound71=r"C:\Projects\PacktDB.gdb\Chapter3Results\Inbound71"
Inbound71_400ft_buffer=
r"C:\Projects\PacktDB.gdb\Chapter3Results\Inbound71_400ft_buffer"
Intersect71Census=
r"C:\Projects\PacktDB.gdb\Chapter3Results\Intersect71Census"
bufferDist=400
lineName='71IB'
busSignage='FerryPlaza'
defselectBufferIntersect(selectIn,selectOut,bufferOut,intersectIn,
intersectOut,bufferDist,lineName,busSignage):
arcpy.Select_analysis(selectIn,
selectOut,
"NAME='{0}'ANDBUS_SIGNAG=
'{1}'".format(lineName,busSignage))
arcpy.Buffer_analysis(selectOut,
bufferOut,
"{0}Feet".format(bufferDist),
"FULL","ROUND","NONE","")
arcpy.Intersect_analysis("{0}#;{1}#".format(bufferOut,intersectIn),
intersectOut,"ALL","","INPUT")
returnintersectOut
defcreateResultDic(resultFC):
dataDictionary={}
witharcpy.da.SearchCursor(resultFC,
["STOPID","POP10"])ascursor:
forrowincursor:
busStopID=row[0]
pop10=row[1]
ifbusStopIDnotindataDictionary.keys():
dataDictionary[busStopID]=[pop10]
else:
dataDictionary[busStopID].append(pop10)
returndataDictionary
defcreateCSV(dictionary,csvname):
withopen(csvname,'wb')ascsvfile:
csvwriter=csv.writer(csvfile,delimiter=',')
forbusStopIDindictionary.keys():
popList=dictionary[busStopID]
averagePop=sum(popList)/len(popList)
data=[busStopID,averagePop]
csvwriter.writerow(data)
analysisResult=selectBufferIntersect(Bus_Stops,Inbound71,
Inbound71_400ft_buffer,CensusBlocks2010,Intersect71Census,
bufferDist,lineName,busSignage)
dictionary=createResultDic(analysisResult)
createCSV(dictionary,r'C:\Projects\Output\Averages.csv')
print"DataAnalysisComplete"
Furthergeneralizationofthefunctions,whilewehavecreatedfunctionsfromtheoriginalscriptthatcanbeusedtoextractmoredataaboutbusstopsinSanFrancisco,ournew
functionsarestillveryspecifictothedatasetandanalysisforwhichtheywerecreated.Thiscanbeveryusefulforlongandlaboriousanalysisforwhichcreatingreusablefunctionsisnotnecessary.Thefirstuseoffunctionsistogetridoftheneedtorepeatcode.Thenextgoalistothenmakethatcodereusable.Let’sdiscusssomewaysinwhichwecanconvertthefunctionsfromone-offsintoreusablefunctionsorevenmodules.
First,let’sexaminethefirstfunction:
defselectBufferIntersect(selectIn,selectOut,bufferOut,intersectIn,
intersectOut,bufferDist,lineName,busSignage):
arcpy.Select_analysis(selectIn,
selectOut,
"NAME='{0}'ANDBUS_SIGNAG=
'{1}'".format(lineName,busSignage))
arcpy.Buffer_analysis(selectOut,
bufferOut,
"{0}Feet".format(bufferDist),
"FULL","ROUND","NONE","")
arcpy.Intersect_analysis("{0}#;{1}#".format(bufferOut,intersectIn),
intersectOut,"ALL","","INPUT")
returnintersectOut
Thisfunctionappearstobeprettyspecifictothebusstopanalysis.It’ssospecific,infact,thatwhilethereareafewwaysinwhichwecantweakittomakeitmoregeneral(thatis,usefulinotherscriptsthatmightnothavethesamestepsinvolved),weshouldnotconvertitintoaseparatefunction.Whenwecreateaseparatefunction,weintroducetoomanyvariablesintothescriptinanefforttosimplifyit,whichisacounterproductiveeffort.Instead,let’sfocusonwaystogeneralizetheArcPytoolsthemselves.
ThefirststepwillbetosplitthethreeArcPytoolsandexaminewhatcanbeadjustedwitheachofthem.TheSelecttoolshouldbeadjustedtoacceptastringastheSQLselectstatement.TheSQLstatementcanthenbegeneratedbyanotherfunctionorbyparametersacceptedatruntime(forexample,passedtothescriptbyaScripttool,whichwillbediscussedinalaterchapter).
Forinstance,ifwewantedtomakethescriptacceptmultiplebusstopsforeachrunofthescript(forexample,theinboundandoutboundstopsforeachline),wecouldcreateafunctionthatwouldacceptalistofthedesiredstopsandaSQLtemplate,andwouldreturnaSQLstatementtoplugintotheSelecttool.Hereisanexampleofhowitwouldlook:
defformatSQLIN(dataList,sqlTemplate):
'afunctiontogenerateaSQLstatement'
sql=sqlTemplate#"OBJECTIDIN"
step="("
fordataindataList:
step+=str(data)
sql+=step+")"
returnsql
defformatSQL(dataList,sqlTemplate):
'afunctiontogenerateaSQLstatement'
sql=''
forcount,datainenumerate(dataList):
ifcount!=len(dataList)-1:
sql+=sqlTemplate.format(data)+'OR'
else:
sql+=sqlTemplate.format(data)
returnsql
>>>dataVals=[1,2,3,4]
>>>sqlOID="OBJECTID={0}"
>>>sql=formatSQL(dataVals,sqlOID)
>>>printsql
Theoutputisasfollows:
OBJECTID=1OROBJECTID=2OROBJECTID=3OROBJECTID=4
Thisnewfunction,formatSQL(),isaveryusefulfunction.Let’sreviewwhatitdoesbycomparingthefunctiontotheresultsfollowingit.Thefunctionisdefinedtoaccepttwoparameters:alistofvaluesandaSQLtemplate.Thefirstlocalvariableistheemptystringsql,whichwillbeaddedtousingstringaddition.Thefunctionisdesignedtoinsertthevaluesintothevariablesql,creatingaSQLstatementbytakingtheSQLtemplateandusingstringformattingtoaddthemtothetemplate,whichinturnisaddedtotheSQLstatementstring(notethatsql+=isequivelenttosql=sql+).Also,anoperator(OR)isusedtomaketheSQLstatementinclusiveofalldatarowsthatmatchthepattern.Thisfunctionusesthebuilt-inenumeratefunctiontocounttheiterationsofthelist;onceithasreachedthelastvalueinthelist,theoperatorisnotaddedtotheSQLstatement.
NotethatwecouldalsoaddonemoreparametertothefunctiontomakeitpossibletouseanANDoperatorinsteadofOR,whilestillkeepingORasthedefault:
defformatSQL2(dataList,sqlTemplate,operator="OR"):
'afunctiontogenerateaSQLstatement'
sql=''
forcount,datainenumerate(dataList):
ifcount!=len(dataList)-1:
sql+=sqlTemplate.format(data)+operator
else:
sql+=sqlTemplate.format(data)
returnsql
>>>sql=formatSQL2(dataVals,sqlOID,"AND")
>>>printsql
Theoutputisasfollows:
OBJECTID=1ANDOBJECTID=2ANDOBJECTID=3ANDOBJECTID=4
WhileitwouldmakenosensetouseanANDoperatoronObjectIDs,thereareothercaseswhereitwouldmakesense,henceleavingORasthedefaultwhileallowingforAND.Eitherway,thisfunctioncannowbeusedtogenerateourbusstopSQLstatementformultiplestops(ignoring,fornow,thebussignagefield):
>>>sqlTemplate="NAME='{0}'"
>>>lineNames=['71IB','71OB']
>>>sql=formatSQL2(lineNames,sqlTemplate)
>>>printsql
Theoutputisasfollows:
NAME='71IB'ORNAME='71OB'
However,wecan’tignoretheBusSignagefieldfortheinboundline,astherearetwostartingpointsfortheline,sowewillneedtoadjustthefunctiontoacceptmultiplevalues:
defformatSQLMultiple(dataList,sqlTemplate,operator="OR"):
'afunctiontogenerateaSQLstatement'
sql=''
forcount,datainenumerate(dataList):
ifcount!=len(dataList)-1:
sql+=sqlTemplate.format(*data)+operator
else:
sql+=sqlTemplate.format(*data)
returnsql
>>>sqlTemplate="(NAME='{0}'ANDBUS_SIGNAG='{1}')"
>>>lineNames=[('71IB','FerryPlaza'),('71OB','48thAvenue')]
>>>sql=formatSQLMultiple(lineNames,sqlTemplate)
>>>printsql
Theoutputisasfollows:
(NAME='71IB'ANDBUS_SIGNAG='FerryPlaza')OR(NAME='71OB'AND
BUS_SIGNAG='48thAvenue')
Theslightdifferenceinthisfunction,theasteriskbeforethedatavariable,allowsthevaluesinsidethedatavariabletobecorrectlyformattedintotheSQLtemplatebyexplodingthevalueswithinthetuple.NoticethattheSQLtemplatehasbeencreatedtosegregateeachconditionalbyusingparentheses.Thefunction(s)arenowreadyforreuse,andtheSQLstatementisnowreadyforinsertionintotheSelecttool:
sql=formatSQLMultiple(lineNames,sqlTemplate)
arcpy.Select_analysis(Bus_Stops,Inbound71,sql)
NextupistheBuffertool.Wehavealreadytakenstepstowardsmakingitgeneralizedbyaddingavariableforthedistance.Inthiscase,wewillonlyaddonemorevariabletoit,aunitvariablethatwillmakeitpossibletoadjustthebufferunitfromfeettometeroranyotherallowedunit.Wewillleavetheotherdefaultsalone.
HereisanadjustedversionoftheBuffertool:
bufferDist=400
bufferUnit="Feet"
arcpy.Buffer_analysis(Inbound71,
Inbound71_400ft_buffer,
"{0}{1}".format(bufferDist,bufferUnit),
"FULL","ROUND","NONE","")
Now,boththebufferdistanceandbufferunitarecontrolledbyavariabledefinedinthepreviousscript,andthiswillmakeiteasilyadjustableifitisdecidedthatthedistancewasnotsufficientandthevariablesmightneedtobeadjusted.
ThenextsteptowardsadjustingtheArcPytoolsistowriteafunction,whichwillallowforanynumberoffeatureclassestobeintersectedtogetherusingtheIntersecttool.ThisnewfunctionwillbesimilartotheformatSQLfunctionsasprevious,astheywillusestringformattingandadditiontoallowforalistoffeatureclassestobeprocessedintothecorrectstringformatfortheIntersecttooltoacceptthem.However,asthisfunctionwillbebuilttobeasgeneralaspossible,itmustbedesignedtoacceptanynumberoffeatureclassestobeintersected:
defformatIntersect(features):
'afunctiontogenerateanintersectstring'
formatString=''
forcount,featureinenumerate(features):
ifcount!=len(features)-1:
formatString+=feature+"#;"
else:
formatString+=feature+"#"
returnformatString
>>>shpNames=["example.shp","example2.shp"]
>>>iString=formatIntersect(shpNames)
>>>printiString
Theoutputisasfollows:
example.shp#;example2.shp#
NowthatwehavewrittentheformatIntersect()function,allthatneedstobecreatedisalistofthefeatureclassestobepassedtothefunction.ThestringreturnedbythefunctioncanthenbepassedtotheIntersecttool:
intersected=[Inbound71_400ft_buffer,CensusBlocks2010]
iString=formatIntersect(intersected)
#Process:Intersect
arcpy.Intersect_analysis(iString,
Intersect71Census,"ALL","","INPUT")
Becauseweavoidedcreatingafunctionthatonlyfitsthisscriptoranalysis,wenowhavetwo(ormore)usefulfunctionsthatcanbeappliedinlateranalyses,andweknowhowtomanipulatetheArcPytoolstoacceptthedatathatwewanttosupplytothem.
MoregeneralizationofthefunctionsTheotherfunctionsthatweinitiallycreatedtosearchtheresults,andgeneratethespreadsheetofresults,canalsobemanipulatedintobeingmoregeneralizedwithafewtweaks.
Ifwewanttogeneratemoreinformationabouteachcensusblockwithinadistancetoabusstop(forexample,ifwehadacensusblockdatasetwithincomedataaswellaspopulationdata),wewouldpasstothefunctionalistofattributestobeextractedfromthefinalfeatureclass.Tomakethispossible,itwouldbenecessarytoadjustthecreateResultDic()functiontoacceptthislistofattributes:
defcreateResultDic(resultFC,key,values):
dataDictionary={}
fields=[key]
fields.extend(values)
witharcpy.da.SearchCursor(resultFC,fields)ascursor:
forrowincursor:
busStopID=row[0]
data=row[1:]
ifbusStopIDnotindataDictionary.keys():
dataDictionary[busStopID]=[data]
else:
dataDictionary[busStopID].append(data)
returndataDictionary
ThisnewversionofthecreateResultDic()functionwillgeneratealistoflists(thatis,thevaluesfromeachrowarecontainedwithinalistandareaddedtoamasterlist)foreachbusstop,whichcanthenbeparsedlaterbyknowingthepositionofeachvalueinthelist.Thissolutionisusefulwhenneedingtosortdataintoadictionary.
However,thisisanunsatisfactorywaytosorttheresults.Whatifthelistoffieldsisnotpassedontothedictionaryandthereisnowayofknowingtheorderofthedatainthelists?Instead,wewanttobeabletousethefunctionalityofPythondictionariestosortthedatabyfieldname.Inthiscase,wewillusenesteddictionariestocreatelistsofresultsaccessiblebythetypeofdatatheycontain(thatis,population,income,oranotherfield):
defcreateResultDic(resultFC,key,values):
dataDic={}
fields=[]
iftype(key)==type((1,2))ortype(key)==type([1,2]):
fields.extend(key)
length=len(key)
else:
fields=[key]
length=1
fields.extend(values)
witharcpy.da.SearchCursor(resultFC,fields)ascursor:
forrowincursor:
busStopID=row[:length]
data=row[length:]
ifbusStopIDnotindataDictionary.keys():
dataDictionary[busStopID]={}
forcounter,fieldinenumerate(values):
iffieldnotindataDictionary[busStopID].keys():
dataDictionary[busStopID][field]=[data[counter]]
else:
dataDictionary[busStopID][field].append(data[counter])
returndataDictionary
>>>rFC=r'C:\Projects\PacktDB.gdb\Chapter3Results\Intersect71Census'
>>>key='STOPID'
>>>values='HOUSING10','POP10'
>>>dic=createResultDic(rFC,key,values)
>>>dic[1122023]
Theoutputisasfollows:
{'HOUSING10':[104,62,113,81,177,0,52,113,0,104,81,177,52],
'POP10':[140,134,241,138,329,0,118,241,0,140,138,329,118]}
Inthisexample,thefunctionispassedasparameterstoafeatureclass,theSTOPID,andthefieldstobeconglomerated.ThefieldsvariableiscreatedtopasstherequiredfieldsontotheSearchCursor.Thecursorreturnseachrowasatuple;thefirstmemberofthetupleisbusStopID,andtherestofthetupleisthedataassociatedwiththatbusstop.Thefunctionthenusesaconditiontoassesswhetherthebusstophasbeenpreviouslyanalyzed;ifnot,itisaddedtothedictionaryandassignedasecondinternaldictionary,whichwillbeusedtostoretheresultsassociatedwiththatstop.Byusingadictionary,wecanthensortthroughtheresultsandassignthemtothecorrectfieldtowhichtheybelong.
Thepreviousexampleshowstheresultsofrequestingdataforoneparticularbusstop(1122023).Astherearetwofieldspassedhere,thedatahasbeenorganizedintotwosets,andthefieldnamesarenowkeysfortheinternaldictionary.Becauseofthisorganization,wecannowcreateaveragesforeachfieldinsteadofjustone.
Speakingofaverages,weleftthejobofaveragingtheresultsofthesearchcursoranalysistothecreateCSV()function.Thisshouldalsobeavoided,asitreducestheusefulnessofthecreateCSV()functionbyaddingadditionaldatamanipulationdutiesthatshouldbetheresponsibilityofanotherfunction.Let’saddressthisissuebyadjustingthecreateCSV()functionfirst:
defcreateCSV(data,csvname,mode='ab'):
withopen(csvname,mode)ascsvfile:
csvwriter=csv.writer(csvfile,delimiter=',')
csvwriter.writerow(data)
Thisisastrippeddownversionofthefunction,butitisinfinitelymoreuseful.Byadjustingthefunctionlikethis,wearelimitingittoonlydoingtwothings:openingtheCSVfileandaddingarowofdatatoit.Becauseweusedtheabmode,iftheCSVfileexists,wewillonlybeaddingdatatoitinsteadofwritingoverit(ifitdoesn’texist,itwillbecreated).Thisaddingmodecanbeoverriddenbypassingwbasthemode,whichwillgenerateanewscripteachtime.
Nowwecansortthroughtheresultsoftheanalysis,averagethem,andpassthemtoour
newcreateCSVscript.Todothis,wewilliteratethroughthedictionarycreatedbythecreateResultDic()function:
csvname=r'C:\Projects\Output\Averages.csv'
dataKey='STOPID'
fields='HOUSING10','POP10'
dictionary=createResultDic(Intersect71Census,dataKey,fields)
header=[dataKey]
forfieldinfields:
header.append(field)
createCSV(header,csvname,'wb')
forcounter,busStopinenumerate(dictionary.keys()):
datakeys=dictionary[busStop]
averages=[busStop]
forkeyindatakeys:
data=datakeys[key]
average=sum(data)/len(data)
averages.append(average)
createCSV(averages,csvname)
ThislaststepshowshowtheCSVfileiscreated:byiteratingthroughthedatacontainedinthedictionaryandthenaveragingthevaluesforeachbusstop.Then,theseaveragesareaddedtoalistthatcontainsthenameofeachbusstop(andthelineitbelongstointhisinstance)andpassedtothecreateCSV()functiontobewrittenintotheCSVfile.
Hereisthefinalcode.NotethatIhaveconvertedmanyoftheautogeneratedcommentsintoprintstatementstogivesomefeedbackonthestateofthescript:
#-*-coding:utf-8-*-
#-------------------------------------------------------------------------
--
#8662_Chapter4Modified2.py
#Createdon:2014-04-2221:59:31.00000
#(generatedbyArcGIS/ModelBuilder)
#Description:
#AdjustedbySilasToms
#20140423
#-------------------------------------------------------------------------
--
#Importarcpymodule
importarcpy
importcsv
Bus_Stops=r"C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops"
CensusBlocks2010=r"C:\Projects\PacktDB.gdb\SanFrancisco\CensusBlocks2010"
Inbound71=r"C:\Projects\PacktDB.gdb\Chapter4Results\Inbound71"
Inbound71_400ft_buffer=
r"C:\Projects\PacktDB.gdb\Chapter4Results\Inbound71_400ft_buffer"
Intersect71Census=
r"C:\Projects\PacktDB.gdb\Chapter4Results\Intersect71Census"
bufferDist=400
bufferUnit="Feet"
lineNames=[('71IB','FerryPlaza'),('71OB','48thAvenue')]
sqlTemplate="NAME='{0}'ANDBUS_SIGNAG='{1}'"
intersected=[Inbound71_400ft_buffer,CensusBlocks2010]
dataKey='NAME','STOPID'
fields='HOUSING10','POP10'
csvname=r'C:\Projects\Output\Averages.csv'
defformatSQLMultiple(dataList,sqlTemplate,operator="OR"):
'afunctiontogenerateaSQLstatement'
sql=''
forcount,datainenumerate(dataList):
ifcount!=len(dataList)-1:
sql+=sqlTemplate.format(*data)+operator
else:
sql+=sqlTemplate.format(*data)
returnsql
defformatIntersect(features):
'afunctiontogenerateanintersectstring'
formatString=''
forcount,featureinenumerate(features):
ifcount!=len(features)-1:
formatString+=feature+"#;"
else:
formatString+=feature+"#"
returnformatString
defcreateResultDic(resultFC,key,values):
dataDictionary={}
fields=[]
iftype(key)==type((1,2))ortype(key)==type([1,2]):
fields.extend(key)
length=len(key)
else:
fields=[key]
length=1
fields.extend(values)
witharcpy.da.SearchCursor(resultFC,fields)ascursor:
forrowincursor:
busStopID=row[:length]
data=row[length:]
ifbusStopIDnotindataDictionary.keys():
dataDictionary[busStopID]={}
forcounter,fieldinenumerate(values):
iffieldnotindataDictionary[busStopID].keys():
dataDictionary[busStopID][field]=[data[counter]]
else:
dataDictionary[busStopID][field].append(data[counter])
returndataDictionary
defcreateCSV(data,csvname,mode='ab'):
withopen(csvname,mode)ascsvfile:
csvwriter=csv.writer(csvfile,delimiter=',')
csvwriter.writerow(data)
sql=formatSQLMultiple(lineNames,sqlTemplate)
print'Process:Select'
arcpy.Select_analysis(Bus_Stops,
Inbound71,
sql)
print'Process:Buffer'
arcpy.Buffer_analysis(Inbound71,
Inbound71_400ft_buffer,
"{0}{1}".format(bufferDist,bufferUnit),
"FULL","ROUND","NONE","")
iString=formatIntersect(intersected)
printiString
print'Process:Intersect'
arcpy.Intersect_analysis(iString,
Intersect71Census,"ALL","","INPUT")
print'ProcessResults'
dictionary=createResultDic(Intersect71Census,dataKey,fields)
print'CreateCSV'
header=[dataKey]
forfieldinfields:
header.append(field)
createCSV(header,csvname,'wb')
forcounter,busStopinenumerate(dictionary.keys()):
datakeys=dictionary[busStop]
averages=[busStop]
forkeyindatakeys:
data=datakeys[key]
average=sum(data)/len(data)
averages.append(average)
createCSV(averages,csvname)
print"DataAnalysisComplete"
SummaryInthischapter,wediscussedhowtotakeautogeneratedcodeandmakeitgeneralized,whileaddingfunctionsthatcanbereusedinotherscriptsandwillmakethegenerationofthenecessarycodecomponents,suchasSQLstatements,mucheasier.Wealsoaddressedwhenitisbestnottogotoofarwiththecreationoffunctionstoavoidmakingthemtoospecific.
Inthenextchapter,wewillinvestigatethepowerfulDataAccessmoduleanditsSearchCursors,UpdateCursors,andInsertCursors.
Chapter5.ArcPyCursors–Search,Insert,andUpdateNowthatweunderstandhowtointeractwithArcToolboxtoolsusingArcPy,andwehavealsocoveredusingPythontocreatefunctionsandimportmodules,wehaveabasicunderstandingofhowtoimproveGISworkflowsusingPython.InthischapterwewillcoverdatacursorsandtheDataAccessmodule,introducedin10.1.Thesedataaccesscursorsareavastimprovementonthecursorsusedinthearcgisscriptingmodule(theprecursortoArcPy)andinearlierversionsofArcPy.Notonlycanthecursorssearchdata,aswehaveseen,buttheycanupdatedatausingtheUpdateCursorsandcanaddnewrowsofdatausingtheInsertCursor.
Datacursorsareusedtoaccessdatarecordscontainedwithindatatables,usingarowbyrowiterativeapproach.Theconceptwasborrowedfromrelationaldatabases,wheredatacursorsareusedtoextractdatafromtablesreturnedfromaSQLexpression.Cursorsareusedtosearchfordata,butalsotoupdatedataortoaddnewdata.
WhenwediscusscreatingdatasearchesusingArcPycursors,wearenotjusttalkingaboutattributeinformation.Thenewdataaccessmodelcursorscaninteractdirectlywiththeshapefield,andwhencombinedwithArcPyGeometryobjects,canperformgeospatialfunctionsandreplacetheneedtopassdatatoArcToolboxtools.DataaccesscursorsrepresentthemostusefulinnovationyetintherealmofPythonautomationforGIS.
Inthischapterwewillcover:
UsingSearchCursorstoaccessattributeandspatialdataUsingUpdateCursorstoadjustvalueswithinrowsUsinginsertcursorstoaddnewdatatoadatasetUsingcursorsandtheArcPyGeometryobjecttypestoperformgeospatialanalysesinmemory
ThedataaccessmoduleIntroducedwiththereleaseofArcGIS10.1,thenewdataaccessmoduleknownasarcpy.dahasmadedatainteractioneasier,andfaster,thanallowedbypreviousdatacursors.Byallowingfordirectaccesstotheshapefieldinavarietyofforms(shapeobject,Xvalues,Yvalues,centroid,area,length,andmore),andavarietyofformats(JavaScriptObjectNotation(JSON),KeyholeMarkupLanguage(KML),WellKnownBinary(WKB),Well-KnownText(WKT)),thedataaccessmodulegreatlyincreasestheabilityofaGISanalysttoextractandcontrolshapefielddata.
Thedataaccesscursorsacceptanumberofrequiredandoptionalparameters.Therequiredparametersarethepathtothefeatureclassasastring(oravariablerepresentingthepath)andthefieldstobereturned.Ifallfieldsaredesired,usingtheasterisknotationandprovidealistwithanasteriskasastringasthefield’sparameter([*]).Ifonlyafewfieldsarerequired,providethosefieldsasstringfieldnames(forexample[“NAME”,“DATE”]).
Theotherparametersareoptionalbutareveryimportant,forbothsearchandUpdateCursors.AwhereclauseintheformofaSQLexpressioncanbeprovidednext;thisclausewilllimitthenumberofrowsreturnedfromthedataset(asdemonstratedbytheSQLexpressioninthescriptsinthelastchapter).TheSQLexpressionsusedbythesearchandupdatecursorsarenotcompleteSQLexpressions,astheSELECTorUPDATEcommandsareprovidedautomaticallybythechoiceofcursor.OnlythewhereclauseoftheSQLexpressionisrequiredforthisparameter.
AspatialreferencecanbeprovidednextintheArcPySpatialReferenceformat;thisisnotnecessaryifthedataisinthecorrectformatbutcanbeusedtotransformdataintoanotherprojectiononthefly.Thereisnowaytospecifythespatialtransformationused,however.ThethirdoptionalparameterisaBoolean(orTrue/False)valuethatdeclareswhetherdatashouldbereturnedinexplodedpoints(thatis,alistoftheindividualvertices)orintheoriginalgeometryformat.Thefinaloptionalparameterisanotherlistthatcanbeusedtoorganizethedatareturnedbythecursor;thislistwouldincludeSQLkeywordssuchasDISTINCT,ORBERBY,orGROUPBY.However,thisfinalparameterisonlyavailablewhenworkingwithageodatabase.
Let’stakealookatusingarcpy.da.SearchCursorforshapefieldinteractions.Ifweneededtoproduceaspreadsheetlistingallbusstopsalongaparticularroute,andincludethelocationofthedatainanX/Yformat,wecouldusetheAddXYtoolfromtheArcToolbox.However,thishastheeffectofaddingtwonewfieldstoourdata,whichisnotalwaysallowed,especiallywhenthedataisstoredinenterprisegeodatabaseswithfixedschemas.Instead,we’llusetheSHAPE@XYtokenbuiltintothedataaccessmoduletoeasilyextractthedataandpassittothecreateCSV()functionfromChapter4,ComplexArcPyScriptsandGeneralizingFunctions,alongwiththeSQLexpressionlimitingresultstothestopsofinterest:
csvname="C:\Projects\Output\StationLocations.csv"
headers='BusLineName','BusStopID','X','Y'
createCSV(headers,csvname,'wb')
sql="(NAME='71IB'ANDBUS_SIGNAG='FerryPlaza')OR(NAME='71OB'
ANDBUS_SIGNAG='48thAvenue')"
witharcpy.da.SearchCursor(Bus_Stops,['NAME','STOPID','SHAPE@XY'],sql)
ascursor:
forrowincursor:
linename=row[0]
stopid=row[1]
locationX=row[2][0]
locationY=row[2][1]
locationY=row[2][1]
data=linename,stopid,locationX,locationY
createCSV(data,csvname)
Notethateachrowofdataisreturnedasatuple;thismakessenseastheSearchCursordoesnotallowanydatamanipulationandtuplesareimmutableassoonastheyarecreated.Incontrast,datareturnedfromUpdateCursorsisinlistformat,aslistscanbeupdated.Bothcanbeaccessedusingtheindexingasshownpreviously.
Eachrowreturnedbythecursorisatuplewiththreeobjects:thenameofthebusstop,thebusstopID,andfinallyanothertuplecontainingtheX/Ylocationofthestop.Theobjectsinthetuple,containedinthevariablerow,areaccessibleusingindexing:thebusstopnameisatindex0,theIDisatindex1,andthelocationtupleisatindex2.
Withinthelocationtuple,theXvalueisatindex0andtheYvalueisatindex1;thismakesiteasytoaccessthedatainthelocationtuplebypassingavalueasshowninthefollowing:
locationX=row[2][0]
TheabilitytoaddlistsandtuplesandevendictionariestoanotherlistortupleordictionaryisastrongcomponentofPython,makingdataaccesslogicalanddataorganizationeasy.
However,thespreadsheetreturnedfromthepreviouscodehasafewissues:thelocationisreturnedinthenativeprojectionofthefeatureclass(inthiscase,aStatePlaneprojection),andtherearerowsofdatathatarerepeated.Itwouldbemuchmorehelpfulifwecouldprovidelatitudeandlongitudevaluesinthespreadsheetandtheduplicatevalueswereremoved.Let’susetheoptionalspatialreferenceparameterandalisttosortthedatabeforewepassittothecreateCSV()function:
spatialReference=arcpy.SpatialReference(4326)
sql="(NAME='71IB'ANDBUS_SIGNAG='FerryPlaza')OR(NAME='71OB'
ANDBUS_SIGNAG='48thAvenue')"
dataList=[]
witharcpy.da.SearchCursor(Bus_Stops,['NAME','STOPID','SHAPE@XY'],sql,
spatialReference)ascursor:
forrowincursor:
linename=row[0]
stopid=row[1]
locationX=row[2][0]
locationY=row[2][1]
data=linename,stopid,locationX,locationY
ifdatanotindataList:
dataList.append(data)
csvname="C:\Projects\Output\StationLocations.csv"
headers='BusLineName','BusStopID','X','Y'
createCSV(headers,csvname,'wb')
fordataindataList:
Thespatialreferenceiscreatedbypassingacoderepresentingthedesiredprojectionsystem.InthiscasethecodefortheWGS1984LatitudeandLongitudegeographicsystemis4326andispassedtothearcpy.SpatialReference()methodtocreateaspatialreferenceobjectthatcanbepassedtotheSearchCursor.Also,theifconditionalisusedtofilterthedata,acceptingonlyonelistperstopintothelistcalleddataList.ThisnewversionofthecodewillproduceaCSVfilewiththedesireddata.ThisCSVcouldthenbeconvertedintoaKMLwiththeserviceprovidedbywww.convertcsv.com/csv-to-kml.htm,orevenbetter,usingPython.Usestringformattingandloopstoinsertthedataintopre-builtKMLstrings.
AttributefieldinteractionsApartfromtheshapefieldinteractions,anotherimprovementofferedbythedataaccessmodulecursorsistheabilitytocallthefieldsinafeatureclassbyusingalist,asdiscussedpreviously.Earlierdatacursorsrequiredtheuseofalessefficientgetvaluefunctioncall,orrequiredthefieldstobecalledasiftheyweremethodsavailabletothefunction.Thenewmethodallowsforallfieldstobecalledbypassinganasterisk,avaluablemethodtoaccessfieldsinfeatureclassesthathavenotbeeninspectedpreviously.
OneofthemorevaluableimprovementsistheabilitytoaccesstheUniqueIDfieldwithoutneedingtoknowwhetherthedatasetisafeatureclassorashapefile.BecauseshapefileshadafeatureIDorFID,andfeatureclasseshadanobjectID,itwashardertoprogramaScripttooltoaccesstheuniqueIDfield.DataaccessmodulecursorsallowfortheuseoftheOID@stringtorequesttheuniqueIDfromeithertypeofinput.ThismakestheneedtoknowthetypeofuniqueIDirrelevant.
Asdemonstratedpreviously,otherattributefieldsarerequestedbyastringinalist.Thefieldnamesmustmatchthetruenameofthefield;aliasnamescannotbepassedtothecursor.Thefieldscanbeinthelistinanyorderdesired,andwillbereturnedintheorderrequested.Onlytherequiredfieldshavetobeincludedinthelist.
Hereisademonstrationofrequestingfieldinformation:
sql="OBJECTID=1"
witharcpy.da.SearchCursor(Bus_Stops,
['STOPID','NAME','OID@'],
sql)ascursor:
forrowincursor:
Ifthefieldsinthefieldslistwereadjusted,thedataintheresultingrowwouldreflecttheadjustment.Also,allofthemembersofthetuplereturnedbythecursorareaccessiblebyzero-basedindexing.
UpdatecursorsUpdatecursorsareusedtoadjustdatawithinexistingrowsofdata.Updatesbecomeveryimportantwhencalculatingdataorconvertingnullvaluestoanon-nullvalue.CombinedwithspecificSQLexpressions,datacanbetargetedforupdatingwithnewlycollectedorcalculatedvalues.
NotethatrunningcodecontaininganUpdateCursorwillchange,orupdate,thedataonwhichitoperates.Itisagoodideatomakeacopyofthedatatotestoutthecodebeforerunningitontheoriginaldata.
AlldataaccessmoduleSearchCursorparametersdiscussedpreviouslyarevalidforUpdateCursors.ThemaindifferenceisthatdatarowsreturnedbyUpdateCursorsarereturnedaslists.Becauselistsaremutable,theycanbeadjustedusingalistvalueassignment.
Asanexample,let’simaginethatthebusline71willberenamedtothe75.Bothinboundandoutboundlineswillbeaffected,soaSQLexpressionmustbeincludedtogetallrowsofdataassociatedwiththeline.Oncethedatacursoriscreated,therowsreturnedmusthavethenameadjusted,addedbackintothelist,andtheUpdatecursor’supdateRowmethodmustbeinvoked.Hereishowthisscenariowouldlookincode:
sql="NAMELIKE'71%'"
witharcpy.da.UpdateCursor(Bus_Stops,['NAME'],sql),)ascursor:
forrowincursor:
lineName=row[0]
newName=lineName.replace('71','75')
row[0]=newName
TheSQLexpressionwillreturnallrowsofdatawithanamestartingwith71;thiswillinclude71IBand71OB.NotethattheSQLexpressionmustbeenclosedindoublequotes,astheattributevalueneedstobeinsinglequotes.
Foreachrowofdata,thenameatpositionzerointherowreturnedisassignedtothevariablelineName.Thisvariable,astring,usesthereplace()methodtoreplacethecharacters71withthecharacters75.Thiscouldalsojustbereplacing1with5butIwantedtobeexplicitastowhatisbeingreplaced.
Oncethenewstringhasbeengenerated,itisassignedtothevariablenewName.Thisvariableisthenaddedtothelistreturnedbythecursorusinglistassignment;thiswillreplacethedatavaluethatinitiallyoccupiedthezeropositioninthelist.Oncetherowvaluehasbeenassigned,itisthenpassedtothecursor’supdateRow()method.Thismethodacceptstherowandupdatesthevalueinthefeatureclassforthatparticularrow.
UpdatingtheshapefieldForeachrow,allvaluesincludedinthelistreturnedbythecursorareavailableforupdate,excepttheuniqueID(whilenoexceptionwillbethrown,theUIDvalueswillnotbeupdated).Eventheshapefieldcanbeadjusted,withafewcaveats.Themaincaveatisthattheupdatedshapefieldmustbethesamegeometrytypeastheoriginalrow,apointcanbereplacedwithapoint,alinewithaline,andapolygonwithanotherpolygon.
AdjustingapointlocationIfabusstopwasmoveddownthestreetfromitscurrentposition,itwouldneedtobeupdatedusinganUpdateCursor.ThisoperationwillrequireanewlocationinanX/Yformat,preferablyinthesameprojectionasthefeatureclasstoavoidanylossoflocationfidelityinaspatialtransformation.Therearetwomethodsavailabletousforcreatinganewpointlocation,dependingonthemethodusedtoaccessthedata.ThefirstmethodisusedwhenthelocationdataisrequestedusingtheSHAPE@tokens,andrequirestheuseofanArcPyGeometrytype,inthiscasethePointtype.TheArcPyGeometrytypesarediscussedindetailinthenextchapter.
sql='OBJECTID<5'
witharcpy.da.UpdateCursor(Bus_Stops,['OID@','SHAPE@'],sql)ascursor:
forrowincursor:
row[1]=arcpy.Point(5999783.78657,2088532.563956)
BypassinganXandYvaluetotheArcPyPointGeometry,aPointshapeobjectiscreatedandpassedtothecursorintheupdatedlistreturnedbythecursor.Assigninganewlocationtotheshapefieldinatuple,thenusingthecursor’supdateRow()methodallowstheshapefieldvaluetobeadjustedtothenewlocation.Becausethefirstfourbusstopsareatthesamelocation,theyareallmovedtothenewlocation.
Thesecondmethodappliestoallotherformsofshapefieldinteractions,includingtheSHAPE@XY,SHAPE@JSON,SHAPE@KML,SHAPE@WKT,andSHAPE@WKBtokens.Theseareupdatedbypassingthenewlocationintheformatrequestedbacktothecursorandupdatingthelist:
sql='OBJECTID<5'
witharcpy.da.UpdateCursor(Bus_Stops,['OID@','SHAPE@XY'],sql)ascursor:
forrowincursor:
row[1]=(5999783.786500007,2088532.5639999956)
HereisthesamecodeusingtheSHAPE@JSONkeywordandaJSONrepresentationofthedata:
sql='OBJECTID<5'
witharcpy.da.UpdateCursor(Bus_Stops,['OID@','SHAPE@JSON'],sql)as
cursor:
forrowincursor:
printrow
row[1]=u'{"x":5999783.7865000069,"y":2088532.5639999956,
"spatialReference":{"wkid":102643}}'
Aslongasthekeyword,thedataformat,andthegeometrytypematch,thelocationisupdatedtothenewcoordinates.Thekeywordmethodisveryusefulwhenupdatingpoints,however,theSHAPE@XYkeyworddoesnotworkwithlinesorpolygonsasthelocationreturnedrepresentsthecentroidoftherequestedgeometry.
DeletingarowusinganUpdateCursorIfweneedtoremovearowofdata,theUpdateCursorhasadeleteRowmethodthatworkstoremovetherow.Notethatthiswillcompletelyremovethedatarow,makingitunrecoverable.Thismethoddoesnotrequireaparametertobepassedtoit;instead,itwillremovethecurrentrow:
sql='OBJECTID<2'
Bus_Stops=r'C:\Projects\PacktDB.gdb\Bus_Stops'
witharcpy.da.UpdateCursor(Bus_Stops,
['OID@',
'SHAPE@XY'],sql)ascursor:
forrowincursor:
UsinganInsertCursorNowthatwehaveagrasponhowtoupdateexistingdata,let’sinvestigateusingInsertCursorstocreatenewdataandaddittoafeatureclass.Themethodsinvolvedareverysimilartousingotherdataaccesscursors,exceptthatwedonotneedtocreateaniterablecursortoextractrowsofdata;instead,wewillcreateacursorthatwillhavethespecialinsertRowmethodthatiscapableofaddingdatatothefeatureclassrowbyrow.
TheInsertCursorcanbecalledusingthesamewith..assyntaxbutgenerallyitiscreatedasavariableintheflowofthescript.
NoteNotethatonlyonecursorcanbeinvokedatatime;anexception(aPythonerror)willbegeneratedwhencreatingtwoinsert(orupdate)cursorswithoutfirstremovingtheinitialcursorusingthePythondelkeywordtoremovethecursorvariablefrommemory.Thisiswhythewith..assyntaxispreferredbymany.
Thedataaccessmodule’sInsertCursorrequiressomeofthesameparametersastheothercursors.Thefeatureclasstobewrittentoandthelistoffieldsthatwillhavedatainserted(thisincludestheshapefield)arerequired.Thespatialreferencewillnotbeusedasthenewshapedatamustbeinthesamespatialreferenceasthefeatureclass.NoSQLexpressionisallowedforanInsertCursor.
Thedatatobeaddedtothefeatureclasswillbeintheformofatupleoralist,inthesameorderasthefieldsthatarelistedinthefieldslistparameter.Onlyfieldsofinterestneedtobeincludedinthelistoffields,meaningnoteveryfieldneedsavalueinthelisttobeadded.Whenaddinganewrowofdatatoafeatureclass,theuniqueIDwillautomaticallybegenerated,makingitunnecessarytoexplicitlyincludetheuniqueID(intheformoftheOID@keyword)inthelistoffieldstobeadded.
Let’sexplorecodethatcouldbeusedtogenerateanewbusstop.We’llwritetoatestdatasetcalledTestBusStops.WeareonlyinterestedintheNameandStopIDfields,sothosefieldsalongwiththeshapefield(whichisinaStatePlaneprojectionsystem)willbeincludedinthedatalisttobeadded:
Bus_Stops=r'C:\Projects\PacktDB.gdb\TestBusStops'
insertCursor=arcpy.da.InsertCursor(Bus_Stops,['SHAPE@',
'NAME','STOPID'])
coordinatePair=(6001672.5869999975,2091447.0435000062)
newPoint=arcpy.Point(*coordinatePair)
dataList=[newPoint,'NewStop1',112121]
insertCursor.insertRow(dataList)
delinsertCursor
Ifthereisaniterablelistofdatatobeinsertedintothefeatureclass,createtheInsertCursorvariablebeforeenteringtheiteration,anddeletetheInsertCursorvariableoncethedatahasbeeniteratedthrough,orusethewith..asmethodtoautomaticallydeletetheInsertCursorvariablewhentheiterationiscomplete:
Bus_Stops=r'C:\Projects\PacktDB.gdb\TestBusStops'
listOfLists=[[(6002672.58675,2092447.04362),'NewStop2',112122],
[(6003672.58675,2093447.04362),'NewStop3',112123],
[(6004672.58675,2094447.04362),'NewStop4',112124]
]
witharcpy.da.InsertCursor(Bus_Stops,
['SHAPE@',
'NAME',
'STOPID'])asiCursor:
fordataListinlistOfLists:
newPoint=arcpy.Point(*dataList[0])
dataList[0]=newPoint
Asalist,thelistOfListsvariableisiterable.EachlistwithinitisconsideredasdataListintheiteration,andthefirstvalueindataList(thecoordinatepair)ispassedtothearcpy.Point()functiontocreateaPointobject.Thearcpy.Point()functionrequirestwoparameters,XandY;theseareextractedfromthecoordinatepairtupleusingtheasterisk,which‘explodes’thetupleandpassesthevaluesitcontainstothefunction.ThePointobjectisthenaddedbackintodataListusinganindex-basedlistassignment,whichwouldnotbeavailabletousifthedataListvariablewasatuple(wewouldinsteadhavetocreateanewlistandaddinthePointobjectandtheotherdatavalues).
InsertingapolylinegeometryTocreateandinsertapolyline-typeshapefieldfromaseriesofpoints,it’sbesttousetheSHAPE@keyword.WewillalsofurtherexploretheArcPyGeometrytypes,whichwillbediscussedinthenextchapter.WhenworkingwiththeSHAPE@keyword,wehavetoworkwithdatainESRI’sspatialbinaryformats,andthedatamustbewrittenbacktothefieldinthesameformatusingtheArcPyGeometrytypes.
Tocreateapolyline,thereisonerequirement,atleasttwovalidpointsmadeoftwocoordinatepairs.WhenworkingwiththeSHAPE@keyword,thereisamethodologytoconvertingthecoordinatepairsintoanArcPyPointandthenaddingittoanArcPyArray,whichisthenconvertedintoanArcPyPolylinetobewrittenbacktotheshapefield:
listOfPoints=[(6002672.58675,2092447.04362),
(6003672.58675,2093447.04362),
(6004672.58675,2094447.04362)
]
line='NewBusLine'
lineID=12345
busLine=r'C:\Projects\PacktDB.gdb\TestBusLine'
insertCursor=arcpy.da.InsertCursor(busLine,['SHAPE@',
'LINE','LINEID'])
lineArray=arcpy.Array()
forpointsPairinlistOfPoints:
newPoint=arcpy.Point(*pointsPair)
lineArray.add(newPoint)
newLine=arcpy.Polyline(lineArray)
insertData=newLine,line,lineID
ThethreecoordinatepairsintuplesareiteratedandconvertedintoPointobjects,whichareinturnaddedtotheArrayobjectcalledlineArray.TheArrayobjectisthenaddedtothePolylineobjectcallednewLine,whichisthenaddedtoatuplewiththeotherdataattributesandinsertedintothefeatureclassbytheInsertCursor.
InsertingapolygongeometryPolygonsarealsoinserted,orupdated,usingcursors.TheArcPyPolygonGeometrytypedoesnotrequirethecoordinatepairstoincludethefirstpointtwice(thatis,asthefirstpointandasthelastpoint).Thepolygonisclosedautomaticallybythearcpy.Polygon()function:
listOfPoints=[(6002672.58675,2092447.04362),
(6003672.58675,2093447.04362),
(6004672.58675,2093447.04362),
(6004672.58675,2091447.04362)
]
polyName='NewPolygon'
polyID=54321
blockPoly=r'C:\Projects\PacktDB.gdb\Chapter5Results\TestPolygon'
insertCursor=arcpy.da.InsertCursor(blockPoly,['SHAPE@','BLOCK',
'BLOCKID'])
polyArray=arcpy.Array()
forpointsPairinlistOfPoints:
newPoint=arcpy.Point(*pointsPair)
polyArray.add(newPoint)
newPoly=arcpy.Polygon(polyArray)
insertData=newPoly,polyName,polyID
insertCursor.insertRow(insertData)
Hereisavisualizationoftheresultoftheinsertoperation:
SummaryInthischapterwecoveredthebasicusesofdataaccessmodulecursors.Search,updateandInsertCursorswereexploredanddemonstrated,andaspecialfocuswasplacedontheuseofthesecursorsforextractingshapedatafromtheshapefield.Cursorparameterswerealsointroduced,includingthespatialreferenceparameterandtheSQLexpressionwhereclauseparameter.Inthenextchapter,wewillfurtherexploretheuseofcursors,especiallywiththeuseofArcPyGeometrytypes.
Chapter6.WorkingwithArcPyGeometryObjectsTheessenceofgeospatialanalysisisusinggeometricshapes–points,lines,andpolygons–tomodelthegeographyofrealworldobjectsandtheirlocation-basedrelationships.Thesimpleshapesandtheirgeometricpropertiesoflocation,lengthandareaareprocessedusinggeospatialoperationstogenerateanalysisresults.Itisthecombinationofmodeledgeographicdataandtheassociatedattributeinformationthatseparategeospatialinformationsystemsfromallotherinformationsystems.
UntilArcPy,processingthefeatureclassgeometryusingthegeospatialoperationswasdependedonthepre-builttoolswithinArcToolbox.ArcPyhasmadeitpossibletodirectlyaccessthegeometricshapeswhicharestoredasmathematicalrepresentationsintheshapefieldoffeatureclasses.Onceaccessed,thisgeometricdataisloadedintoArcPygeometryobjectstomakethedataavailableforanalysiswithinanArcPyscript.Becauseofthisadvance,writingscriptsthataccessgeometryfieldsandusethemtoperformanalysishastransformedArcGISgeospatialanalysis.Inthischapter,we’llexplorehowtogenerateandusetheArcPygeometryobjectstoperformgeospatialoperations,andapplythemtothebusstopsanalysis.
Inthischapter,wewillcover:PointandArrayconstructorobjectsandPointGeometry,Polyline,andPolygongeometryobjects
HowtousethegeometryobjectstoperformgeospatialoperationsHowtointegratethegeometryobjectsintoscriptsHowtoperformcommongeospatialoperationsusingthegeometryobjectsHowtoreplacetheuseofArcToolboxtoolsinthescriptwithgeometryobjectmethods
ArcPygeometryobjectclassesIndesigninggeometryobjects,theauthorsofArcPymadeitpossibletoperformgeospatialoperationsinmemory,reducingtheneedtousetoolsintheArcToolboxfortheseoperations.Thiswillresultinspeedgainsasthereisnoneedtowritetheresultsofthecalculationstodiskateachstepoftheanalysis.Instead,theresultsofthestepscanbepassedfromfunctiontofunctionwithinthescript.Thefinalresultsoftheanalysiscanbewrittentotheharddriveasafeatureclass,ortheycanbewrittenintoaspreadsheetorpassedtoanotherprogram.
ThegeometryobjectsarewrittenasPythonclasses-specialblocksofcodethatcontaininternalfunctions.Theinternalfunctionsarethemethodsandpropertiesofthegeometryobjects;whencalledtheyallowtheobjecttoperformanoperation(amethod)ortorevealinformationaboutthegeometryobject(aproperty).Pythonclassesarewrittenwithamainclassthatcontainssharedmethodsandproperties,andwithsub-classesthatreferencethemainclassbutalsohavespecificmethodsandpropertiesthatarenotshared.Here,themainclassistheArcPyGeometryobject,whilethesub-classesarethePointGeometry,Multipoint,PolylineandPolygonobjects.
Thegeometryobjectsaregeneratedinthreeways.Thefirstrequiresusingdatacursorstoreadexistingfeatureclassesandpassingaspecialkeywordasafieldname.Theshapedatareturnedbythecursorisageometryobject.Thesecondmethodistocreatenewdatabypassingrawcoordinatestoaconstructorobject(eitheraPointorArrayobject),whichisthenpassedtoageometryobject.ThethirdmethodistoreaddatafromafeatureclassusingtheCopyFeaturestoolfromtheArcToolbox.
Eachgeometryobjecthasmethodsthatallowforreadaccessandwriteaccess.Thereadaccessmethodsareimportantforaccessingthecoordinatepointsthatconstitutethepoints,linesandpolygons.Thewriteaccessmethodsareimportantwhengeneratingnewdataobjectsthatcanbeanalyzedorwrittentodisk.
ThePointGeometry,Multipoint,Polyline,andPolygongeometryobjectsareusedforperforminganalysisupontheirrespectivegeometrytypes.Thegenericgeometryobjectcanacceptanygeometrytypeandanoptionalspatialreferencetoperformgeospatialoperationswhenthereisnoneedtodiscernthegeometrytype.
TwootherArcPyclasseswillbeusedforperforminggeospatialoperationsinmemory:theArrayobjectandthePointobject.Theyareconstructorobjects,astheyarenotsub-classedfromthegeometryclass,butareinsteadusedtoconstructthegeometryobjects.ThePointobjectisusedtocreatecoordinatepointsfromrawcoordinates.TheArrayobjectisalistofcoordinatepointsthatcanbepassedtoaPolylineorPolygonobject,asaregularPythonlistofArcPyPointobjectscannotbeusedtogeneratethosegeometryobjects.
ArcPyPointobjectsPointobjectsarethebuildingblocksusedtogenerategeometryobjects.Also,allofthegeometryobjectswillreturncomponentcoordinatesasPointobjectswhenusingreadaccessmethods.PointobjectsallowforsimplegeometryaccessusingitsX,YandZproperties,andalimitednumberofgeospatialmethods,suchascontains,overlaps,within,touches,crosses,equals,anddisjoint.Let’suseIDLEtoexploresomeofthesemethodswithtwoPointgeometryobjectswiththesamecoordinates:
>>>Point=arcpy.Point(4,5)
>>>point1=arcpy.Point(4,5)
>>>Point.equals(point1)
True
>>>Point.contains(point1)
True
>>>Point.crosses(point1)
False
>>>Point.overlaps(point1)
False
>>>Point.disjoint(point1)
False
>>>Point.within(point1)
True
>>>point.X,Point.Y
(4.0,5.0)
Intheseexamples,weseesomeoftheidiosyncrasiesofthePointobject.Withtwopointsthathavethesamecoordinates,theresultsoftheequalsmethodandthedisjointmethodareasexpected.ThedisjointmethodwillreturnTruewhenthetwoobjectsdonotsharecoordinates,whiletheoppositeistruewiththeequalsmethod.ThecontainsmethodwillworkwiththetwoPointobjectsandreturnTrue.Thecrossesmethodandoverlapsmethodaresomewhatsurprisingresults,asthetwoPointobjectsdooverlapinlocationandcouldbeconsideredtocross;however,thosemethodsdonotreturntheexpectedresultastheyarenotbuilttocomparetwopoints.
ArcPyArrayobjectsBeforeweprogressuptoPolylineandPolygonobjects,weneedtounderstandtheArcPyArrayobject.ItisthebridgebetweenthePointobjectsandthosegeometryobjectsthatrequiremultiplecoordinatepoints.ArrayobjectsacceptPointobjectsasparameters,andtheArrayobjectisinturnpassedasaparametertothegeometryobjecttobecreated.Let’susePointobjectswithanArrayobjecttounderstandbetterhowtheyworktogether.
TheArrayobjectissimilartoaPythonlist,withextend,append,andreplacemethods,andalsohasuniquemethodssuchasaddandclone.TheaddmethodwillbeusedtoaddPointobjectsindividually:
>>>Point=arcpy.Point(4,5)
>>>point1=arcpy.Point(7,9)
>>>Array=arcpy.Array()
>>>Array.add(point)
>>>Array.add(point1)
Theextend()methodwouldaddalistofPointobjectsallatonce:
>>>Point=arcpy.Point(4,5)
>>>point1=arcpy.Point(7,9)
>>>pList=[Point,point1]
>>>Array=arcpy.Array()
>>>Array.extend(pList)
TheinsertmethodwillputaPointobjectintheArrayataspecificindex,whilethereplacemethodisusedtoreplaceaPointobjectinanArraybypassinganindexandanewPointobject:
>>>Point=arcpy.Point(4,5)
>>>point1=arcpy.Point(7,9)
>>>point2=arcpy.Point(11,13)
>>>pList=[Point,point1]
>>>Array=arcpy.Array()
>>>Array.extend(pList)
>>>Array.replace(1,point2)
>>>point3=arcpy.Point(17,15)
>>>Array.insert(2,point3)
TheArrayobject,whenloadedwithPointobjects,canthenbeusedtogeneratetheothergeometryobjects.
ArcPyPolylineobjectsThePolylineobjectisgeneratedwithanArrayobjectthathasatleasttwoPointobjects.AsgiveninthefollowingIDLEexample,onceanArrayobjecthasbeengeneratedandloadedwiththePointobjects,itcanthenbepassedasaparametertoaPolylineobject:
>>>Point=arcpy.Point(4,5)
>>>point1=arcpy.Point(7,9)
>>>pList=[Point,point1]
>>>Array=arcpy.Array()
>>>Array.extend(pList)
>>>pLine=arcpy.Polyline(Array)
NowthatthePolylineobjecthasbeencreated,itsmethodscanbeaccessed.Thisincludesmethodstorevealtheconstituentcoordinatepointswithinthepolyline,andotherrelevantinformation:
>>>pLine.firstPoint
<Point(4.0,5.0,#,#)>
>>>pLine.lastPoint
<Point(7.0,9.0,#,#)>
pLine.getPart()
<Array[<Array[<Point(4.0,5.0,#,#)>,<Point(7.0,9.0,#,#)>]>]>
>>>pLine.trueCentroid
<Point(5.5,7.0,#,#)>
>>>pLine.length
5.0
>>>pLine.pointCount
2
ThisexamplePolylineobjecthasnotbeenassignedaspatialreferencesystem,sothelengthisunitless.Whenageometryobjectdoeshaveaspatialreferencesystem,thelinearandarealunitswillbereturnedinthelinearunitofthesystem.
ThePolylineobjectisalsoourfirstgeometryobjectwithwhichwecaninvokegeometryclassmethodsthatperformgeospatialoperations,suchasbuffers,distanceanalyses,andclips:
>>>bufferOfLine=pLine.buffer(10)
>>>bufferOfLine.area
413.93744395
>>>bufferOfLine.contains(pLine)
True
>>>newPoint=arcpy.Point(25,19)
>>>pLine.distanceTo(newPoint)
20.591260281974
AnotherusefulmethodofPolylineobjectsisthepositionAlongLinemethod.ItisusedtoreturnaPointGeometryobject,discussedinthefollowing,ataspecificpositionalongtheline.ThispositionalongthelinecaneitherbeanumericdistancefromthefirstPointorasapercentage(expressedasafloatfrom0-1),whenusingtheoptionalsecondparameter:
>>>nPoint=pLine.positionAlongLine(3)
>>>nPoint.firstPoint.X,nPoint.firstPoint.Y
(5.8,7.4)>>>pPoint=pLine.positionAlongLine(.5,True)
>>>pPoint.firstPoint.X,pPoint.firstPoint.Y
(5.5,7.0)
ThereareanumberofothermethodsavailabletoPolylineobjects.Moreinformationisavailablehere:http://resources.arcgis.com/en/help/main/10.2/index.html#//018z00000008000000
ArcPyPolygonobjectsTocreateaPolygonobject,anArrayobjectmustbeloadedwithPointobjectsandthenpassedasaparametertothePolygonobject.OncethePolygonobjecthasbeengenerated,themethodsavailabletoitareveryusefulforperforminggeospatialoperations.ThegeometryobjectscanalsobesavedtodiskusingtheArcToolboxCopyFeaturestool.ThisIDLEexampledemonstrateshowtogenerateashapefilebypassingaPolygonobjectandarawstringfilenametothetool:
>>>importarcpy
>>>point1=arcpy.Point(12,16)
>>>point2=arcpy.Point(14,18)
>>>point3=arcpy.Point(11,20)
>>>Array=arcpy.Array()
>>>Points=[point1,point2,point3]
>>>Array.extend(points)
>>>Polygon=arcpy.Polygon(array)
>>>arcpy.CopyFeatures_management(polygon,r'C:\Projects\Polygon.shp')
<Result'C:\\Projects\\Polygon.shp'>
PolygonobjectbuffersPolygonobjects,likePolylineobjects,havemethodsthatmakeiteasytoperformgeospatialoperationssuchasbuffers.Bypassinganumbertothebuffermethodasaparameter,abufferwillbegeneratedinmemory.TheunitofthenumberisdeterminedbytheSpatialReferencesystem.Internalbufferscanbegeneratedbysupplyingnegativebuffernumbers;thebuffergeneratedbeingtheareawithinthePolygonobjectatthespecifieddistancefromthePolygonperimeter.Clips,unions,symmetricaldifferences,andmoreoperationsareavailableasmethods,asarewithinorcontainsoperations;evenprojectionscanbeperformedusingthePolygonobjectmethodsaslongasithasaSpatialReferencesystemobjectpassedasaparameter.FollowingisascriptthatwillcreatetwoshapefileswithtwoseparateSpatialReferencesystems,eachidentifiedbyanumericcode(2227and4326)fromtheEPSGcodingsystem:
importarcpyPoint=arcpy.Point(6004548.231,2099946.033)
point1=arcpy.Point(6008673.935,2105522.068)
point2=arcpy.Point(6003351.355,2100424.783)Array=arcpy.Array()
Array.add(point1)
Array.add(point)
array.add(point2)
Polygon=arcpy.Polygon(array,2227)
buffPoly=Polygon.buffer(50)
features=[Polygon,buffPoly]
arcpy.CopyFeatures_management(features,
r'C:\Projects\Polygons.shp')
spatialRef=arcpy.SpatialReference(4326)
polygon4326=Polygon.projectAs(spatialRef)
arcpy.CopyFeatures_management(polygon4326,
r'C:\Projects\polygon4326.shp')
HereishowthesecondshapefilelooksintheArcCatalogPreviewwindow:
OtherPolygonobjectmethodsUnlikethecliptoolintheArcToolbox,whichcanclipaPolygonusinganotherpolygon,theclipmethodrequiresanextentobject(anotherArcPyclass)andislimitedtoarectangularenvelopearoundtheareatobeclipped.Toremoveareasfromapolygon,thedifferencemethodcanworklikethecliporerasetoolintheArcToolbox:
buffPoly=Polygon.buffer(500)
donutHole=buffPoly.difference(Polygon)
features=[Polygon,donutHole]
arcpy.CopyFeatures_management(features,
r"C:\Projects\Polygons2.shp")
Hereisthedonuthole-likeresultofthebufferanddifferenceoperation.ThebufferwiththedonutholesurroundstheoriginalPolygonobject:
ArcPygeometryobjectsThegenericgeometryobjectisquiteusefulforcreatinginmemoryacopyofthegeometryofafeatureclass,withoutfirstneedingtoknowwhichtypeofgeometrythefeatureclasscontains.LikealloftheArcPygeometryobjects,itsreadmethodsincludetheextractionofthedatainmanyformatssuchasJSON,WKT,andWKB.Thearea(ifitisapolygon),thecentroid,theextent,andtheconstituentpointsofeachgeometryarealsoavailable,asdemonstratedpreviously.
HereisanexampleofreadingthegeometryofafeatureclassintomemoryusingtheCopyFeaturestool:
importarcpy
cen2010=r'C:\Projects\ArcPy.gdb\SanFrancisco\CensusBlocks2010'
blockPolys=arcpy.CopyFeatures_management(cen2010,
arcpy.Geometry())
ThevariableblockPolysisaPythonlistcontainingallofthegeometriesloadedintoit;inthiscaseitiscensusblocks.Thelistcanthenbeiteratedtobeanalyzed.
ArcPyPointGeometryobjectsThePointGeometryobjectisveryusefulforperformingthesesamegeospatialoperationswithpoints,whicharenotavailablewiththePointobjects.WhenacursorisusedtoretrieveshapedatafromafeatureclasswithaPointGeometrytype,theshapedataisreturnedasaPointGeometryobject.WhilePointobjectsarerequiredtoconstructallothergeometryobjectswhenacursorisnotusedtoretrievedatafromafeatureclass,it’sthePointGeometryobjectthatisusedtoperformpointgeospatialoperations.
Let’sexploregettingPointGeometryobjectsfromadataaccessmoduleSearchCursorandusingthereturneddatarowstocreatebufferedpoints.Inourbusstopanalysis,thiswillreplacetheneedtousetheArcToolboxBuffertooltocreatethe400footbuffersaroundeachstop.ThescriptinthefollowingusesadictionarytocollectthebufferobjectsandthensearchesthecensusblocksusinganotherSearchCursor.ToaccesstheshapefieldusingtheSearchCursor()method,[email protected],thescriptwilliteratethroughthebusstopsandfindallcensusblockswithwhicheachstopintersects:
#Generate400footbuffersaroundeachbusstop
importarcpy,csv
busStops=r"C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops"
censusBlocks2010=r"C:\Projects\PacktDB.gdb\SanFrancisco\CensusBlocks2010"
sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
dataDic={}
witharcpy.da.SearchCursor(busStops,['NAME','STOPID','SHAPE@'],sql)as
cursor:
forrowincursor:
linename=row[0]
stopid=row[1]
shape=row[2]
dataDic[stopid]=shape.buffer(400),linename
NowthatthedatahasbeenretrievedandthebuffershavebeengeneratedusingthebuffermethodofthePointGeometryobjects,thebufferscanbecomparedagainstthecensusblockgeometryusingiterationandaSearchCursor.Therewillbetwogeospatialmethodsusedinthisanalysis:overlapandintersect.Theoverlapsmethodisabooleanoperation,returningavalueoftrueorfalsewhenonegeometryiscomparedagainstanother.Theintersectmethodisusedtogettheactualareaoftheintersectaswellasidentifyingthepopulationofeachblock.Usingtheintersectrequirestwoparameters:asecondgeometryobject,andanintegerindicatingwhichtypeofgeometrytoreturn(1forpoint,2forline,4forpolygon).Wewantthepolygonalareaofintersectreturnedtohaveanareaofintersectionavailablealongwiththepopulationdata:
#Intersectcensusblocksandbusstopbuffers
processedDataDic={}={}
forstopidindataDic.keys():
values=dataDic[stopid]
busStopBuffer=values[0]
linename=values[1]
blocksIntersected=[]
witharcpy.da.SearchCursor(censusBlocks2010,
['BLOCKID10','POP10','SHAPE@'])ascursor:
forrowincursor:
block=row[2]
population=row[1]
blockid=row[0]
ifbusStopBuffer.overlaps(block)==True:
interPoly=busStopBuffer.intersect(block,4)
data=row[0],row[1],interPoly,block
blocksIntersected.append(data)
processedDataDic[stopid]=values,blocksIntersected
Thisportionofthescriptiteratesthroughtheblocksandintersectsagainstthebufferedbusstops.Nowthatwecanidentifytheblocksthattouchthebufferaroundeachstopandthedataofinteresthasbeencollectedintothedictionary,itcanbeprocessedandtheaveragepopulationofalloftheblockstouchedbythebuffercanbecalculated:
#Createanaveragepopulationforeachbusstop
dataList=[]
forstopidinprocessedDataDic.keys():
allValues=processedDataDic[stopid]
popValues=[]
blocksIntersected=allValues[1]
forblocksinblocksIntersected:
popValues.append(blocks[1])
averagePop=sum(popValues)/len(popValues)
busStopLine=allValues[0][1]
busStopID=stopid
finalData=busStopLine,busStopID,averagePop
dataList.append(finalData)
Nowthatthedatahasbeencreatedandaddedtoalist,itcanbeoutputtedtoaspreadsheetusingthecreateCSVmodulewecreatedinChapter4,ComplexArcPyScriptsandGeneralizingFunctions:
#Generateaspreadsheetwiththeanalysisresults
defcreateCSV(data,csvname,mode='ab'):
withopen(csvname,mode)ascsvfile:
csvwriter=csv.writer(csvfile,delimiter=',')
csvwriter.writerow(data)
csvname="C:\Projects\Output\StationPopulations.csv"
headers='BusLineName','BusStopID','AveragePopulation'
createCSV(headers,csvname,'wb')
fordataindataList:
createCSV(data,csvname)
Thedatahasbeenprocessedandwrittentothespreadsheet.Thereisonemorestepthatwecantakewiththedataandthatistousetheareaoftheintersectiontocreateaproportionalpopulationvalueforeachbuffer.Let’sredotheprocessingofthedatatoincludetheproportionalareas:
dataList=[]
forstopidinprocessedDataDic.keys():
allValues=processedDataDic[stopid]
popValues=[]
blocksIntersected=allValues[1]
forblocksinblocksIntersected:
pop=blocks[1]
totalArea=blocks[-1].area
interArea=blocks[-2].area
finalPop=pop*(interArea/totalArea)
popValues.append(finalPop)
averagePop=round(sum(popValues)/len(popValues),2)
busStopLine=allValues[0][1]
busStopID=stopid
finalData=busStopLine,busStopID,averagePop
dataList.append(finalData)
NowthescriptistakingfulladvantageofthepowerofArcPygeometryobjects,andthescriptisrunningcompletelyinmemorywhichavoidsproducinganyintermediatedatasets.
SummaryInthischapter,wediscussedindetailtheuseofArcPygeometryobjects.Thesevariedobjectshavesimilarmethodsandare,infact,sub-classedfromthesamePythonclass.Theyareusefulforperformingin-memorygeospatialanalyses,whichavoidshavingtoreadandwritedatafromtheharddriveandalsoskipscreatinganyintermediatedata.
ArcPygeometryobjectswillbecomeanimportantpartofautomatinggeospatialworkflows.CombiningthemwithSearchCursorsmakesArcPymoreusefulthananyearlierimplementationofPythonscriptingtoolsforArcGIS.Next,wewillconverttherawscriptintoascripttoolthatcanbeexecuteddirectlyfromtheArcToolboxorapersonaltoolboxinageodatabase.
Chapter7.CreatingaScriptToolNowthatthebasicsofcreatingandexecutingArcPyscriptshavebeencovered,weneedtotakethenextstepandcreatere-useableScripttools.CreatingScripttoolswillallowforgreatercodereuse,andwillmakeiteasytocreatecustomtoolsforotherGISanalystsandcustomers.WithaPythonscriptbackendorcode,andafamiliarArcGIStoolfrontendoruserinterface,theparticularsofthecodearehiddenfromtheuser;itbecomesjustanothertool,albeitatoolthatcansavedaysandweeksofwork.
Thischapterwillcoverthefollowingtopics:
AddingparameterstoscriptstoacceptinputandproduceoutputasrequiredbytheuserCreatingacustomtoolfrontendandacustomtoolboxSettingtheparametersofthetoolfrontendtoallowittopassargumentstothecodebackend
AddingdynamicparameterstoascriptThescriptswehavegeneratedinpreviouschaptershaveallhadhard-codedinputs.Theinputvalueswerewritteninthescriptasstringsornumbersandassignedtovariables.WhiletheycanbeupdatedmanuallytoreplacetheinputandoutputfilepathsandSQLstatements,programmersshouldaimtocreatescriptsthatwillnotrequireeditingeachtimetheyareused.Instead,scriptsshouldbedesignedtobedynamicandacceptfilepathsandotherinputsasparametersorarguments,inmuchthesamemannerthatthefunctionswehavecreatedacceptparameters.
Pythonwasdesignedwiththisinmind,andthesysmodulehasamethodcalledsys.argvthatacceptsinputspassedtothescriptwhenitisexecuted.WhilethedesignersofArcPyanditspredecessorarcgisscriptingmoduleinitiallytookadvantageofthesys.argvmethod,intimetheydesignedanArcPymethodforacceptingscriptparameters.AseithermethodcanbeusedwhenwritingArcPyscripts,andbotharefoundinexamplescriptsontheweb,itisimportanttorecognizetheminutedifferencesbetweenthesys.argvmethodandarcpy.GetParameterAsText().Themajordifferencebetweenthetwomethodsisthatsys.argvacceptsthedynamicargumentsasalist.Membersofthelistareaccessedusinglistindexingandassignedtovariables.Arcpy.GetParameterAsText()isafunctionthatacceptsanindexnumberparameter.Theindexnumberpassedreflectstheorderoftheparameterwithinthetool’sfrontend;thefirstparameteriszero,thenextisone,andsoon.
NoteIftheorderoftheparametersisadjustedinthetoolfrontend,thisadjustmentmustbereflectedinthecodebackend.
Displayingscriptmessagesusingarcpy.AddMessageItisimportanttoreceivefeedbackfromscriptstoassesstheprogressofthescriptasitperformsananalysis.Asbasicasthiswouldseem,Pythonscriptsandprogramminglanguagesingeneraldonot,bydefault,provideanyfeedbackexceptforerrorsandtheterminationofthescript.Thiscanbeabitdiscouragingtothenoviceprogrammer,asallbuilt-infeedbackisnegative.
Toalleviatethislackoffeedback,theuseofprintstatementsallowsthescripttogivereportsontheprogressoftheanalysisasitruns.However,whenusingaScripttool,printstatementsdonothaveanyeffect.Theywillnotbedisplayedanywhere,andareignoredbythePythonexecutable.TodisplaymessagesinthescriptconsolewhenScripttoolsareexecuted,ArcPyhasaarcpy.AddMessage()method.
Arcpy.AddMessagestatementsareaddedtoscriptswhereverfeedbackisrequiredbytheprogrammer.Thefeedbackrequiredispassedtothemethodasaparameteranddisplayed;whetheritbealist,string,floatorinteger.Arcpy.AddMessagemakesiteasytocheckontheresultsofanalysiscalculations,toensurethatthecorrectinputsareusedandthatthecorrectoutputsareproduced.Asthisfeedbackfromthescriptcanbeapowerfuldebuggingtool,usearcpy.AddMessagewheneverthereisaneedforfeedbackfromtheScripttool.
NoteNotethatstatementspassedtoarcpy.AddMessagewillonlydisplaywhenthescriptisrunasaScripttool.
AddingdynamiccomponentstothescriptTostartmakingthescriptintoaScripttool,weshouldfirstcopythescriptthatwecreatedinChapter6,WorkingwithArcPyGeometryObjectsChapter6_1.py,asChapter7_1.pyinanewfoldercalledChapter7.Wecanthenstartreplacingthehard-codedvariableswithdynamicvariablesusingarcpy.GetParameterAsText.ThereisanotherArcPymethodcalledGetParameterthatacceptstheinputsasanobject,butforourpurposes,GetParameterAsTextisthemethodtouse.
Byaddingarcpy.GetParameterAsTextandarcpy.AddMessagetothescript,wewillhavetakenthefirststeptowardscreatingaScripttool.Caremustbetakentoensurethatthevariablescreatedfromthedynamicparametersareinthecorrectorder,asreorderingthemcanbetime-consuming.Oncetheparametersareaddedtothescriptandthehard-codedportionsofthescriptreplacedwithvariables,thescriptisreadytobecomeaScripttool.
First,moveallofthevariablesthatarehard-codedintothetopofthescript.Then,replacealloftheassignedvalueswitharcpy.GetParameterAsText,makingthemdynamicvalues.Eachparameterisreferredtousingzero-basedindexing;however,theyarepassedtoafunctionindividuallyinsteadofasamemberofalist:
#Chapter7.py
importarcpy,csv
busStops=arcpy.GetParameterAsText(0)
censusBlocks2010=arcpy.GetParameterAsText(1)
censusBlockField=arcpy.GetParameterAsText(2)
csvname=arcpy.GetParameterAsText(3)
headers=arcpy.GetParameterAsText(4).split(',')
sql=arcpy.GetParameterAsText(5)
keyfields=arcpy.GetParameterAsText(6).split(';')
dataDic={}
censusFields=['BLOCKID10',censusBlockField,'SHAPE@']
if"SHAPE@"notinkeyfields:
keyfields.append("SHAPE@")
arcpy.AddMessage(busStops)
arcpy.AddMessage(censusBlocks2010)
arcpy.AddMessage(censusBlockField)
arcpy.AddMessage(csvname)
arcpy.AddMessage(sql)
arcpy.AddMessage(keyfields)
Asyoucanseefromthevariablekeyfieldsandthevariableheaders,somefurtherprocessingmustbeappliedtocertainvariables,asnotallofthemaregoingtobeusedasstrings.Inthiscase,alistiscreatedfromthestringpassedtothevariablekeyfieldsbyusingthestringfunctionssplitandsplittingthestringoneverysemi-colon,whiletheheadersvariableiscreatedbysplittingonthecommas.Toothervariables,suchasthecensusBlockFieldvariableandthevariablekeyfields,theSHAPE@keywordisaddedbecauseitwillberequiredeachtimetheanalysisisrun.Ifaparticularfieldisrequiredforeachrunofthedata,suchastheBLOCKID10field,itcanremainhard-codedinthescript,oroptionallycouldbecomeitsownselectablefieldparameterintheScripttool.
Thevariableswillthenbeaddedtotheremainderofthescriptinthecorrectplaces,makingthescriptreadyfortheScripttooltobecomepartofacustomToolboxinageodatabaseorinArcToolbox.However,wemustfirstcreatethetoolpartoftheScripttoolforthevaluestobecollectedandpassedtothescript.
CreatingaScripttoolCreatingascripttoolisapowerfulcombinationofthepowerofArcPyandtheeaseofuseofthetoolsinArcToolbox.
Thefirststepistocreateacustomtoolboxtoholdthescripttool.Toachievethis,dothefollowing:
1. OpenupArcCatalogandrightclickintheSanFrancisco.gdbFileGeodatabase.2. SelectNewandthenToolboxfromthemenu.3. CallthetoolboxChapter8Tools.4. RightclickonChapter8Tools,selectAdd,andthenselectScript.
Thefollowingmenuwillappearallowingyoutosetupthescripttool.Addatitlewithnospacesandalabel,aswellasadescription.Iprefertorunscripttoolsintheforegroundtoseethemessagesitpasses,butitisnotnecessaryandcanbeannoyingwhenneedingtostartascriptandstillworkonothertasks.ClickNextoncethemenuhasbeenfilledout.
Thenextmenucontainsanentryfieldandafiledialogbutton,allowingtheusertofindthescripttowhichtheparameterscollectedwillbepassed.Usethefiledialogtonavigatetoandselectthescript,andmakesurethatRunPythonscriptinprocessischecked.
LabellinganddefiningparametersThenextdialogboxisthemostimportantone.Itiswheretheparameterstobepassedarelabeledandtheirdatatypesaredefined.Caremustbetakentochoosethecorrectdatatypeforeachparameterastherearemultipledatatypesthatcanbesuppliedforsomeoftheparameters.Also,propertiesforeachparameterwillbedefined;thisinformationwillcharacterizethedatatobecollectedandhelptomakeitpossibleforthedatatobeinthecorrectformataswellasthecorrectdatatype.
Startbyaddingthedisplaynameforeachparametertobecollected.Thedisplaynameshouldexplainthetypeofinputthatisrequired.Forinstance,thefirstparameterwillbethebusstop’sfeatureclass,sothedisplaynamecouldbeBusStopFeatureClass.
NoteMakesuretoaddthedisplaynamesintheorderthattheywillbepassedtovariablesinthescript.
AddingdatatypesNext,addintheDataTypesforeachparameter.Thisisintricatebecausetherewillbealargelistofdatatypestochoosefrom,andoftenthereareafewtypesthatwouldworkformanyparameters.Forinstance,ifashapefileparameteriscreated,itwouldallowtheusertoselectashapefileasexpected.However,itmightbebettertousetheFeatureClassdatatype,asthenbothshapefilesandfeatureclassescouldbeusedintheanalysis.
AddingtheBusStopfeatureclassasaparameterThefirstparameteristheBusStopfeatureclass,anditshouldbeaFeatureClassdatatype.ClickontheDataTypeFieldnexttothedisplaynameandadrop-downlistwillappear.Oncethedatatypeisselected,checkouttheparameterpropertiesbelowthelistofparameters.FortheBusStopfeatureclass,thedefaultswillbeacceptable,asthe
featureclassisrequired,isnotamulti-value,hasnodefaultorenvironmentsettings,andisnotobtainedfromanyotherparameter.
NoteSomeoftheparameterswillrequireanotherparametertobeselectedfirstastheyarederivedvaluesobtainedfromthefirstparameter.TheCensusBlockFieldparameterandtheSQLstatementparameterderivevaluesfromtheCensusBlockfeatureclassandBusStopfeatureclassparameters,respectively.
AddingtheCensusBlockfeatureclassasaparameterTheCensusBlockfeatureclassissimilartotheBusStopfeatureclass.ItwillbeaFeatureClassdatatype,allowingbothshapefilesandfeatureclassestobeselected,andthereisnoneedtoadjustthedefaultparameterproperties.Oncethedatatypehasbeenset,the
CensusBlockparameterisreadyforuse.
AddingtheCensusBlockfieldasaparameterTheCensusBlockfieldparameterhasanewtwist;itisobtainedfromtheCensusBlockfeatureclassparameter,andwillonlybepopulatedoncethatfirstparameterhasbeencreated.Tomakethispossible,theObtainedfromparameterpropertywillhavetobeset.SelectFieldasthedatatype,andthenclickontheblankareanexttotheObtainedfromparameterpropertyandselectCensus_Block_Feature_Class.ThiswillcreatealistofthefieldscontainedwithintheCensusBlockfeatureclass.
AddingtheoutputspreadsheetasaparameterAsthespreadsheetthatwillbeproducedfromtheanalysisrunbythescripttoolisaCommaSeparatedValue(CSV)file,selectTextFileastheDataType.Settingthe
Defaultparameterpropertytoafilenamecansavetime,andwillmaketherequiredfileextensioneasiertoidentify.Also,asthespreadsheetwillbeproducedbytheScripttoolasanoutput,theDirectionparameterpropertyshouldbeOutput.
AddingthespreadsheetfieldnamesasaparameterThefieldnameschosenasheadersfortheoutputspreadsheetshouldbeaStringdatatype,withthefieldnamesseparatedbycommasandnospaces.Thescriptusesthestringdatatype’ssplitmethodtoseparatethefieldnames.Passingacommatothesplitmethodseparatestheparameterbysplittingtheinputstringonthecommastocreatealistoffieldnames.Thelistoffieldnameswillbeusedasaheaderbythecsvmodulewhencreatingthespreadsheet.
AddingtheSQLStatementasaparameterTheSQLStatementparameterwillrequirethehelpfulSQLExpressionBuildermenuandshouldthereforebeaSQLExpressiondatatype.TheSQLExpressionBuilderwilluseafieldobtainedfromtheBusStopfeatureclass.SettheObtainedfromparameterpropertytotheBusStopfeatureclassbyclickingonthatpropertyandselectingBus_Stop_Feature_Classfromthedrop-downlist.
AddingthebusstopfieldsasaparameterThefinalparameteristhebusstopfieldsparameter,whichisaFielddatatypethatwillbeobtainedfromtheBusStopfeatureclass.ChangetheMultiValueparameterpropertyfromNotoYestoallowtheusertoselectmultiplefields.AlsoremembertosettheObtainedfromparameterpropertytoBus_Stop_Feature_ClasssothatthefieldsarepopulatedfromtheBusStopfeatureclassparameter.
Nowthatalltheparametershavebeendescribedandtheirpropertieshavebeenset,thescripttoolisready.ClickonFinishtoclosethemenu.
InspectingthefinalscriptOncealloftheparametershavebeencollected,thevariablesarethenusedtoreplacethehard-codedfieldlistsorotherstaticscriptelementswiththenewdynamicparameterscollectedfromthescripttool.Inthismanner,thescripthasbecomeavaluabletoolthatcanbeusedformultipledataanalyses,asthefieldstobeanalyzedarenowdynamic:
importarcpy,csv
busStops=arcpy.GetParameterAsText(0)
censusBlocks2010=arcpy.GetParameterAsText(1)
censusBlockField=arcpy.GetParameterAsText(2)
csvname=arcpy.GetParameterAsText(3)
headers=arcpy.GetParameterAsText(4).split(',')
sql=arcpy.GetParameterAsText(5)
keyfields=arcpy.GetParameterAsText(6).split(';')
dataDic={}
censusFields=['BLOCKID10',censusBlockField,'SHAPE@']
if"SHAPE@"notinkeyfields:
keyfields.append("SHAPE@")
arcpy.AddMessage(busStops)
arcpy.AddMessage(censusBlocks2010)
arcpy.AddMessage(censusBlockField)
arcpy.AddMessage(csvname)
arcpy.AddMessage(sql)
arcpy.AddMessage(keyfields)
x=0
witharcpy.da.SearchCursor(busStops,keyfields,sql)ascursor:
forrowincursor:
stopid=x
shape=row[-1]
dataDic[stopid]=[]
dataDic[stopid].append(shape.buffer(400))
dataDic[stopid].extend(row[:-1])
x+=1
processedDataDic={}
forstopidindataDic.keys():
values=dataDic[stopid]
busStopBuffer=values[0]
blocksIntersected=[]
witharcpy.da.SearchCursor(censusBlocks2010,censusFields)ascursor:
forrowincursor:
block=row[-1]
population=row[1]
blockid=row[0]
ifbusStopBuffer.overlaps(block)==True:
interPoly=busStopBuffer.intersect(block,4)
data=population,interPoly,block
blocksIntersected.append(data)
processedDataDic[stopid]=values,blocksIntersected
dataList=[]
forstopidinprocessedDataDic.keys():
allValues=processedDataDic[stopid]
popValues=[]
blocksIntersected=allValues[-1]
forblocksinblocksIntersected:
pop=blocks[0]
totalArea=blocks[-1].area
interArea=blocks[-2].area
finalPop=pop*(interArea/totalArea)
popValues.append(finalPop)
averagePop=round(sum(popValues)/len(popValues),2)
busStopLine=allValues[0][1]
busStopID=stopid
finalData=busStopLine,busStopID,averagePop
dataList.append(finalData)
defcreateCSV(data,csvname,mode='ab'):
withopen(csvname,mode)ascsvfile:
csvwriter=csv.writer(csvfile,delimiter=',')
csvwriter.writerow(data)
headers.insert(0,"ID")
createCSV(headers,csvname,'wb')
fordataindataList:
createCSV(data,csvname)
ThevariablexwasaddedtokeeptrackofthemembersofthedictionarydataDic,whichinthescriptinChapter6,WorkingwithArcPyGeometryObjectshadreliedontheSTOPIDfield.Toeliminatethisdependency,xwasintroduced.
RunningtheScriptToolNowthatthefrontendhasbeendesignedtoacceptparametersfromauser,andthebackendscriptisreadytoaccepttheparametersfromthefrontend,thetoolisreadytobeexecuted.DoubleclickontheScriptToolinthetoolboxtoopenthetooldialogbox.
SelecttheparametersaswithanyArcToolboxtool(forexampleusingthefiledialogtonavigateafiletreetotheBusStopfeatureclass).Oncetheparametershavebeenset,clickonOKtoexecutethescript.
Oneoptionaladjustmentwouldbetoaddanarcpy.AddMessagelinewheretheaveragepopulationiscalculated.Bydoingthis,theindividualblockpopulationwouldbedisplayedandthescriptconsolewouldgivefeedbackabouttheprogressofthescript.
InsertinthescriptjustbelowthelinewherethevariablefinalDataisdefined:
arcpy.AddMessage(finalData)
Thefeedbackprovidedbythislinewillmakeitobviousthatthescriptisworking,whichisusefulwhenthescriptexecutesalonganalysis.Whenperforminglonganalyses,itisgoodpracticetoprovidefeedbacktotheusersothattheycanseethatthescriptisworkingasexpected.Passnewlinecharacters(\n)asparameterstoarcpy.AddMessagewhenthereisalargeamountofdatabeingpassedtoarcpy.AddMessage.Thiswillbreakupthedataintodiscretechunksandmakeiteasiertoread.
SummaryInthischapter,welearnedhowtoconvertascriptintoapermanentandsharablescripttoolthatcanbeusedbyanArcGISuserwithnoprogrammingexperience.BycreatingafrontendusingthefamiliarArcGIStoolinterface,andthenpassingparameterstocustombuilttoolsthatemployArcPy,GISprogrammerscancombinetheeaseofuseoftheArcToolboxwiththepowerofPython.
Inthenextchapter,wewillexplorehowtouseArcPytocontroltheexportofmapsfrommapdocuments.Byadjustingmapelementssuchastitlesandlegends,wecancreatedynamicmapoutputstoreflectthenatureofthedataproducedbymapanalysis.InChapter9,MoreArcpy.MappingTechniqueswewilladdtheoutputofmapstoourscripttoolcreatedinthischapter.
Chapter8.IntroductiontoArcPy.MappingCreatingmapsisanart,onethatcanbelearnedthroughyearsofdedicatedstudyofcartography.Thevisualdisplayofinformationisbothexcitinganddifficult,andcanbearewardingpartofthedailyworkflowofgeospatialprofessionals.Oncethebasicshavebeenlearnedandmastered,cartographicoutputbecomesaconstantbattletoproducemoremapsatafasterpace.ArcPy,onceagain,hasapowerfulsolution:thearcpy.mappingmodule.
Byallowingfortheautomaticmanipulationofallmapcomponents,includingthemapwindow,thelayers,thelegend,andalltextelements,arcpy.mappingmakescreating,modifying,andoutputtingmultiplemapsfastandsimple.Mapbookcreation–anotherimportantskillforgeospatialprofessionals,isalsomadeeasyusingthemodule.Inthischapterwewilldiscusssomebasicfunctionalitiesavailablethrougharcpy.mappinganduseittooutputamapbookofbusstopsandtheirsurroundingcensusblocks.
Thischapterwillcoverthefollowingtopics:
InspectingandupdatingMapDocument(MXD)layerdatasourcesExportingMXDstoPDForotherimageformatsAdjustingmapdocumentelements
UsingArcPywithmapdocumentsRecognizingthelimitationsofthepreviousarcgisscriptingmodule,ESRIdesignedtheArcPymoduletonotonlyworkwithdatabutalsoincludedthearcpy.mappingmoduletoallowdirectinteractionwithmapdocuments(MXDs)andthelayerstheycontain.Thisnewmoduleopenedupamultitudeofmapautomationpossibilities.Ascriptmightaidinidentifyingbrokenlayerlinks,updatethedatasourceoftheselayers,andapplynewcolorschemestolayers.Anotherscriptmightuseamaptemplateandcreateasetofmaps,onefromeachfeatureclassinafeaturedataset.AthirdscriptcouldcreateamapbookfromanMXD,movingfromcelltocellinagridlayertooutputthepagesofthebook,orevencalculatingthecoordinatesonthefly.Dynamicallycreatedmaps,basedondatafromafreshanalysis,canbeoutputtedatthesametimethedataisproduced.Arcpy.mappingmovestheArcPymodulefromhelpfultoinstrumental,inanygeospatialworkflow.
Toinvestigatetheutilityofthearcpy.mappingmodule,we’llneedthehelpofanMXDtemplate.I’vepreparedamappackagecontainingthedataandMXDthatwewillusefortheexercisesinthischapter.ItincludesthedatafromourSanFranciscobusstop’sanalysis,whichwewillcontinueandextendtoincludemaps.
InspectingandreplacinglayersourcesThefirstandmostimportantarcpy.mappingmoduleuseistoidentifyandfixthebrokenlinksbetweenlayersinamapdocumentandtheirdatasources.LayersymbologyandGISdatastorageareseparated,meaningthatlayerdatasourcesareoftenmoved.Arcpy.mappingoffersaquicksolution,thoughimperfect.
Thissolutiondependsonanumberofmethodsincludedinthearcpy.mappingmodule.First,wewillneedtoidentifythebrokenlinks,andthenwewillfixthem.ToidentifythebrokenlinkswewillusetheListBrokenDataSources()methodincludedinarcpy.mapping.
TheListBrokenDataSources()methodrequiresanMXDpathtobepassedtotheMapDocument()methodofarcpy.mapping.Oncethemapdocumentobjecthasbeencreated,itispassedtotheListBrokenDataSources()method,andalistwillbegeneratedcontaininglayerobjects,oneforeachlayerwithabrokenlink.Thelayerobjectshaveanumberofpropertiesavailabletothem.Usingtheseproperties,let’sprintoutthenameanddatasourceofeachlayerusingthenameanddatasourcepropertiesofeachobject:
importarcpy
mxdPath='C:\Projects\MXDs\Chapter8\BrokenLinks.mxd'
mxdObject=arcpy.mapping.MapDocument(mxdPath)
brokenLinks=arcpy.mapping.ListBrokenDataSources(mxdObject)
forlinkinbrokenLinks:
printlink.name,link.dataSource
FixingthebrokenlinksNowthatwehaveidentifiedthebrokenlinks,thenextstepistofixthem.Inthiscase,itwasrevealedthatthedatasourcesshouldbeinafoldercalledData,buttheyarenotcontainedwithinthatfolder.Thescriptmustthenbesteppeduptoreplacethedatasourcesofeachlayer,sothattheypointattheactuallocationofthedatasource.
Therearemethodsincludedinbothlayerobjectsandmapdocumentobjectsthatcanaccomplishthisnextstep.IfallofthedatasourcesforanMXDhavebeenmoved,itisbettertousetheMXDobjectanditsmethodstofixthesources.IntheexampleMXD,thedatasourceshaveallbeenmovedintoanewfoldercalledNewData,sowewillemploythefindAndReplaceWorkspacePaths()methodtorepairthelinks:
oldPath=r'C:\Projects\MXDs\Data'
newPath=r'C:\Projects'
mxdObject.findAndReplaceWorkspacePaths(oldPath,newPath)
mxdObject.save()
Aslongasthedatasourcesarestillinthesameformat(suchthatshapefilesarestillshapefilesorfeatureclassesarestillfeatureclasses),thefindAndReplaceWorkspacePaths()methodwillwork.Ifthedatasourcetypeshavebeenchanged(suchthat,shapefilesareimportedintoafilegeodatabase),thereplaceWorkspaces()methodwillhavetobeusedinsteadasitrequiresworkspacetypeasaparameter:
oldPath=r'C:\Projects\MXDs\Data'
oldType='SHAPEFILE_WORKSPACE'
newPath=r'C:\Projects'
newType='FILEGDB_WORKSPACE'
mxdObject.replaceWorkspaces(oldPath,oldType,newPath,newType)
mxdObject.save()
FixingthelinksofindividuallayersIftheindividuallayersdonotshareadatasource,thelayerobjectswillneedtobeadjustedusingthefindAndReplaceWorkspacePath()methodavailabletolayers.Thismethodissimilartothemethodusedpreviously,butitwillonlyreplacethedatasourceofthelayerobjectitisappliedtoinsteadofallofthelayers.Whencombinedwithadictionary,thelayerdatasourcescanbeupdatedusingthelayernameproperty:
importarcpy
layerDic={'Bus_Stops':[r'C:\Projects\OldDataPath',r'C:\Projects'],
'stclines_streets':[r'C:\Projects\OtherPath',r'C:\Projects']}
mxdPath=r'C:\Projects\MXDs\Chapter8\BrokenLinks.mxd'
mxdObject=arcpy.mapping.MapDocument(mxdPath)
brokenLinks=arcpy.mapping.ListBrokenDataSources(mxdObject)
forlayerinbrokenLinks:
oldPath,newPath=layerDic[layer.name]
layer.findAndReplaceWorkspacePath(oldPath,newPath)
mxdObject.save()
Thesesolutionsworkwellforindividualmapdocumentsandlayers.TheycanalsobeextendedtofoldersfullofMXDsbyusingtheglob.glob()methodofthebuilt-inglobmodule(whichhelpstogeneratealistoffilesthatmatchacertainfileextension)andtheos.path.join()methodoftheosmodule:
importarcpy,glob,os
oldPath=r'C:\Projects\MXDs\Data'
newPath=r'C:\Projects'
folderPath=r'C:\Projects\MXDs\Chapter8'
mxdPathList=glob.glob(os.path.join(folderPath,'*.mxd'))
forpathinmxdPathList:
mxdObject=arcpy.mapping.MapDocument(mxdPath)
mxdObject.findAndReplaceWorkspacePaths(oldPath,newPath)
mxdObject.save()
ExportingtoPDFfromanMXDThenextmostimportantuseofarcpy.mappingistoautomaticallyexportMXDs.ThefollowingcodewillhighlighttheexportofPDFs,butnotethatthemodulealsosupportstheexportofJPEGsandotherimageformats.Usingarcpy.mappingforthisprocessisajoy,astheusualprocessofopeningandexportingtheMXDsinvolvesalotofwaitingforArcMaptostartandthemaptoload,whichcanbeatimesink:
importarcpy,glob,os
mxdFolder=r'C:\Projects\MXDs\Chapter8'
pdfFolder=r'C:\Projects\PDFs\Chapter8'
mxdPathList=glob.glob(os.path.join(mxdFolder,'*.mxd'))
formxdPathinmxdPathList:
mxdObject=arcpy.mapping.MapDocument(mxdPath)
arcpy.mapping.ExportToPDF(mxdObject,
os.path.join(pdfFolder,
basepath(
mxdPath.replace('mxd','pdf')
)))
NoteNotethattheoutputfoldermustexistforthiscodetoruncorrectly.Whilethereareosmodulemethodstocheckwhetherapathexists(os.path.exists)andtocreateafolder(os.mkdir),thatisnotincludedinthiscodesnippetandthearcpy.mapping.ExportToPDF()methodwillthrowanexceptioniftheinputoroutputpathsdonotexist.
Thisexamplecodeisveryusefulandcanbeconvertedintoafunctionthatwouldacceptthefolderpathasaparameter.Thefunctioncouldthenbeaddedtoascripttool,asdiscussedinthelastchapter.
AdjustingmapdocumentelementsArcpy.mappingincludesimportantmethodsthatwillfacilitatetheautomationofmapdocumentmanipulation.TheseincludetheabilitytoaddnewlayersorturnlayersonandoffwithinMXDs,theabilitytoadjustthescaleofthedataframeormoveadataframetofocusonaspecificregion,andtheabilitytoadjusttextcomponentsofthemap(suchastitlesorsubtitles).Thesemethodswillbeaddressedaswecontinueourbusstopanalysis.
OpenuptheMXDcalledMapAdjust.mxd.Thisrepresentsourbasemapdocument,withlayersandelementsthatwewilladjusttoourneeds.Itcontainslayersthatwehavegeneratedfromouranalysis,andthebaselayersthatfilloutthemap.Therearealsoanumberoftextelementsthatwillbeautomaticallyreplacedbythescripttofitthespecificneedsofeachmap.However,itdoesnotdoagoodjobofrepresentingtheresultsoftheanalysisasthecensusblocksthatintersectthebusstopbuffersoverlap,makingithardtointerpretthecartography.
Thescriptwillreplacethedatasourceofthecensusblocklayerandthebusstoplayertomakeitpossibletoonlyproduceonemapforeachbusstop,andthecensusblocksthatareintersectedwitheachbuffersurroundingthestops.
Tomakethispossible,wewillhavetocreatetwoemptyfeatureclasses:one,withalloftheattributesofthecensusblocks,andtheother,withtheattributesofthebusstops.Thiswillallowthedatasourcetobereplacedwiththedataproducedbytheanalysis.
OpenuptheSanFrancisco.gdbFileGeodatabaseandrightclickontheChapter8Resultsfeaturedataset.SelectNewfromthedrop-downmenuandthenselectFeatureClass.NamethefirstfeatureclassSelectedCensusBlocksandmakeitapolygon.Selectthedefaultskeywordonthenextmenu,andthenonthefollowingmenu,pushtheimportbutton.SelecttheCensusBlocksfeatureclassfromtheSanFranciscofeaturedataset;thiswillloadthefieldsintothenewfeatureclass.DothesamethingforasecondfeatureclasscalledSelectedBusStops,butmakesurethatitisapointgeometrytypeandimportthe
schemafromtheBusStopsfeatureclass.RepeatthesameprocessforathirdfeatureclasscalledSelectedStopBuffers,butmakesurethatitisapointgeometrytypeandimporttheschemafromtheBuffersfeatureclass.
Oncethefeatureclasseshavebeencreated,itisnowpossibletousethemtoloadtheresultsoftheanalysis.Wewillberedoingtheanalysisinmemoryandwritingouttheresultstothenewlycreatedfeatureclasses,sothattheentirecensusblockwillbecaptured,insteadofonlytheportionthatintersectswiththebuffer,asitwillbetterillustratetheresultsoftheanalysis.
TheinitialstateoftheMapAdjust.mxdmapdocumentfeaturesanumberoffeatureclasseswithwhichwearenowfamiliar:thedownloadedfeatureclassBus_Stops,thegeneratedfeatureclassBuffers,theintersectedandclippedCensusBlocks,andfourfeatureclassesusedforcartographicpurposes,namelytheStreetsfeatureclass,theParksfeatureclass,aNeighborhoodsfeatureclass,andanoutlineofSanFrancisco.Therearetwodataframes,onewiththedefaultnameLayersandanothercalledInset,thatareusedtocreatethesmallinsetthatwillshowthepositionoftheLayersdataframeasitmovesaroundSanFrancisco.ThesmallrectanglethatdepictstheextentoftheLayersdataframeisanExtentframecreatedinthepropertiesoftheInsetdataframe.
Hereisanexportedviewoftheinitialstateofthemapdocument:
Theideahere,istousetheinitialresultsofouranalysistogeneratethesymbologyofthepopulationlayeraswellasthebusstoplayerandthebufferlayer.Oncetheyhavebeensetandsaved,theycanbeusedasabasisfortheindividualmappagesthatwewillbeproducingfromthisbasicmapdocument.
NoteNotethetextelementsthatmakeupthetitleandsubtitle,aswellasthelegendandattributiontextatthebottomoftherightpane.Theseelementsareavailableforadjustmentalongwiththelayersanddatasourcesthatmakeupthemapdocumentbyusingthearcpy.mapping.ListElements()method.
AutomatedmapdocumentadjustmentNowthatweunderstandtheinitialconfigurationofthemapdocument,wewillintroduceascriptthatwillautomatetheadjustment.Thisscriptwillincludeanumberofconceptsthatwehavecoveredinthischapterandearlierchapters,andwillalsointroducesomenewmethodsformapdocumentadjustmentsthatwewilldetailinthefollowing:
importarcpy,os
dirpath=os.path.dirname
basepath=os.path.basename
Bus_Stops=r"C:\Projects\SanFrancisco.gdb\Bus_Stops"
selectedBusStop=
r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedBusStop'
selectedStopBuffer=
r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedStopBuffer'
CensusBlocks2010=r"C:\Projects\SanFrancisco.gdb\CensusBlocks2010"
selectedBlock=
r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedCensusData'
pdfFolder=r'C:\Projects\PDFs\Chapter8\Map_{0}'
bufferDist=400
sql="(NAME='71IB'ANDBUS_SIGNAG='FerryPlaza')"
mxdObject=arcpy.mapping.MapDocument("CURRENT")
dataFrame=arcpy.mapping.ListDataFrames(mxdObject,"Layers")[0]
elements=arcpy.mapping.ListLayoutElements(mxdObject)
forelinelements:
ifel.type=="TEXT_ELEMENT":
ifel.text=='TitleElement':
titleText=el
elifel.text=='SubtitleElement':
subTitleText=el
arcpy.MakeFeatureLayer_management(CensusBlocks2010,'blocks_lyr')
layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)
layerStops=layersList[0]
layerCensus=layersList[1]
layerBuffer=layersList[2]
layerBlocks=layersList[3]
iflayerBlocks.dataSource!=selectedBlock:
layerBlocks.replaceDataSource(dirpath(dirpath(layerBlocks.dataSource)),
'FILEGDB_WORKSPACE',basepath(selectedBlock))
iflayerStops.dataSource!=selectedBusStop:
layerStops.replaceDataSource(dirpath(dirpath(layerStops.dataSource)),
'FILEGDB_WORKSPACE',basepath(selectedBusStop))
iflayerBuffer.dataSource!=selectedStopBuffer:
layerBuffer.replaceDataSource(dirpath(dirpath(layerBuffer.dataSource)),
'FILEGDB_WORKSPACE',basepath(selectedStopBuffer))
layerStops.visible=True
layerBuffer.visible=True
layerCensus.visible=False
witharcpy.da.SearchCursor(Bus_Stops,['SHAPE@','STOPID','NAME',
'BUS_SIGNAG','OID@','SHAPE@XY'],sql)
ascursor:
forrowincursor:
stopPointGeometry=row[0]
stopBuffer=stopPointGeometry.buffer(bufferDist)
witharcpy.da.UpdateCursor(layerBlocks,['OID@'])asdcursor:
fordrowindcursor:
dcursor.deleteRow()
arcpy.SelectLayerByLocation_management('blocks_lyr','intersect',
stopBuffer,"","NEW_SELECTION")
witharcpy.da.SearchCursor('blocks_lyr',['SHAPE@','POP10','OID@'])
asbcursor:
inCursor=arcpy.da.InsertCursor(selectedBlock,
['SHAPE@','POP10'])
fordrowinbcursor:
data=drow[0],drow[1]
inCursor.insertRow(data)
delinCursor
witharcpy.da.UpdateCursor(selectedBusStop,['OID@'])asdcursor:
fordrowindcursor:
dcursor.deleteRow()
inBusStopCursor=arcpy.da.InsertCursor(selectedBusStop,['SHAPE@'])
data=[row[0]]
inBusStopCursor.insertRow(data)
delinBusStopCursor
witharcpy.da.UpdateCursor(selectedStopBuffer,['OID@'])asdcursor:
fordrowindcursor:
dcursor.deleteRow()
inBufferCursor=arcpy.da.InsertCursor(selectedStopBuffer,
['SHAPE@'])
data=[stopBuffer]
inBufferCursor.insertRow(data)
delinBufferCursor
layerStops.name="Stop#{0}".format(row[1])
arcpy.RefreshActiveView()
dataFrame.extent=arcpy.Extent(row[-1][0]-1200,row[-1][1]-1200,
row[-1][0]+1200,row[-1][1]-1200)
subTitleText.text="Route{0}".format(row[2])
titleText.text="BusStop{0}".format(row[1])
outPath=pdfFolder.format(str(row[1])+"_"+str(row[-2]))+
'.pdf'
printoutPath
arcpy.mapping.ExportToPDF(mxdObject,outPath)
titleText.text='TitleElement'
subTitleText.text='SubtitleElement'
arcpy.RefreshActiveView()
Wow!That’salotofcode.Let’sreviewitsectionbysectiontoaddresswhateachpartofthescriptisdoing.
ThiscodewillberuninthePythonWindowoftheMXD,somakesuretoopentheMXD.Onceitis,openthePythonWindowandrightclickinit,andthenselectLoadfromtheright-clickmenu.Usingthefilenavigationbrowser,findthescriptcalledChapter8_6_AdjustmapCURRENT.pyandselectitbyclickingonit.PushOKanditwillloadinthePythonWindow.PushingEnterwillexecutethescript,orusethescrollbartoperusetheloadedlines.
ThevariablesWithinthescript,anumberofvariablesarefirstcreatedtoholdthestringfilepaths,theintegerbufferdistance,andthesqlstatementusedtoidentifythebuslineofinterest:
importarcpy,os
Bus_Stops=r"C:\Projects\SanFrancisco.gdb\Bus_Stops"
selectedBusStop=
r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedBusStop'
selectedStopBuffer=
r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedStopBuffer'
CensusBlocks2010=r"C:\Projects\SanFrancisco.gdb\CensusBlocks2010"
selectedBlock=
r'C:\Projects\SanFrancisco.gdb\Chapter8Results\SelectedCensusData'
pdfFolder=r'C:\Projects\PDFs\Chapter8\Map_{0}'
bufferDist=400
sql="(NAME='71IB'ANDBUS_SIGNAG='FerryPlaza')"
Thesewillbeusedlatertoallowustosearchthelayersandperformanalysisonthem.
ThemapdocumentobjectandthetextelementsBecausethiscodewillbeexecutedinanopenmapdocument,wedon’thavetopassanMXDfilepathtothearcpy.mapping.MapDocument()method.Instead,wewillusethekeywordCURRENTtoindicatethatwearereferencingtheopenmapdocument:
mxdObject=arcpy.mapping.MapDocument("CURRENT")
dataFrame=arcpy.mapping.ListDataFrames(mxdObject,"Layers")[0]
elements=arcpy.mapping.ListLayoutElements(mxdObject)
forelinelements:
ifel.type=="TEXT_ELEMENT":
ifel.text=='TitleElement':
titleText=el
elifel.text=='SubtitleElement':
subTitleText=el
Oncethemapdocumentobjecthasbeencreated,theLayersdataframeisselectedfromalistofdataframesusingtheListDataFrames()methodandpassedtothevariablecalleddataFrame.
Next,thelayoutelementsarepassedasalisttotheelementsvariableusingtheListLayoutElements()method.Thelayoutelementsincludethevariouselementsofthemapdocumentlayoutview:thelegend,theneatlines,thenortharrow,thescalebar,andthetextelementsusedastitlesanddescriptions.Unfortunately,thereisnoniceordertothelistreturned,astheirpositionthroughoutthelayoutisundetermined.Accesstothetextelements,whichwewouldliketoassigntoavariableforlateruse,mustbeidentifiedusingtwopropertiesoftheelementobjects:thetypeandthetext.Wewanttoadjustthetitleandsubtitleelements,soaforloopisusedtosearchthroughthelistofelementsandthepropertiesareusedtofindtheelementsofinterest.
ThelayerobjectsTheMakeFeatureLayertool,partoftheDataManagementtoolset,isusedtocopydatafromdiskintomemoryasalayer.ArcGISrequiresthegenerationoflayerstoperformselectionsandoperationsondata,insteadofoperatingonthefeatureclassesdirectly.Byusinglayerstoperformtheseoperations,thesourcefeatureclassesareprotected.
TheMakeFeatureLayertoolisaccessedusingArcPy’sMakeFeatureLayer_management()method.WhenusingthistoolinthePythonWindow,theresultisaddedtothemapdocumentasalayerthatwillbevisibleintheTableofContents.WhenthetoolisnotusedinthePythonWindowinArcMap,theresultinglayerisonlygeneratedinmemoryandisnotaddedtothemapdocument.
Intheportionofthefollowingcode,alayercalledblocks_lyrisgeneratedinmemorybypassingthefilepathofthecensusblocksfeatureclass.ThelayerobjectscontainedwithintheinitialMXDarethenaccessedusingtheListLayers()methodofthearcpy.mapping()module.TheyarereturnedintheorderthattheyarelistedintheTableofContentsofthemapdocumentandareassignedtovariablesusinglistindexing,includingthenewlycreatedblocks_lyr:
arcpy.MakeFeatureLayer_management(CensusBlocks2010,'blocks_lyr')
layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)
layerStops=layersList[0]
layerCensus=layersList[1]
layerBuffer=layersList[2]
layerBlocks=layersList[3]
ReplacingthedatasourcesNowthatwehaveassignedthelayerobjectstovariables,wewillcheckwhethertheirdatasourcesarethecorrectfeatureclassesthatweuseformapproduction.UsingthedataSourcepropertyofeachlayerobject,theyarecomparedtothefilepathvariablesthatwewanttouseasdatasources:
iflayerBlocks.dataSource!=selectedBlock:
layerBlocks.replaceDataSource(dirpath(dirpath(layerBlocks.dataSource)),
'FILEGDB_WORKSPACE',basepath(selectedBlock))
iflayerStops.dataSource!=selectedBusStop:
layerStops.replaceDataSource(dirpath(dirpath(layerStops.dataSource)),
'FILEGDB_WORKSPACE',basepath(selectedBusStop))
iflayerBuffer.dataSource!=selectedStopBuffer:
layerBuffer.replaceDataSource(dirpath(dirpath(layerBuffer.dataSource)),
'FILEGDB_WORKSPACE',basepath(selectedStopBuffer))
Ifstatementsareusedtocheckwhetherthedatasourcesarecorrect.Ifnot,theyarereplacedwiththecorrectdatasourcesusingthereplaceDataSource()layermethod.Thismethodrequiresthreeparameters:theworkspace(inthiscase,theFileGeodatabase),theworkspacetype,andthenameofthenewfeatureclassdatasource,whichmustbeinthesameworkspaceforthereplaceDataSource()methodtowork(thoughitdoesnotneedtobeinthesamefeaturedataset).
AdjustinglayervisibilityThelayerobjectshaveapropertythatallowsustoadjusttheirvisibility.SettingthisBooleanpropertytoTrueorFalsewilladjustthelayer’svisibilityon(True)oroff(False):
layerStops.visible=True
layerBuffer.visible=True
layerCensus.visible=False
WewantthelayervariablelayerCensus,whichisthenewblocks_lyrobject,tobeturnedoff,soitissettoFalse,butthebusstopsandbufferlayerobjectsneedtobevisible,sotheyaresettoTrue.
GeneratingabufferfromthebusstopsfeatureclassAllofthevariableshavebeengeneratedorassigned,sothenextstepistouseaSearchCursortosearchthroughtheselectedbusstops.Foreachbusstop,bufferobjectswillbegeneratedtofindcensusblocksthatintersectwiththeseindividualbusstops:
witharcpy.da.SearchCursor(Bus_Stops,['SHAPE@','STOPID','NAME',
'BUS_SIGNAG','OID@','SHAPE@XY'],sql)
ascursor:
forrowincursor:
stopPointGeometry=row[0]
stopBuffer=stopPointGeometry.buffer(bufferDist)
witharcpy.da.UpdateCursor(layerBlocks,['OID@'])as
dcursor:
fordrowindcursor:
dcursor.deleteRow()
ForeachrowofdataretrievedfromtheBusStopsfeatureclass,anumberofattributesarereturned,containedinatuple.Thefirstofthese,row[0],isaPointGeometryobject.ThisobjecthasabuffermethodthatisusedtogenerateabufferPolygonobjectinmemory,whichisthenassignedtothestopBuffervariable.Oncethebufferobjectiscreated,thedataaccessUpdateCursor’sdeleteRow()methodisusedtoerasetherowsinthecensusblockslayer.Oncetherowshavebeendeleted,thelayercanthenberepopulatedwithnewlyselectedcensusblocksthatwillbeidentifiedinthenextsection.
IntersectingthebusstopbufferandcensusblocksToidentifythecensusblocksintersectingwiththebufferaroundeachbusstop,theArcToolboxtoolSelectLayerByLocationisinvokedusingtheArcPymethodSelectLayerByLocation_management():
arcpy.SelectLayerByLocation_management('blocks_lyr','intersect',
stopBuffer,"","NEW_SELECTION")
witharcpy.da.SearchCursor('blocks_lyr',['SHAPE@',
'POP10','OID@'])asbcursor:
inCursor=arcpy.da.InsertCursor(selectedBlock,['SHAPE@',
'POP10'])
fordrowinbcursor:
data=drow[0],drow[1]
inCursor.insertRow(data)
delinCursor
Thismethodrequiresthein-memoryblocks_lyrlayerobjectandthenewlycreatedbufferobjectassignedtothevariablestopBuffer.Italsorequiresthetypeofselectionintersectandanotherparameterthatcontrolswhethertheselectionwillbeaddedtoanexistingselectionorwillbeanewselection.Inthiscase,wewantanewselection,asonlythecensusblocksthatintersectthecurrentbusstopareneeded.
Oncethecensusblockshavebeenselectedandidentified,theshapedataandpopulationdataispassedtothefeatureclassrepresentedbythevariableselectedBlockusinganInsertCursor.TheInsertCursormustbedeletedusingthedelkeyword,asonlyoneInsertCursororUpdateCursorcanbeinmemoryatatime.
PopulatingtheselectedbusstopandbufferfeatureclassesInasimilarmanner,thenextstepistopopulatethebusstopandbufferfeatureclassesthatwillbeusedinthemapproduction.ThebusstopsfeatureclassisfirstmadeblankusingthedeleteRow()method,andthentheselectedbusstopshapefielddataisinsertedintothefeatureclass.Thesamestepsarethentakenwiththebusstopbuffersfeatureclassandthebuffergeometryobject:
witharcpy.da.UpdateCursor(selectedBusStop,['OID@'])asdcursor:
fordrowindcursor:
dcursor.deleteRow()
inBusStopCursor=arcpy.da.InsertCursor(selectedBusStop,['SHAPE@'])
data=[row[0]]
inBusStopCursor.insertRow(data)
delinBusStopCursor
witharcpy.da.UpdateCursor(selectedStopBuffer,['OID@'])asdcursor:
fordrowindcursor:
dcursor.deleteRow()
inBufferCursor=arcpy.da.InsertCursor(selectedStopBuffer,['SHAPE@'])
data=[stopBuffer]
inBufferCursor.insertRow(data)
delinBufferCursor
UpdatingthetextelementsNowthatthedatahasbeengeneratedandwrittentothefeatureclassescreatedtoholdthem,thenextstepistoupdatethelayoutelements.Thisincludeslayerpropertiesthatwillaffectthelegend,theextentofthedataframe,andthetextelements:
layerStops.name="Stop#{0}".format(row[1])
dataFrame.extent=arcpy.Extent(row[-1][0]-1200,row[-1][1]-1200,
row[-1][0]+1200,row[-1][1]-1200)
subTitleText.text="Route{0}".format(row[2])
titleText.text="BusStop{0}".format(row[1])
arcpy.RefreshActiveView()
Thenameofthebusstopslayerisadjustedusingitsnamepropertytoreflectthecurrentbusstop.Thedataframeextentisadjustedbycreatinganarcpy.Extentobjectandpassingitfourparameters:Xmin,Ymin,Xmax,Ymax.TogeneratethesevaluesIhaveusedthesomewhatarbitraryvalueof1200feettocreateasquarearoundthebusstop.Thetextelementsareupdatedusingtheirtextproperty.Finally,theRefreshActiveView()methodisusedtoensurethatthemapdocumentwindowiscorrectlyupdatedtothenewextent.
ExportingtheadjustedmaptoPDFThefinalstepistopassthenewlyadjustedmapdocumentobjecttoArcPy’sExportToPDFmethod.Thismethodrequirestwoparameters,themapdocumentobjectandastringthatrepresentsthefilepathofthePDF:
outPath=pdfFolder.format(str(row[1])+"_"+str(row[-2]))+'.pdf'
arcpy.mapping.ExportToPDF(mxdObject,outPath)
titleText.text='TitleElement'
subTitleText.text='SubtitleElement'
arcpy.RefreshActiveView()
ThePDFfilepathstringisgeneratedfromthepdfFolderstringtemplateandtheIDofthebusstop,alongwiththeobjectIDandthefileextension.pdf.OncethatandthemapdocumentobjectrepresentedbythevariablemxdObjectarepassedtotheExportToPDFmethod,thePDFwillbegenerated.Thetextelementsarethenresetandtheviewisrefreshedtoensurethatthemapdocumentwillbereadyforthenexttimethescriptisused.
RunningthescriptinthePythonWindowOpenupthemapdocumentcalledMapAdjust.mxdifitisnotopenalready.OpenthePythonWindowandrightclickinthewindow.SelectLoadfromthemenu.Whenthefiledialogopens,findthescriptcalledChapter8_6_AdjustmapCURRENT.pyandselectit,makingsurethatthefilepathswithinitarecorrect.PushOKanditwillloadinthePythonWindow.PushEnteroncethescriptisloadedtorunthescript.Itcantakeafewsecondsormoreforittobeobviousthatthescriptisrunning.
NoteNotethatthePythonWindowisnotagreatplacetoexecuteArcPyscriptsinmostcases,asitissomewhatlimitedwhencomparedtoIDEs.UsingittoloadandexecuteascriptthatperformsthesemapdocumentadjustmentsisoneofthebestusesofthePythonWindow.
Oncethescriptisrunning,theadjustmentstothemapdocumentwillbegintoappearandrepeat.Thisisafascinatingprocess,astheeffectsofrunningthescriptarevisibleinamannerthatisnotreadilyavailablewhenrunningPythonscripts.OncethePDFsbegintobegenerated,openoneuptoviewtheoutput.Thescriptwillgenerateamapforeachbusstopontheselectedbusline,sofeelfreetoshutdownthemapdocumentaftergeneratingasetnumberofthePDFs.
Hereisanexampleoftheoutput:
Themapsgeneratedbythescriptshoweachbusstopatthecenter,surroundedbythebufferandthesymbolizedcensusblockswithwhichthebufferintersects.Thetitle,subtitleandthelegendhavebeenadjustedtoindicatethebusstopdepictedinthemap.WithArcPy,wearenowincontrolofboththepartsofgeospatialanalysis:theanalysisitself,andthecartographicproductiondepictingtheresultoftheoutput.
SummaryInthischapterarcpy.mappingwasintroducedandusedtocontroltheelementsofmapdocumentsthatneedtobeadjustedtocreatecustommaps.Byjoininggeospatialanalysisandmapproductiontogether,weareclosertoutilizingthefullpowerofArcPy.
Inthenextchapter,wewillgofurtherwitharcpy.mappingandcreateascripttoolthatcanbeaddedtoArcToolbox,whichwillruntheanalysisaswellasgeneratemapsfromtheresultingdata.WewillalsorefinethescriptandintroduceDataDrivenPagestodiscusshowthatpowerfultoolcanbeusedinanArcPyscript.
Chapter9.MoreArcPy.MappingTechniquesTheabilitytocontrolmapdocumentcartography,whilealsorunninggeospatialanalyses,increasesthepowerandusefulnessofArcPy.Thepropertiesandmethodsofarcpy.mappingcanbeutilizedtomanipulatelayerobjects,mapscalesanddataframeextents,oreventosetdefinitionqueries.Bycombiningautomatedgeospatialanalysiswithdynamicmapproduction,scriptedmappingsystemsaremadepossible.Thischapterwillcoverthefollowingtopics:
Arcpy.mappingLayerobjectsLayerobjectdefinitionqueriesandextentsArcpy.mappingDataFrameobjectsCreatingdynamicallyscaledmaps
Usingarcpy.mappingtocontrolLayerobjectsArcpy.mappingLayerobjectsareusedtocontrolthepropertiesoflayerswithinmapdocumentdataframes.Turninglayervisibilityonandoff,addingnewlayers,andadjustinglayerordercanallbeaccomplishedusingLayerobjectproperties.
CreatingLayerobjectsinvolvespassingparameterstothearcpy.mapping.ListLayers()method.AsdiscussedinChapter8,IntroductiontoArcPy.Mapping,whenreferencinganarcpy.mapping.MapDocumentobject,thelayerswithinthemapdocumentcanbeaccessedusingzero-basedindexing.ThiscodewillprintthelistofLayerobjectscontainedwithinthedataframecalledLayersinanMXD:
importarcpy
mxdPath=r'C:\Projects\MXDs\Chapter9\MapDocument1.mxd'
mxdObject=arcpy.mapping.MapDocument(mxdPath)
dataFrame=arcpy.mapping.ListDataFrames(mxdObject,"Layers")[0]
layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)
printlayersList
ThelayerswithinthedataframecalledLayers,havebeenassignedtothevariablelayersListusingtheListLayers()method.EachlayerinlayersListcanbeaccessedusingzero-basedindexing.Oncethelayershavebeenaccessedwithinthelistandeitherassignedtoavariableorplacedinsideaforloop,thepropertiesandmethodsoftheLayerobjectscanbeutilized.
NoteThesecondparameteroftheListLayersmethodisemptyhere,butdoesnothavetobe.ItisawildcardparameterthatwilllimitthereturnedLayerobjectstothosethatmatchthepatternofthewildcard.Forinstance,*StopswouldreturnalllayerswiththenameStopsattheend.Multipleasteriskscanbeusedtofindlayerswiththewordatthebeginning,middle,orendofthelayername.
LayerobjectmethodsandpropertiesLayerobjectpropertiesandmethodscaneitherbereadonly,meaningtheycanbecheckedbutnotadjusted,ortheyarereadandwrite,meaningtheycanbeadjustedwithinthescript.Let’sexploreanumberofthesepropertiesandmethods,andseehowtheycanbeusedtocontrolthelookandfeelofthemapsproducedfromthemapdocument,aswellasthedatafromthescriptanalysis.
DefinitionqueriesAnimportantpropertyofLayerobjectsistheabilitytodynamicallysetdefinitionqueries.AdefinitionqueryisaSQLstatementwhereclausethatlimitsthedataavailablefordisplay,query,orotherdataoperations(buffers,intersections,etc.)toonlytherowsthatmatchthewhereclause.DefinitionqueriescouldbesetinanMXDbyopeningalayer’spropertiesmenuandusingtheDefinitionQuerytab,buthereweareconcernedwithhowtoaddthemprogrammatically.Followingisanexampleofhowtodothis:
layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)
busStops=layersList[0]
busStops.definitionQuery="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
ThisvaluablepropertycanbeutilizedtoreformatthecodefromChapter8,IntroductiontoArcPy.Mapping.RememberthecomplicatedsecondportionoftheChapter8_6.pyscript,whereeachbusstopalongthe71Inboundlineisselectedanditsgeometryiswrittentoanotherfeatureclass?Instead,wecanuseLayerobjectsanddefinitionqueriestoperformthesametypeofgeometryoperation.Let’sexaminehowthefirstpartofthatoperation(selectingthebusstopgeometryandcreatingabufferaroundit)lookswhenadefinitionqueryisused:
importarcpy
bufferDist=400
mxdPath=r'C:\Projects\MXDs\Chapter9\MapDocument1.mxd'
mxdObject=arcpy.mapping.MapDocument(mxdPath)
dataFrame=arcpy.mapping.ListDataFrames(mxdObject,"Layers")[0]
layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)
busStops=layersList[0]
defQuery="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
busStops.definitionQuery=defQuery
idList=[]
witharcpy.da.SearchCursor(busStops,['OID@'])ascursor:
forrowincursor:
idList.append(row[0])
foroidinidList:
newQuery="OBJECTID={0}".format(oid)
printnewQuery
busStops.definitionQuery=newQuery
witharcpy.da.SearchCursor(busStops,
['SHAPE@','STOPID','NAME','BUS_SIGNAG','OID@','SHAPE@XY'])ascursor:
forrowincursor:
stopPointGeometry=row[0]
stopBuffer=stopPointGeometry.buffer(bufferDist)
Inthisexample,thedefinitionqueryisusedtolimitthepotentialresultsfromtheSearchCursortothebusstopspecifiedbythequery.However,thisisoverlycumbersomeandthedefinitionquerydoesn’taddmuch,asfirstanotherSearchCursorisneededtoextracttheObjectIDinformationfromthebusStopslayer.ThiscomplicatesthecodewhenonlyoneSearchCursorisnecessary.
Definitionqueriesshouldbeusedtoselecttheblocksthatintersectwiththebuffer,asthis
willeliminatetheneedtousethecomplicatedSearchCursorandInsertCursorsetupthatwasemployedinChapter8,IntroductiontoArcPy.Mapping.Let’sreformulatethecodesothatdefinitionqueriesareproperlyusedonthecensusblockLayerobject.
ThefirststepistoaddsomecodethatwillgeneratetheSQLstatementthatwillbeusedasthedefinitionquery:
importarcpy
bufferDist=400
mxdPath=r'C:\Projects\MXDs\Chapter9\MapDocument1.mxd'
mxdObject=arcpy.mapping.MapDocument(mxdPath)
dataFrame=arcpy.mapping.ListDataFrames(mxdObject,
"Layers")[0]
layersList=arcpy.mapping.ListLayers(mxdObject,"",dataFrame)
busStops=layersList[0]
censusBlocks=layersList[3]
sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
witharcpy.da.SearchCursor(busStops,['SHAPE@','STOPID','NAME',
'BUS_SIGNAG','OID@'],sql)ascursor:
forrowincursor:
busQuery='OBJECTID={0}'.format(row[-1])
busStops.definitionQuery=busQuery
stopPointGeometry=row[0]
stopBuffer=stopPointGeometry.Buffer(bufferDist)
arcpy.SelectLayerByLocation_management(censusBlocks,'intersect',stopBuffer,
"","NEW_SELECTION")
blockList=[]
witharcpy.da.SearchCursor(censusBlocks,
['OID@'])asbcursor:
forbrowinbcursor:
blockList.append(brow[0])
newQuery='OBJECTIDIN('forCOUNTER,oidinenumerate(blockList):
ifCOUNTER<len(blockList)-1:
newQuery+=str(oid)+','
else:
newQuery+=str(oid)+')'
printnewQuery
Inthissection,thecodeassignsthecensusblockslayerintheMXDtothevariablecensusBlocks.ThebusstopsSearchCursoristhencreated,andthe400footbufferisgeneratedforeachrowtoselectthecensusblockssurroundingthebusstop.Oncethecorrectblockshavebeenselected,asecondSearchCursorisusedonthecensusBlocksLayerobjecttofindtheObjectID(usingtheOID@token)oftheselectedblocks.TheObjectIDsarethenappendedtothelistcalledblockList.
ThislististheniteratedinaforlooptogenerateastringSQLstatement.UsingtheinitialstringassignedtothevariablenewQuery,theforloopwilladdtheObjectIDsofeachselectblocktothestringtocreateavalidSQLstatement.Theforloopusesthefunctionenumeratetocountthenumberofloopsthattheforloopperforms;thisallowsforanif/thenstatementtobeused.Theif/thenstatementdetermineswhatcomesaftertheObjectIDinthestring,aseachObjectIDmustbeseparatedbyacomma,exceptforthefinalObjectID,whichmustbefollowedbytheclosingparenthesis.Theforloopproduces
aSQLstatementsimilartothisexample:
OBJECTIDIN(910,1664,1812,1813,2725,6382)
Theprintstatementattheendisusedtodemonstratetheresultsofthissectionofthecode,andalsotogivethatwarmfuzzyfeelingthatcomesfromseeingtheresultsofthecodeworking.OncewearesurethatthecodeisgeneratingvalidSQLstatements(closedparenthesisandcommaseparatedObjectIDs),thenextstepistoassignthedefinitionquerytothecensusBlocksLayerobjectandusetheresulttogenerateamapofthearea.
ControllingthedataframewindowextentandscaleInChapter8,IntroductiontoArcPy.Mappingwestartedtoexplorethepropertiesandmethodsofthedataframe.Usingthearcpy.Extentobject,wewereabletosettheextentofthedataframetoanextentthatwashard-codedintothescript.However,thisdoesnotalwayscapturetheentireextentoflargecensusblocks.Usingacombinationofdefinitionqueriesandthedataframeextentandscaleproperties,wecanavoidtheseunwantedresults.
Therearetwodataframeobjectmethodsusedtoshiftthedataframewindowtotheareaofinterest,inthiscasetheselectedcensusblocks.Thefirst,whichwearenotusinghere,isdataFrame.zoomToSelectedFeatures.Thesecond,istoassignthedataframe’sextentpropertytotheextentofthecensusblocklayerafterthedefinitionqueryhasbeenassignedtoit.
Ipreferthesecondmethod,asitwillworkevenwhenthereisnoselectedcensusblocks.Also,asthemapsthatareproducedbythisscriptshouldnotshowtheselectionoftheblocks,wewillhavetoaddcodetoexplicitlycleartheselectiononcethecorrectcensusblockshavebeenidentified:
censusBlocks.definitionQuery=newQuery
dataFrame.extent=censusBlocks.getExtent()
arcpy.SelectLayerByAttribute_management(censusBlocks,
"CLEAR_SELECTION")
Thedefinitionqueryhasmadeiteasytomovethedataframewindowtotheareaofinterest,astheextentrectangle(orenvelope)ofthelayerisnowonlyaroundthespecifiedblocksandthedataFrameextentpropertycanbesettotheextentrectangle.However,thisisnotalwayscartographicallydesirableasitseemsbettertomovethedataframewindowbackfromtheextentrectangle.Todothat,we’llaccessthedataframetheobject’sscaleproperty.
Thescalepropertycanbesettobeamultiplierofthecurrentscaletoavoidhard-codinganyspecificdistanceswhenadjustingthedataframeextent.Whenusingthescaleproperty,itisimportanttoremembertousethearcpy.RefreshActiveView()method,asitwillrefreshthedataframewindowtothenewscale.
dataFrame.scale=dataFrame.scale*1.1
arcpy.RefreshActiveView()
Asthedataframeextentwassetinthefewlinesbeforethis,thecurrentscalerepresentstheenvelopeoftheselectedcensusblocks.Toadjustit,assessthepropertyandapplyamultiplier.Inthiscase,themultiplieris1.1,butitcouldbeanyvalue.Thismakestheresultingmaplookbetterbygivingtheanalysisresultssomebackgroundcontext.
AddingaLayerobjectThelaststepbeforeexportingoutthemapsistoaddthe400footbufferscreatedaboveasalayertothedataframeobject.Toaccomplishthis,weneedtocreateasymbolizedlayeraheadoftimeandcopyitssymbologytoensureitlooksasdesired.ThiswillbeaddedtotheMXDasaplaceholderlayer,andassignedtothebufferLayervariableinthescript.
1. OpenupanMXDandaddthebusstopfeatureclass.2. RuntheBufferToolintheProximitytoolsetintheAnalysistoolsetofthe
ArcToolbox,addingthebusstopfeatureclassastheinputandsettingthebuffersizeto400feet.Afterthetoolhasrun,openthepropertiesofthebufferlayerandsymbolizethelayerasdesired.
3. Oncethelayerhasbeensymbolized,right-clickonthelayerandselectSaveAsLayerFile.
4. SavethelayerinafolderandclosetheMXD.5. OpenuptheMapDocument1.mxdmapdocumentandaddthelayerusingtheAdd
Databutton.6. Makesuretochangethenameto400FootBufferandtoaddittothelegendabove
thePopulationsection.7. Inthescript,assignthebufferlayertothevariablebufferLayer.8. Lowerinthescript,inthebusstopSearchCursor,addtheselinesbelowwherethe
bufferisgeneratedaroundthebusstopgeometry:
arcpy.CopyFeatures_management(stopBuffer,
r"C:\Projects\Output\400Buffer.shp")
bufferLayer.replaceDataSource(r"C:\Projects\Output","SHAPEFILE_WORKSPAC
E","400Buffer")
ThesetwolinescopythebuffergeneratedtodiskasashapefileandthenreplacethedatasourceofthebufferLayerLayerobjectwiththenewlycreatedbuffer.Notethatthenameoftheshapefiledoesnotincludethe.shpextension;theSHAPEFILE_WORKSPACEparametermakesthisunnecessary.
NoteTomakesurethateachnewbuffershapefilecanbewrittenoveranexistingshapefile,addthefollowinglinebelowtheimportarcpylinetomakesurethatfilescanbeoverwritten:
arcpy.env.overwriteOutput=1
ExportingthemapsThefinalstepofthisscriptistoexportthemapsoftheareasurroundingeachbusstop.Todothis,wewillborrowsomecodefromthescriptChapter8_6_AdjustMap.pyandaddthewholescripttoafilecalledChapter9.py.Thiscodewillidentifyandadjustthetitleandsubtitleelements,makingitpossibletocustomizeeachresultingPDF:
importarcpy
arcpy.env.overwriteOutput=1
bufferDist=400
pdfFolder=r'C:\Projects\PDFs\Chapter9\Map_{0}'
mxdPath=r'C:\Projects\MXDs\Chapter9\MapDocument1.mxd'
mxdObject=arcpy.mapping.MapDocument(mxdPath)
dataFrame=arcpy.mapping.ListDataFrames(mxdObject,"Layers")[0]
elements=arcpy.mapping.ListLayoutElements(mxdObject)
forelinelements:
ifel.type=="TEXT_ELEMENT":
ifel.text=='TitleElement':
titleText=el
elifel.text=='SubtitleElement':
subTitleText=el
layersList=arcpy.mapping.ListLayers(mxdObject,
"",dataFrame)
busStops=layersList[0]
bufferLayer=layersList[2]
censusBlocks=layersList[4]
sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
witharcpy.da.SearchCursor(busStops,['SHAPE@',
'STOPID',
'NAME',
'BUS_SIGNAG',
'OID@'],sql)ascursor:
forrowincursor:
busQuery='OBJECTID={0}'.format(row[-1])
busStops.definitionQuery=busQuery
stopPointGeometry=row[0]
stopBuffer=stopPointGeometry.buffer(bufferDist)
arcpy.CopyFeatures_management(stopBuffer,r"C:\Projects\Output\400Buffer.shp
")
bufferLayer.replaceDataSource(r"C:\Projects\Output",
"SHAPEFILE_WORKSPACE",
"400Buffer")
arcpy.SelectLayerByLocation_management(censusBlocks,
'intersect',
stopBuffer,
"",
"NEW_SELECTION")
blockList=[]
witharcpy.da.SearchCursor(censusBlocks,
['OID@'])asbcursor:
forbrowinbcursor:
blockList.append(brow[0])
newQuery='OBJECTIDIN('
forCOUNTER,oidinenumerate(blockList):
ifCOUNTER<len(blockList)-1:
newQuery+=str(oid)+','
else:
newQuery+=str(oid)+')'
printnewQuery
censusBlocks.definitionQuery=newQuery
dataFrame.extent=censusBlocks.getExtent()
arcpy.SelectLayerByAttribute_management(censusBlocks,
"CLEAR_SELECTION")
dataFrame.scale=dataFrame.scale*1.1
arcpy.RefreshActiveView()
subTitleText.text="Route{0}".format(row[2])
titleText.text="BusStop{0}".format(row[1])
outPath=pdfFolder.format(str(row[1]))+'.pdf'
printoutPath
arcpy.mapping.ExportToPDF(mxdObject,outPath)
titleText.text='TitleElement'
subTitleText.text='SubtitleElement'
censusBlocks.definitionQuery=''
busStops.definitionQuery=''
SummaryInthischapter,wecoveredtheuseoflayerdefinitionqueries,dataframeextentsandscales,andlayersourcereplacementtoeasetheproductionofmaps.Byusingdefinitionqueries,thelayerscanbemodifiedtonewextents,makingiteasiertozoomintothelayerextentandtosetthescaleofthedataframe.Thedefinitionqueriesalsolimitwhichmembersofalayeraredisplayedwithinthedataframe.Layersourcereplacementwasusedasacartographiccontrol,allowingustopre-generatethestyleofalayerandadjustthedatathatitrepresenteddynamically.
Inthenextchapter,wewillcombinethelessonsfromthelastthreechapters,allowingustocreateascripttoolthatwillrunanalysisandproducespreadsheetsandmapsfromtheanalysisresults.
Chapter10.AdvancedGeometryObjectMethodsInthischapter,wewilldiscussadvancedGeometryobjectmethods,previouslydiscussedinChapter6,WorkingwithArcPyGeometryObjects.ThegoalofthisbookistogiveanintroductiontoArcPyanditsmodules,whilealsodemonstratinghowtoapplythesetoolswhencreatingenduringGISworkflows.Performingananalysisonceisgood,butdoingitoverandover,withtheclickofabutton,isbetter.Makingtheanalysisresultssharableinanindustrystandardformatisalsodesirable.IntheArcGISworld,thebestwaytodothisiswithArcPyandscripttoolsthattakeadvantageofGeometryobjectmethods.
Thischapterwillcoverthefollowingtopics:
AddingcommonfunctionstoamoduleinthePythonpathMakingtheanalysismoreadvancedbyaddingpointgenerationAdvancedPolygonobjectmethodsUsingtheXLWTtocreateExcelspreadsheets
CreatingaPythonmoduleAnimportantsteptowardscreatingreusablecodeistopackageitscomponentfunctionsintoamodulethatcanbecalledfromthePythonpathbyanyscript.Tostart,weneedtocreateafolderinthesite-packagesfolderwherePythonmodulesareplacedwhendownloadedandextractedusingthePythonmoduleprocess,orwhenrunningthesetup.pyscriptincludedwithsharedmodules.
Modulespackagetogetherfunctionsinoneormorescriptsintoafolderthatcanbesharedwithothers(thoughtheyoftendependonothermodulestorun).Wehaveusedsomeofthebuilt-inmodulessuchasthecsvmoduleandthird-partymodulessuchasArcPy.Let’sexploretheirconstructiontogetafeelofhowamoduleispackagedforuseandsharing.
NoteManymodulesarenotplacedwithinthesite-packagesfolder,buttheyrequirethePythonpathtobemodifiedtomakethemimportable.Placingmoduleswithinthesite-packagesfoldereliminatesthisrequirement.
Openupthesite-packagesfolderinWindowsExplorerbynavigatingtoC:\Python27\ArcGIS10.2\Lib\site-packages(orC:\Python27\Lib\site-packagesifyou’reusingthestandardPython2.7installation)folder.Onceinthefolder,createanewfoldercalledcommon,asshowninthefollowingscreenshot:
The__init__.pyfileWithinthisfolder,aspecialfileneedstobeaddedtoletPythonrecognizethefolderasamodule.Thisfile,called__init__.py,takesadvantageofthespecialpropertyofPythoncalledmagicobjectsorattributesthatarebuiltintoPython.Thesemagicobjectsusetheleadingandtrailingdoubleunderscoretoavoidanyconfusionwithcustomfunctions.
NoteNotethatthesearedoubleunderscores;singleunderscoresareusuallyusedforso-calledprivatefunctionswithincustomPythonclasses.
The__init__.pyfileisusedtoindicatethatthefolderisamodule(makingitimportableusingtheimportkeyword),andtoinitiatethemodulebycallinganymodulesthatitmayinturnrelyon.However,thereisnorequirementtoaddimportcommandstothe__init__.pyfile;itcanbeanemptyfileandwillstillperformthemodulerecognitionfunctionalitythatwerequire.
1. OpenupIDLEorAptanaoryourfavoriteIDE,andinthefoldercalledcommon,addanewPythonfileandcallit__init__.py.Thisfilewillremainemptyfornow.
2. Nowthatwehaveinitiatedthemodule,weneedtocreateascriptthatwillholdourcommonfunctions.Let’scallituseful.pybecausethesefunctionswillbemostusefulforthisanalysisandothers.
3. Thenextstepistotransferfunctionsthatwehadcreatedinearlierchapters.Thesevaluablefunctionsarelockedintothosescripts,sobyaddingthemtouseful.py,wewillmakethemavailabletoallotherscriptswecraft.
NoteOneimportantfunctionistheformatSQLMultiplefromChapter4,ComplexArcPy
ScriptsandGeneralizingFunctions,whichgeneratesSQLstatementsusingatemplateandalistofdata.Byaddingittouseful.py,wewillbeabletocallthefunctionanytimeaSQLstatementisrequired.
4. OpenthescriptChapter4Modified2.pyandcopythefunction,andthenpasteitintouseful.py.Ithasnodependencies,soitdoesnothavetobemodified.
AnotherusefulfunctionfromthatscriptistheformatIntersectfunctionthatgeneratesastringoffilepathsthatareusedwhenrunningtheArcToolboxIntersecttool.WhilewehavereacheddeeperintoArcPysincethatfunctionwasdesigned,andnolongerneedtocalltheIntersecttoolinourbusstopanalysis,itdoesnotmeanthatwewillneverneedtocallitinthefuture.Itisstillusefulandshouldbeaddedtouseful.py.
ThelastfunctionthatwecanraidisthecreateCSV()function.CopyandpasteitfromChapter4Modified.pyintouseful.py.However,toavoidtheneedtoimporttheCSVmoduleseparately,wewillneedtomodifythefunctionslightly.Hereishowitshouldlook:
defcreateCSV(data,csvname,mode='ab'):
'createsacsvfile'
importcsv
withopen(csvname,mode)ascsvfile:
csvwriter=csv.writer(csvfile,delimiter=',')
csvwriter.writerow(data)
delcsv
Byimportingandthendeletingthecsvmodule,weareabletouseittogeneratethecsvfileandthenremovethemodulefrommemoryusingthedelkeyword.
Nowthatwehavethefunctionswewillbereusingsavedintheuseful.pyscript,insidethecommonmodule,let’sexplorehowtocallthemusingPython’simportmethod.OpenupaPythonexecutable,usingeitherPython.exeorIDLE,orthebuilt-interminalinAptana.Atthetriplechevronprompt(>>>),writethefollowingline:
>>>fromcommon.usefulimportcreateCSV>>>
Ifthesecondtriplechevron-shapedpromptappears,thefunctionwascorrectlyimportedfromthemodule.Toimportthefunctionsinthismoduleinascript,usethesameimportstructureandlistthefunctionsdesired,separatingthemusingacomma:
fromcommon.usefulimportcreateCSV,formatSQLMultiple
Thefunctionsinthescriptuseful.pywerecalledusingPythondotnotation.Thisismadepossiblebecausethe__init__.pyfileindicatestoPythonthatthefoldercommonisnowamodule,andthatitshouldexpectamethodcalledusefultobepresent,withthefunctionscreateCSVandformatSQLMultipleinsideit.
AddingadvancedanalysiscomponentsThebusstopanalysiswehaveusedtointroduceArcPycanbefurtherextendedtogeneratemorerefinedresults.Tobetterestimatethetruenumberofpeoplethateachbusstopserves,let’saddafunctionthatwillgeneraterandompointswithintheblocksconsidered,whileeliminatingparksandotherareasthatdonotcontainhousing.
Todothis,weneedtointroduceanewdatasetfromtheSanFranciscogeodatabase,theRPD_Parksfeatureclass.Byusingthisfeatureclasstoreducetheareaconsideredforouranalysis,wecangenerateamorerealisticassessmentoftheserviceareapopulationforeachbusstop.
WhileusingtheArcToolboxErasetooltoerasethearearepresentedintheRPD_Parkspolygonswouldbeausualstepwhenrunningaspatialanalysis,therearedrawbackstothisoption.ThefirstisthattheErasetoolisonlyavailablewiththeArcGISforDesktopAdvancedlicenselevel,makingitavailableonlytocertainusers.Theseconddrawbackisthatthetoolproducesanintermediatedataset,somethingtobeavoidedwhereverpossible.
UsingArcPywillgiveustheabilitytoavoidbothofthesedrawbacks.WecancreateascriptthatwillgeneraterandompointsonlywithinthefractionofthecensusblockpolygonsthatdonotintersectwiththeRPD_Parksfeatureclass.Todothis,wewillreachdeeperintothemethodsoftheArcPyPolygonobject.
AdvancedPolygonobjectmethodsInChapter6,WorkingwithArcPyGeometryObjectswestartedexploringtheArcPyGeometryobjectsandhowtousetheirmethodstoperformin-memoryspatialanalysis.TheBufferandIntersectmethodsoftheseobjectswereintroducedandusedtogenerateanalysisresults.Next,wewilldiscussmoreofthesemethodsandshowhowtheycanhelpimprovein-memoryspatialanalysis.
ThePolygonobjecthasamethodcalledDifferencethatallowsustofindtheareaofnon-intersectionwhentwopolygonsintersect.Passingacensusblockpolygonandaparkpolygonasparameterswillreturn(asapolygonobject)thefractionofthefirstparameterwherenooverlapoccurs.AnotherimportantmethodiscalledOverlaps,whichiscalledtotestwhethertwoGeometryobjects(points,lines,orpolygons)intersect.Ifthereisanoverlap,theOverlapsmethodwillreturnTrue,whilereturningFalseifthereisnooverlapbetweenthetwoobjects.Unionisalsoanimportantmethodthatwillbeusedwithinthischapter,itallowsfortwoGeometryobjectstobeunionedintooneobject.
Let’sexploretheseimportantmethods.Tofindthenon-intersectareaoftwopolygonobjects,thefollowingfunctioncombinestheOverlapsandDifferencemethods:
defnonIntersect(poly1,poly2):
'returnsareaofnon-intersectbetweentwopolygons'
ifpoly1.overlaps(poly2)==True:
returnpoly1.difference(poly2)
ThefunctionnonIntersectacceptstwoPolygonobjectsasparameters.Thefirstparameter,poly1,isthepolygonofintersect(thecensusblockpolygon)andthesecondparameter,poly2,isthepolygontobecheckedforoverlap.TheifconditionalusestheOverlapsmethodandreturnsTrueifthereisanoverlapbetweenthetwoparameters.Ifthereisanyoverlap,thedifference()methodreturnsthenon-intersectareaasapolygonobject.However,thisfunctionshouldbeextendedtocoversituationswheretheOverlaps()methodreturnsFalse:
defnonIntersect(poly1,poly2):
'returnsareaofnon-intersectbetweentwopolygons'
ifpoly1.overlaps(poly2)==True:
returnpoly1.difference(poly2)
else:
returnpoly1
ThefunctionwillnowreturnthefirstparameterwhentheOverlapsmethodreturnsFalse,indicatingthatthereisnooverlapbetweenthetwopolygonobjects.Thisfunctionisnowcompleteandavailabletobeusedinananalysis.BecausenonIntersect()isafunctionthatcanbeusedinotherspatialanalyses,copyitandaddittouseful.py.
GeneratingrandompointstorepresentpopulationThenextsteptoimprovethebusstopanalysisistogeneratepointstorepresentthepopulationofeachcensusblock.Whilerandompointswillnotprovideaperfectrepresentationofthepopulation,itwillserveasagoodmodelofthepopulationandallowustoavoidareaaveragingtofindtheroughpopulationofeachcensusblockservedbyabusstop.TheCreateRandomPointstoolintheArcToolboxDataManagementtoolsetmakesitsimpletogeneratethepoints.
TheCreateRandomPointstoolacceptsanumberofrequiredandoptionalparameters.Asthetoolgeneratesafeatureclass,therequiredparametersaretheworkspacewherethefeatureclasswillbeplacedandthenameofthefeatureclass.Theoptionalparametersofinterestaretheconstrainingfeatureclassandthenumberofpointstobegenerated.Aswearelookingtoavoidcreatingnewfeatureclassesintheintermediatestepsofouranalysis,wecanutilizethein_memoryworkspace,whichallowsfeatureclassestobegeneratedinmemory,meaningtheyarenotwrittentotheharddrive.
Becausethereisaneedtogenerateaspecificnumberofrandompointsforeachcensusblock,weshouldcreateafunctionthatwillacceptaconstrainingpolygonandpopulationfigurethatrepresentseachcensusblock.Thein_memoryworkspacewon’tworkforeverysituation,however,sowe’llprovidetheworkspaceparameterwithadefaultvalue:
defgeneratePoints(fc,pop,constrant,workspace='in_memory'):
'generaterandompoints'
importos,arcpy
arcpy.CreateRandomPoints_management(workspace,fc,
constrant,"",pop,"")
returnos.path.join(workspace,fc)
Thefunctionwillcreatethefeatureclassintheworkspacedesiredandwillreturnthepath(joinedusingtheosmodule)tothefeatureclassforuseintherestofthescript.Thisfunctionisalsoreusableandshouldbecopiedintouseful.py.
UsingthefunctionswithinascriptNowthatwehavecreatedthefunctionsthatwillhelpustorunamoreadvancedspatialanalysis,let’saddthemtoascriptalongwithsomeSearchCursorstoiteratethroughthedata:
#Importthenecessarymodules
importarcpy,os
fromcommon.usefulimportnonIntersect,generatePoints,createCSV
#Addanoverwritestatement
arcpy.env.overwriteOutput=True
#Definethedatainputs
busStops=r'C:\Projects\SanFrancisco.gdb\SanFrancisco\Bus_Stops'
parks=r'C:\Projects\SanFrancisco.gdb\SanFrancisco\RPD_Parks'
censusBlocks=
r'C:\Projects\SanFrancisco.gdb\SanFrancisco\CensusBlocks2010'
csvName=r'C:\Projects\Output\Chapter10Analysis.csv'
#Createthespreadsheetinmemoryandaddfieldheaders
headers='LineName','StopID','TotalPopulationServed'
createCSV(headers,csvName,mode='wb')
#Copythecensusblockdataintoafeaturelayer
arcpy.MakeFeatureLayer_management(censusBlocks,'census_lyr')
#CopytheparkdatageometriesintoalistandunionthemallparkGeoms=
arcpy.CopyFeatures_management(parks,arcpy.Geometry())
parkUnion=parkGeoms[0]
forparkinparkGeoms[1:]:
parkUnion=parkUnion.union(park)
#Createasearchcursortoiteratethebusstopdata
sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
witharcpy.da.SearchCursor(busStops,['NAME','STOPID','SHAPE@'],sql)as
cursor:
forrowincursor:
lineName=row[0]
stopID=row[1]
stop=row[2]
busBuf=stop.buffer(400)
#Selectcensusblocksthatintersectthebusbuffer
arcpy.SelectLayerByLocation_management("census_lyr","intersect",
busBuf,'','NEW_SELECTION')
#UseasecondCursortofindtheselectedpopulation
totalPopulation=0
witharcpy.da.SearchCursor("census_lyr",['SHAPE@','POP10',
'BLOCKID10'])asncursor:
fornrowinncursor:
block=nrow[0]
checkedBlock=nonIntersect(block,parkUnion)
blockName=nrow[2]
population=nrow[1]
ifpopulation!=0:
points=generatePoints("PopPoints",
population,checkedBlock)
pointsGeoms=
arcpy.CopyFeatures_management(points,arcpy.Geometry())
pointsUnion=pointsGeoms[0]
forpointinpointsGeoms[1:]:
pointsUnion=pointsUnion.union(point)
pointsInBuffer=busBuf.intersect(pointsUnion,1)
intersectedPoints=pointsInBuffer.pointCount
totalPopulation+=intersectedPoints
#Addthetallieddatatothespreadsheet
data=lineName,stopID,totalPopulation
print'datawritten',data
createCSV(data,csvName)
#Startthespreadsheettoseetheresults
os.startfile(csvName)
Let’sreviewthecode,sectionbysection,asthatisalottotakeinatfirst.
Theimportportioniswherewecalltheusualmodules,arcpyandos,alongwithourcustomfunctionsinthecommonmodule:
importarcpy,os
fromcommon.usefulimportnonIntersect
fromcommon.usefulimportgeneratePoints
fromcommon.usefulimportformatSQLMultiple
fromcommon.usefulimportnonIntersectcreateCSV
Asdiscussedpreviously,thefunctionsinthecommonmodule’susefulmethodarecalledusingthePythondotnotationandthefrom…import…importationstyle,makingthemavailabledirectly.Manyfunctionscanbeimportedononeline,separatedbycommas,orindividuallyasshownhere.
Thenextline,whichsetstheArcPyEnvironmentoverwritepropertytoTrue,isveryimportantbecauseitallowsustooverwritetheresultsoftheCreaterandompointsoperation.Iftheresultswerenotoverwritten,thefunctionresults,whichotherwisewoulduseallavailablememoryandcausethescripttofail:
arcpy.env.overwriteOutput=True
NoteItisimportanttobecarefulwiththisoverwritesettingbecauseitwillallowforanyfeatureclasstobeoverwritten.Allofouroutputisinmemoryandonlygeneratedfortheanalysis,sothereislittleneedtoworryhere,buttakecaretomakesurethatnothingimportantisoverwrittenwhenrunningascript.
Thenextportionisthesetofvariablesthatwillbeusedinthisscript,andwillinitiatethespreadsheetthatwillbeusedtocollecttheresultsoftheanalysis:
busStops=r'C:\PacktDB.gdb\SanFrancisco\Bus_Stops'
parks=r'C:\PacktDB.gdb\SanFrancisco\RPD_Parks'
censusBlocks=r'C:\PacktDB.gdb\SanFrancisco\CensusBlocks2010'
csvName=r'C:\Projects\Output\Chapter10Analysis.csv'
headers='LineName','StopID','TotalPopulationServed'
createCSV(headers,csvName,mode='wb')
ThefilepathsassignedtovariablesherecouldbereplacedwithArcPyparametersifweweretoturnthisintoascripttool,butfornow,thehard-codedpathsarefine.Belowthevariables,theresultsspreadsheetiscreatedandthecolumnfieldheadersareadded.
Itisworthnotingthatthespreadsheetiscreatedusingthewbmode.Thismodeofbinaryfileopening,knownaswb(writebinary),isusedforcreatinganewfile.ItmustbeexplicitlypassedintothecreateCSV()functionasthedefaultmodeparameterisab(appendbinary),whichwillcreateanewfileifitdoesnotexist,oraddtoonethatalreadyexists(athirdbinarymodeisrborreadbinary,whichisusedforopeninganexistingfile).
Thenextfewlinesmakedatainthefeatureclassesavailableinmemory.ThecensusblockdataisconvertedintoaFeatureLayer,whiletheRPD_ParksdataisreadintomemoryasalistofPolygonobjectsthatisthenunionedintoasingle,unifiedPolygonobjectcalledparkUnion:
arcpy.MakeFeatureLayer_management(censusBlocks,'census_lyr')parkGeoms=
arcpy.CopyFeatures_management(parks,arcpy.Geometry())
parkUnion=parkGeoms[0]
forparkinparkGeoms[1:]:
parkUnion=parkUnion.union(park)
ByusingtheCopyFeaturestoolintheDataManagementtoolset,theparkGeomsvariableispassedalistofthegeometriesforeachrowofdataintheRPD_Parksfeatureclass.However,wedon’twanttohavetoiteratethroughtheparkgeometriestocomparethemtoeachcensusblock,sotheUnionmethodisinvokedtocreateonePolygonobjectfromtheentirelist.ByassigningthefirstmemberofthelisttotheparkUnionvariable,andtheniteratingthroughtheparkGeomslisttouniontheothergeometriesonebyone,theresultisonePolygonobjectthatrepresentsallparkswithintheRPD_Parksdataset.
Onceallofthemoduleshavebeenimportedandthevariableshavebeenassigned,wecanentertheforloopofthedataaccessSearchCursortobegintheanalysis.However,wedon’twanttorunthisforallofthebusstops,sowewilluseaSQLstatementwhereclause,tolimittheanalysistoasinglebusline:
sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
witharcpy.da.SearchCursor(busStops,['NAME','STOPID','SHAPE@'],sql)as
cursor:
forrowincursor:
lineName=row[0]
stopID=row[1]
stop=row[2]
busBuf=stop.buffer(400)
arcpy.SelectLayerByLocation_management("census_lyr","intersect,busBuf,'','N
EW_SELECTION')
totalPopulation=0
Thefirstportionoftheiterationinvolvesenteringtheforloopandassigningthevaluesofeachrowtoavariable.APolygonobjectbufferof400feetiscreatedaroundthePointGeometryobjectreturnedbytheSearchCursor.ThisbufferisthenusedtointersectwiththecensusblocksFeatureLayertofindandselectallofthecensusblocksthatintersectthebuffer.Totallythepopulationservedbyeachbuffer,thevariabletotalPopulationiscreated.
Oncetheselectionhasbeenperformed,asecondSearchCursorcanbeusedtoiteratethroughtheselectedblockstoretrievetheirpopulationvaluesandPolygonobjectsforrandompointgeneration:
witharcpy.da.SearchCursor("census_lyr",['SHAPE@','POP10',
'BLOCKID10'])asncursor:
fornrowinncursor:
block=nrow[0]
checkedBlock=nonIntersect(block,parkUnion)
blockName=nrow[2]
population=nrow[1]
Inthisiteration,onceeachcensusblockhasbeenretrieved(intheformofaPolygonobject),theblockisthencheckedagainsttheunionedparkgeometryusingthenonIntersectfunctioncreatedpreviously.Thisensuresthatthepointswillonlybecreatedwithinareasthatarenotparks,thatis,morelikelytorepresentwherepeoplewouldlive.Thepopulationvaluesarealsoretrieved.
Oncetheconstrainingpolygon(forexamplethecensusblock)hasbeenevaluatedandanypotentialparkportionhasbeenremoved,andthepopulationvalueisavailable,therandompointscanbegeneratedusingthegeneratePoints()function:
ifpopulation!=0:
points=generatePoints("PopPoints",population,checkedBlock)
pointsGeoms=arcpy.CopyFeatures_management(points,arcpy.Geometry())
pointsUnion=pointsGeoms[0]
forpointinpointsGeoms[1:]:
pointsUnion=pointsUnion.union(point)
pointsInBuffer=busBuf.intersect(pointsUnion,1)
intersectedPoints=pointsInBuffer.pointCount
totalPopulation+=intersectedPoints
ThegeneratePoints()functionrequiresthreeparameters.Thefirstisthenameofthefeatureclasstobegenerated;thiswillbeoverwritteneachtimeitisgenerated,thusavoidingtheoveruseofmemorybycreatinganin_memoryfeatureclassforeachcensusblock.TheothertwoparametersarethepopulationvalueandtheconstrainingPolygonobject.
Oncethesehavebeenpassedtothefunction,itreturnsafilepathtothenewlycreatedfeatureclassandassignsthefilepathtothevariablepoints.ThegeometriesinpointsarethenextractedusingtheCopyFeaturestoolandassignedtothevariablepoints.TheUnionmethodisagainusedtocreateasingle,unifiedpopulationPointGeometryobject
thatwillbeintersectedwiththebusstopbuffer.Oncethisintersectionhasbeenrun,theresultinggeometriesareassignedtothepointsInBuffervariableandthepointCountmethodisusedtofindthenumberofpointsthatweregeneratedwithinthebufferedarea.Thisisourestimateofpopulationwithinthecensusblock,andthisvalueisaddedtothetotalPopulationvariabletoeventuallyyieldthetotalestimatedpopulationwithin400feetofthebusstop.
ThefinallinesofthescriptdemonstratehowthedataiscollectedintoatupleandpassedtothecreateCSV()moduletobewrittentoourfinalspreadsheet:
data=lineName,stopID,totalPopulation
print'datawritten',data
createCSV(data,csvName)
os.startfile(csvName)
Thelastline,os.startfile(csvName),usesthestartfilemethodoftheosmoduletoautomaticallyopenthespreadsheetoncetheanalysisiscompleted.Inthiscase,thespreadsheetC:\Projects\Output\Chapter10Analysis.csvhasbeenpopulatedwiththeresultsoftheanalysisandisopenedtodisplaytheseresults.However,theusermayhavetoindicatethatthelinesarecommaseparatedvaluestoopenthescript.
Insteadofcreatingacommaseparatedvalue,wecantakeadvantageofanotherPythonmodulethatisinstalledwhenArcGIS10.2andArcPyisinstalled.Thismodule,calledXLWT,isusedtogenerateExcelspreadsheets,andalongwiththeExcelspreadsheetreadingmoduleXLRD,isoneofthemostusefulmodulesavailabletousersofPython.
CreatinganXLSusingXLWTXLWTisapowerfulmodulethatallowsforamultitudeofstylingoptions.However,forourpurposeswecanignorethoseoptionsandcreateafunctionthatwillgenerateaspreadsheetwiththeresultsofourspatialanalysis.Thisfunctioncanofcoursebeaddedtocommon.useful:
defgenerateXLS(indatas,sheetName,fileName):
importxlwt
workbook=xlwt.Workbook()
sheet=workbook.add_sheet(sheetName)
forYCOUNTER,datainenumerate(indatas):
forXCOUNTER,valueinenumerate(data):
sheet.write(YCOUNTER,XCOUNTER,value)
workbook.save(fileName)
Thisfunctionrequiresthreeparameters,indatas-alistcontainingrowsofiterabledata,astringsheetname,andastringfilenamethatendswiththe.xlsextension.
Tousethisfunction,addittocommon.useful.Onceithasbeenadded,copyandrenametheolderanalysisscriptsothatitcanbeadjusted:
importarcpy,os
fromcommon.usefulimportnonIntersect,generatePoints,generateXLS
arcpy.env.overwriteOutput=True
busStops=r'C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops'
parks=r'C:\Projects\PacktDB.gdb\SanFrancisco\RPD_Parks'
censusBlocks=r'C:\Projects\PacktDB.gdb\SanFrancisco\CensusBlocks2010'
xlsName=r'C:\Projects\Output\Chapter10Analysis.xls'
headers='LineName','StopID','TotalPopulationServed'
indatas=[headers]
arcpy.MakeFeatureLayer_management(censusBlocks,'census_lyr')parkGeoms=
arcpy.CopyFeatures_management(parks,arcpy.Geometry())
parkUnion=parkGeoms[0]
forparkinparkGeoms[1:]:
parkUnion=parkUnion.union(park)
sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
witharcpy.da.SearchCursor(busStops,['NAME','STOPID',
'SHAPE@'],sql)ascursor:
forrowincursor:
lineName=row[0]
stopID=row[1]
stop=row[2]
busBuf=stop.buffer(400)
arcpy.SelectLayerByLocation_management("census_lyr","intersect",busBuf,'','
NEW_SELECTION')
totalPopulation=0
witharcpy.da.SearchCursor("census_lyr",['SHAPE@','POP10',
'BLOCKID10'])asncursor:
fornrowinncursor:
block=nrow[0]
checkedBlock=nonIntersect(block,parkUnion)
blockName=nrow[2]
population=nrow[1]
ifpopulation!=0:
points=
generatePoints("PopPoints",population,checkedBlock)
pointsGeoms=arcpy.CopyFeatures_management(points,arcpy.Geometry())
pointsUnion=pointsGeoms[0]
forpointinpointsGeoms[1:]:
pointsUnion=pointsUnion.union(point)
pointsInBuffer=busBuf.intersect(pointsUnion,1)
intersectedPoints=pointsInBuffer.pointCount
totalPopulation+=intersectedPoints
data=lineName,stopID,totalPopulation
indatas.append(data)
generateXLS(indatas,"Results",xlsName)
os.startfile(xlsName)
WecannowgenerateExcelspreadsheetsjustaseasilyaswehavegeneratedCSVfileswhileemployingareusablefunction.Wenowhavetheabilitytoperformrepeatablespatialanalysisfastandcanproduceresultsinindustrystandardformats.
SummaryInthischapter,wehaveexploredhowtocreatemodulesandreusablefunctionsthatwillsavescriptingtimeinthefuturebyallowingustoavoidrewritingtheseusefulfunctions.WefurtherexploredthemethodsavailablethroughArcPyGeometryobjects,includingtheIntersect,Overlaps,andUnionmethods.Wecreatedaspatialanalysisthatwritesnofeatureclassestodisk,makingitsothattheanalysistimeisreducedandunnecessaryfilesareavoided.Finally,weexploredhowtogenerateExcelspreadsheetsusingtheXLWTmodulesothatanalysisresultscanbesharedinindustrystandardformats.
Inthenextchapter,wewillexplorehowtouseArcPytointeractwiththeArcGISfordesktopextensionssuchasNetworkAnalystandSpatialAnalyst.Byincorporatingtheirfunctionalitywithinascript,wefurtherincreaseourabilitytocreatefastandrepeatablespatialanalysisworkflows.
Chapter11.NetworkAnalystandSpatialAnalystwithArcPyUseoftheArcGISforDesktopextensionsalsobenefitsfromthepowerofPythonandArcPy.TheabilitytomodelroutesusingastreetsdatasetorabusroutesdatasetusingArcPywillhelpusconvertentireworkflowsintoscripttools.BothNetworkAnalystsandSpatialAnalystshaveaccessmodulesbuiltintoArcPyforimprovedcontroloftheiravailabletools,methods,andproperties.
Thischapterwillcoverthefollowingtopics:
CreatingasimplenetworkdatasetCheckingouttheextensionsTheArcPyNetworkAnalystmoduleTheArcPySpatialAnalystmodule
TheNetworkAnalystextensionTheESRI’sNetworkAnalystextensionisapowerfultooltoenableroutingandnetworkconnectivityfunctionalitywithinArcGIS.Theextension,whenusedforstreetrouting,allowsuserstofindthequickestpathbetweentwopointsalongaroadnetwork.Theroutecanbeconstrainedbyanumberoffactors,suchastrafficorleftturns,tobettermodelroadtravel.Similaranalysiscanberunusingothertypesofnetworks,suchaswaterpipenetworksorelectricalnetworks.
UsingNetworkAnalystTousetheNetworkAnalystextension,theArcGISforDesktopAdvancedlicenseisrequired.InArcCatalogorArcMap,clickontheCustomizemenuandselectExtensions.OncetheExtensionsmenuisopen,clickonthecheckboxnexttoturnontheNetworkAnalystExtension.
CreatingaFeatureDatasetThefirststeptousinganetworkdatasetistocreateonewithinafeaturedataset.Todoso,wewillgenerateafeaturedatasettoholdthedataofinterest.Right-clickontheFilegeodatabasethathousestheBusStopdataandselectNew,andthenselectFeatureDatasetfromtheNewmenu.NameitChapter11ResultsandclickonNext.
Next,selecttheSpatialReferenceSystem(SRS).Inthiscase,wewillbeusingtheSRSofthelocalStatePlanezoneforSanFrancisco.Itisaprojectedcoordinatesystem,soselectthatfolder,andthenclickontheStatePlanefolder.Onceitisopened,selectthefoldercalledNAD1983(USFeet).Fromtheavailablereferencesystems,selecttheone
calledNAD1983StatePlaneCaliforniaIIIFIPS0403(USFeet).ClickonNexttogotothenextmenu.
NoteThissystemisalsoknownas2227inWellKnownID(WKID)orEuropeanPetroleumSurveyGroup(EPSG)systems.Moreinformationaboutthesecodesisavailableathttp://spatialreference.org,awebsiteusedtofindthethousandsofspatialreferencesystemsusedthroughouttheworld.
ClickontheVerticalCoordinateSystemsfolderandthenselecttheNorthAmericafolder.SelecttheNorthAmericanVerticalDatumof1988infeet(NAVD1988USsurveyfeet).Thiswillmakeitpossibletohavetheverticalandhorizontallinearunitsinthesamemeasurementsystem.ClickonNexttogotothenextmenu.
Thetolerancesonthenextpagearealsoveryimportant,butwewillnotcoverthemindetailhere.AcceptthedefaultsandclickonFinishtofinalizetheFeatureDataset.
ImportingthedatasetsImportthebusstops,streets,andbusroutesfeatureclassesintotheChapter11ResultsFeatureDataset.Right-clickonthedatasetandselectImport,andthenFeatureClass(Single).AddthefeatureclassesonebyonetogivethemanewnamethatwillkeepthemseparatedfromtheversionscontainedwithintheSanFranciscoFeatureDataset.ImportingthemwillmakesurethattheyareinthecorrectSRSandthatanetworkdatasetcanbecreated.
CreatingtheNetworkDatasetNowthatwehaveadatacontainer,wecancreateanetworkdatasetfromthestreetsfeatureclass.Right-clickontheChapter11ResultsfeaturedatasetandselectNew,andthenchooseNetworkDataset.
CalltheNetworkDatasetStreet_NetworkandclickonNext.SelecttheStreetsfeatureclassastheclassthatwillparticipateinthenetworkdatasetandclickonNexttomovetothenextmenu.SelectGlobalTurnstomodelturnswithinthenetwork.Inthenextmenu,usethedefaultconnectivitysettings.Then,accepttheUsingZCoordinateValuesfromGeometrysetting.Acceptthedefaultcostrestrictionanddrivingdirectionssettings,andfinallyclickonFinishtogeneratethenetworkdataset.Then,buildthenetworkdatasetusingthefinalmenu.Thenetworkdatasetisreadytobeused.
AccessingtheNetworkDatasetusingArcPyNowthatthenecessarysetuphasbeencompleted,thestreet_networknetworkdatasetcanbeaddedtoascriptforuseingeneratingroutes.Becausethisisasimpleanalysis,theonlyimpedancevaluetobeusedwillbethelengthofthestreetsegments.ThroughtheuseofaSearchCursor,PointGeometryobjectsfromthebusstopscanbeaccessedandaddedaslocationstobesearched:
importarcpy
arcpy.CheckOutExtension("Network")
busStops=r'C:\Projects\PacktDB.gdb\Chapter11Results\BusStops'
networkDataset=r'C:\Projects\PacktDB.gdb\Chapter11Results\street_network'
networkLayer="streetRoute"
impedance="Length"
routeFile="C:\Projects\Layer\{0}.lyr".format(networkLayer)
arcpy.MakeRouteLayer_na(networkDataset,
networkLayer,impedance)
print'layercreated'
sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
witharcpy.da.SearchCursor(busStops,['SHAPE@','STOPID'],sql)ascursor:
forrowincursor:
stopShape=row[0]
printrow[1]
arcpy.AddLocations_na(networkLayer,'Stops',stopShape,"","")
arcpy.Solve_na(networkLayer,"SKIP")
arcpy.SaveToLayerFile_management(networkLayer,routeLayerFile,"RELATIVE")
print'finished'
BreakingdownthescriptLet’sdissectthescript,whichoncefinished,willgeneratealayerfilecontainingtheaddedStops,andtheRoutesalongstreetstobestgetfromtheoriginstoptothedestinationstop.
ThescriptbeginsbyimportingthearcPymodule.ThenextlineallowsustousetheNetworkAnalystextension:
arcpy.CheckOutExtension("Network")
Usingthearcpy.CheckOutExtension()methodtoinvoketheNetworkAnalystextensioninvolvespassingthecorrectkeywordtothemethodasaparameter.Onceithasbeeninvoked,thetoolsoftheextensioncanbecalledandexecutedinthescript.
Assigningthebusstopsfeatureclassandthestreet_networknetworkdatasettovariables,theycanthenbepassedtoArcPy’sMakeRouteLayer_na()method,alongwithavariablerepresentingtheimpedancevalue:
arcpy.MakeRouteLayer_na(networkDataset,
networkLayer,impedance)
TheMakeRouteLayer_natoolproducesaRouteLayerinmemory.Thisblanklayerneedstobepopulatedwithstopstoproducetheroute(s)betweenthem.Forthispurpose,weneedaSearchCursortoaccessthePointGeometryobjectsandaSQLstatementthatwilllimitthereturnedresultstothelineofinterest:
sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
witharcpy.da.SearchCursor(busStops,['SHAPE@','STOPID'],sql)ascursor:
forrowincursor:
stopShape=row[0]
printrow[1]
arcpy.AddLocations_na(networkLayer,'Stops',stopShape,"","")
TheSearchCursorwillallowtheStopssublayerofthelayerproducedbytheMakeRouteLayertooltobepopulatedwhenusedinconjunctionwiththeAddLocationstool.Oncepopulated,theRouteLayercanbepassedtotheSolvetooltofindtheroutesbetweenthepointsofinterest.Again,theroutesaresolvedbasedonfindingthelowestimpedancebetweenthetwopoints.Inthisexample,theonlyimpedanceisthesegmentlength,butitcouldbetrafficorelevationorotherrestrictiontypes,ifthatdataisavailable:
arcpy.Solve_na(networkLayer,"SKIP")
arcpy.SaveToLayerFile_management(networkLayer,routeLayerFile,"RELATIVE")
ThefinalresultisalayerfilethatiswrittentodiskusingtheSaveToLayerFiletool.
TheNetworkAnalystmoduleInanefforttomaketheuseoftheNetworkAnalystextensionmorePythonic,thenewerNetworkAnalyst(na)moduleadjustshowthemethodsthatcorrespondtotheArcToolboxNetworkAnalysttoolsareaccessed.InsteadofcallingthetoolsdirectlyfromArcPy,thetoolsarenowmethodsofthenamodule.RemovingtheinitialsoftheNetworkAnalysttoolsetalsoreducesconfusionandmakesiteasiertorememberthenameofthemethod.Seethedifferenceasfollows:
importarcpy
arcpy.CheckOutExtension("Network")
busStops=r'C:\Projects\SanFrancisco.gdb\SanFrancisco\Bus_Stops
networkDataset=
r'C:\Projects\SanFrancisco.gdb\Chapter11Results\street_network'
networkLayer="streetRoute"
impedance="Length"
routeLayerFile="C:\Projects\Layer\
{0}_2.lyr".format(networkLayer)arcpy.na.MakeRouteLayer(networkDataset,
networkLayer,impedance)
print'layercreated'
sql="NAME='71IB'ANDBUS_SIGNAG='FerryPlaza'"
witharcpy.da.SearchCursor(busStops,['SHAPE@','STOPID'],sql)ascursor:
forrowincursor:
stopShape=row[0]
printrow[1]
arcpy.na.AddLocations(networkLayer,'Stops',stopShape,"","")
arcpy.na.Solve(networkLayer,"SKIP")
arcpy.management.SaveToLayerFile(networkLayer,routeLayerFile,"RELATIVE")
print'finished'
Thetoolwillproducethesamelayeroutputastheoriginalscript,butthereorganizationoftheNetworkAnalysttoolsintothenamodulehasmadethecodemorelogical.Forinstance,itmakesmoresensetocallSolveusingarcpy.na.Solve(),insteadofarcpy.Solve_na(),asitreinforcesthatSolveisamethodoftheNetworkAnalyst(na)module.AsArcPycontinuestobedeveloped,IexpectmorePythoniccodereorganizationtooccur.
AccessingtheSpatialAnalystExtensionTheSpatialAnalystExtensionisveryimportanttoperformanalysisonbothrasterandvectordatasets,butitisgenerallyusedtoperformsurfaceanalysisandrastermath.TheseoperationsaremadeeveneasierbytheuseofArcPy,asallofthetoolsavailableintheSpatialAnalystToolboxareexposedwiththeSpatialAnalystaccessmodule.ThisincludestheRasterCalculatortools,makingmapalgebraeasybyusingthetoolsandoperatorsinsimpleexpressions.
AddingelevationtothebusstopsTheelevationraster“sf_elevation”hasbeendownloadedfromNOAAandaddedtotheFileGeodatabase.However,itcoverstheentireBayArea,andweshouldwriteascripttoonlyextractanareaofthecityofSanFranciscoasitwillreducethetimeneededtorunourscripts.We’lluseaSQLstatementasthewhereclausetolimittheresultstotheSouthofMarket(SoMa)neighborhood.Todoso,let’stakeadvantageofaSearchCursorandtheSpatialAnalystaccessmodule’sExtractbyPolygonproperty:
importarcpy
arcpy.CheckOutExtension("Spatial")
busStops=r'C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops'
sanFranciscoHoods=
r'C:\Projects\PacktDB.gdb\SanFrancisco\SFFind_Neighborhoods'
sfElevation=r'C:\Projects\PacktDB.gdb\sf_elevation'
somaGeometry=[]
sql="name='SouthofMarket'"
witharcpy.da.SearchCursor(sanFranciscoHoods,['SHAPE@XY'],sql,None,True)
ascursor:
forrowincursor:
X=row[0][0]
Y=row[0][1]
somaGeometry.append(arcpy.Point(X,Y))
somaElev=arcpy.sa.ExtractByPolygon(sfElevation,somaGeometry,"INSIDE")
somaOutPath=sfElevation.replace('sf_elevation','SOMA_elev')
somaElev.save(somaOutPath)
print'extractionfinished'
TheExtractByPolygon()methodisabitmisleading,asitdoesnotacceptaPolygonobjectasaparameter.Instead,itrequiresalistofPointobjectsthatrepresenttheverticesoftheareathatwewanttoextract.AstheSearchCursorisiteratingthroughtheneighborhoodsdataset,aPolygonobjectisreturnedbythecursor.Fortunately,theSearchCursorhasafinalparameter,whichwehavenotyetexplored,thatallowsustoextracttheindividualpointsorverticesthatmakeuptheSomaneighborhoodpolygon.BysettingtheSearchCursor’soptionalExplodetoPointsparameter(whichconvertsPolygonobjectsintocoordinatepairsforeachvertex)toTrue,PointobjectscanbegeneratedbypassingtheXYvaluesofeachreturnedvertextothearcpy.Pointmethod.ThesePoint()objectsareappendedtothesomaGeometrylistandthenpassedtotheSpatialAnalystaccessmodule’sExtractByPolygonmethod.
NotePassingaPolygonObjectinsteadofPointObjectswillreturnanerror.
UsingMapAlgebratogenerateelevationinfeetWenowhavearastertousetoextractelevationvalues.However,boththeoriginalrasterandthegeneratedSoManeighborhoodrastercontainelevationvaluesinmeters,anditwouldbebettertoconvertthemtofeettokeepthemconsistentwiththeprojectionofthebusstops.Let’suserastermathandtheTimes()methodtoconvertthevaluesfrommeterstofeet:
somaOutPath=sfElevation.replace('sf_elevation','SOMA_elev')
outTimes=arcpy.sa.Times(somaOutPath,3.28084)
somaFeetOutPath=sfElevation.replace('sf_elevation','SOMA_feet')
outTimes.save(somaFeetOutPath)
TheTimes()methodgeneratesanewrastertogleantheelevationvaluesweneedforthebusstopsofinterest.
AddinginthebusstopsandgettingelevationvaluesNowthatwehavegeneratedarasterthatwecanusetofindelevationvaluesinfeet,weneedtoaddanewarcpy.sa()methodtogeneratethepoints.TheExtractValuesToPoints()methodwillgenerateanewbusstopsfeatureclasswithanewfieldthatholdstheelevationvalues:
witharcpy.da.SearchCursor(sanFranciscoHoods,['SHAPE@'],sql)ascursor:
forrowincursor:
somaPoly=row[0]
arcpy.MakeFeatureLayer_management(busStops,'soma_stops')
arcpy.SelectLayerByLocation_management("soma_stops","INTERSECT",somaPoly)
outStops=r'C:\Projects\PacktDB.gdb\Chapter11Results\SoMaStops'
arcpy.sa.ExtractValuesToPoints("soma_stops",
somaOutFeet,outStops,"INTERPOLATE","VALUE_ONLY")
print'pointsgenerated'
ThefinalresultWeproducedasubsetfeatureclassofthebusstopsthathastheelevationvaluesaddedasafield.Thisprocesscouldberepeatedfortheentirecity,oneneighborhoodatatime,oritcouldbeperformedwiththeoriginalelevationrasterontheentirebusstopsfeatureclasstogenerateavalueforeachstop:
importarcpy
arcpy.CheckOutExtension("Spatial")
arcpy.env.overwriteOutput=True
busStops=r'C:\Projects\PacktDB.gdb\SanFrancisco\Bus_Stops'
sanFranciscoHoods=
r'C:\Projects\SanFrancisco.gdb\SanFrancisco\SFFind_Neighborhoods'
sfElevation=r'C:\Projects\SanFrancisco.gdb\sf_elevation'
somaGeometry=[]
sql="name='SouthofMarket'"
witharcpy.da.SearchCursor(sanFranciscoHoods,['SHAPE@XY'],sql,None,True)
ascursor:
forrowincursor:
somaGeometry.append(arcpy.Point(row[0][0],row[0][1]))
somaElev=arcpy.sa.ExtractByPolygon(sfElevation,somaGeometry,"INSIDE")
somaOutput=sfElevation.replace('sf_elevation','SOMA_elev')
somaElev.save(somaOutput)
print'extractionfinished'
somaOutput=sfElevation.replace('sf_elevation','SOMA_elev')
outTimes=arcpy.sa.Times(somaOutput,3.28084)
somaOutFeet=sfElevation.replace('sf_elevation','SOMA_feet')
outTimes.save(somaOutFeet)
print'conversioncomplete'
witharcpy.da.SearchCursor(sanFranciscoHoods,['SHAPE@'],sql)ascursor:
forrowincursor:
somaPoly=row[0]
arcpy.MakeFeatureLayer_management(busStops,'soma_stops')
arcpy.SelectLayerByLocation_management("soma_stops","INTERSECT",somaPoly)
outStops=r'C:\Projects\SanFrancisco.gdb\Chapter11Results\SoMaStops'
arcpy.sa.ExtractValuesToPoints("soma_stops",
somaOutFeet,outStops,"INTERPOLATE","VALUE_ONLY")
print'pointsgenerated'
ThisscriptdemonstrateswellthevalueofaccessingtheadvancedextensionsinArcPyandcombiningthemwithSearchCursorsandGeometryobjects.ThescriptcouldbetakenevenfurtherbyaddingaSearchCursortolookthroughtheoutstopsdatasetandexportingtheresultstoaspreadsheet,orevenaddinganewfieldtotheoriginalbusstopsdatasettopopulatewiththeelevationvalues.ItcouldevenbeusedasimpedancevaluestobeenteredintoaNetworkAnalystextensionanalysis—afuncodingtaskthatIhopeyouwillattempt.
SummaryInthischapter,wecoveredthebasicsofusingcommonArcGISforDesktopAdvancedextensionswithinArcPy,withafocusontheNetworkAnalystaccessmoduleandtheSpatialAnalystaccessmodule.WeexploredhowtogenerateanetworkandhowtocreatenetworkpathsusingArcPy.WealsoexploredhowtoaccessSpatialAnalysttoolsandusetheminconjunctionwithSearchCursorstoworkwithrastersandvectorsforspatialanalysis.
Inthenextchapter,wewillexploresomefinalpiecestotheArcPypuzzlethatwillallowthecreationofadvancedscriptsandscripttools.
Chapter12.TheEndoftheBeginningThisbookisalmostdone,butthereissomuchmoretoknowaboutwritingcodeinPythonandArcPy.Unfortunately,Ican’tfititallintoonebook,butthatalsomeansthatyougettohavefunexploringallofthemethodsandpropertiesofArcPy.Asaconclusiontothebook,wewillcoversomeotherimportanttopicsthatcancropupwhenwritingArcPyscripts.Combinedwiththelessonsfromearlierchapters,Ihopeyou’llsoonbeusingArcPyatwork,atschool,orjustforfun(whynot?).
Thischapterwillcoverthefollowingtopics:
Workingwithfieldinformation–types,aliases,domains,spatialtypes,andmoreAccessinginformationdescribingaFeatureClassAutomaticallygeneratingaFeatureClassandpopulatingitwithfieldsAutomaticallycreatingFileGeodatabasesandFeatureDatasetsCreatingaScripttoolthatwillrunthebusstopanalysisandgenerateresultsinanautomaticallygeneratedFileGeodatabase,FeatureDataset,andFeatureClass
GettingfieldinformationfromfeatureclassesWhencreatingscripttools,orjustrunningascript,therecanbetimesthatextractingfieldinformationfromafeatureclass(orshapefile)isnecessary.Thisinformationcanincludefieldnamesandaliases,fieldtypeandlength,scale,domains,orsubtypes.Theseareallpropertiesavailablethroughthearcpy.ListFieldsmethod.We’llexplorethemanyproperties,howtoextractthem,andhowtousetheminascript.
ByorganizingtheArcPymethodsintoafunction,thedataisorganizedinaformthatweprefer,insteadofrelyingonthedefaultorganizationusedbythedesignersofArcPy.It’simportanttorememberthatscriptsyoucreateshouldreflectyourneeds,andcreatingthesefunctionwrappersisonestepforwardtowardspolishingtherawArcPytoolstoworkinyourworkflows.
AccessingtheListFields’propertiesTheListFieldstoolisavailableasanArcPymethod.Arcpy.ListFieldsacceptsonlyoneparameter,afeatureclass,orshapefile.Oncetheparameterhasbeenpassed,aseriesofimportantpropertiesareavailableusingdotnotation.Totakefurtheradvantageoftheseproperties,wewillcreatefunctionsthatmakeiteasytogettheinformationwewant,intheformatwerequire.
ListcomprehensionsWithinthesefieldinformationfunctions,wewilltakeadvantageofaPythondatastructureknownaslistcomprehensions.Theysimplifytheforloopstructuretomakeiteasiertopopulatealistwiththevaluesrequired(thefieldinformationinthiscase).
Tocreatealistcomprehension,aforloopisgeneratedinsideasetofbrackets,andthelistispopulatedwiththegeneratedvalues.Hereisanexampleofalistcomprehensionthatcreatesalistwiththesquarevaluesofthenumbersfrom1to10,asruninthePythoninterpreter:
>>>originalList=range(1,11)
>>>printoriginalList
[1,2,3,4,5,6,7,8,9,10]
>>>newList=[x**2forxinoriginalList]
>>>printnewList
[1,4,9,16,25,36,49,64,81,100]
Listcomprehensionsareusedbecausetheyarefasterandeasiertowrite,thoughitmaytakesometimetogetusedtothesyntax.Experimentwiththemtobetterunderstandtheiruseandlimitations,andalsoconsultsomeofthemanyresourcesavailableonline.
CreatingthefieldinformationfunctionsEachofthefunctionswillbeaseparateentity,buttheywillallhaveasimilarstructure.Oneparameterwillbeacceptedbyeachfunction,thefeatureclassofinterest.ArcPywillbeimported,andlaterdeletedfrommemory,tomakesurethattheListFields()methodcanbecalledwithoutanerror.OncethefeatureclassispassedtotheListFields()method,thevaluesdesiredwillpopulatealistinsidealistcomprehension.Onceithasbeenpopulated,itisreturnedfromthefunctionusingthereturnkeyword.
Hereisthesetoffunctionsforthefieldnames:
defreturnfieldnames(fc):
importarcpy
fieldnames=[f.nameforfinarcpy.ListFields(fc)]
delarcpy
returnfieldnames
defreturnfieldalias(fc):
importarcpy
fieldalias=[f.aliasNameforfinarcpy.ListFields(fc)]
delarcpy
returnfieldalias
defreturnfieldbasename(fc):
importarcpy
fieldtypes=[f.baseNameforfinarcpy.ListFields(fc)]
delarcpy
returnfieldtypes
Thesenamefunctionsareusefulwhencreatinganewfeatureclassbasedonanotherfeatureclass.Sometimesthereisaneedtopreservetheexactnamesandaliasesfromtheoriginalfeatureclass,andusingthesefunctionswillmakethispossible.Whendoingthis,thereisaneedtoprovideotherfieldinformationaswell.Herearethefunctionsrelatedtofieldtypes,lengths,precision,andscale:
defreturnfieldtypes(fc):
importarcpy
fieldtypes=[f.typeforfinarcpy.ListFields(fc)]
delarcpy
returnfieldtypes
defreturnfieldlength(fc):
importarcpy
fieldlengths=[f.lengthforfinarcpy.ListFields(fc)]
delarcpy
returnfieldlengths
defreturnfieldprecision(fc):
importarcpy
fieldprecise=[f.precisionforfinarcpy.ListFields(fc)]
delarcpy
returnfieldprecise
defreturnfieldscale(fc):
importarcpy
fieldscales=[f.scaleforfinarcpy.ListFields(fc)]
delarcpy
returnfieldscales
Thereisevenapropertyusedtorequestdomaininformation:
defreturnfielddomain(fc):
importarcpy
fielddomains=[f.domainforfinarcpy.ListFields(fc)]
delarcpy
returnfielddomains
Thesefunctionsallsharethestructurediscussedearlier,andhavetheadvantageofbeingsimpletouseandeasytosearchthroughout.Becausefieldsinafeatureclasshaveaspecificorder,eachlistreturnedbythefunctionswillhaveanordertotheinformationreturned,accessiblebyaspecificindexnumber.
Thefieldsubtypesarealsoavailablethroughthedataaccessmodule.Becausetheyarerelatedtothefields,theyarereturnedasadictionary:
defreturnfieldsubtypes(fc):
importarcpy
fieldsubdic={}
subtypes=arcpy.da.ListSubtypes(fc)
forstcode,stdictinsubtypes.iteritems():
forstkeyinstdict.iterkeys():
ifstkey=='FieldValues':
fields=stdict[stkey]
forfield,fieldvalsinfields.iteritems():
sub=fieldvals[0]
desc=fieldvals[1]
fieldsubdic[field]=sub,desc
delarcpy
returnfieldsubdic
NoteAddingthesefunctionstotheuseful.pyscriptinthecommonmodulewillmakethemavailabletoanyscriptorscripttool.Usetheimportkeywordtoaddthemtoanynewscript.Theyareself-containedfunctionsthatonlyrequirethefilepathtothefeatureclassofinterest.
QueryingfeatureclassinformationSomeimportantpiecesofinformationaboutanincomingfeatureclasscannotbeaccessedusingtheListFields()method.Instead,anumberofdifferentmethodswillbeusedtofindtheGeometrytype,orSpatialReference,orthefieldsubtypeofeachfeatureclass.SomeofthesearediscoveredusingArcPy’sDescribemethod,builttoprovide
FortheGeometrytype,wewillusetheshapeTypepropertyoftheDescribe()method:
defreturngeometrytype(fc):
importarcpy
arcInfo=arcpy.Describe(fc)
geomtype=arcInfo.shapeType
delarcpy
returnstr(geomtype)
ThenameoftheShapefield(whichusuallydefaultstoShape)canalsoberequestedusingtheDescribemethodandreturnsastringdatatype:
defreturngeometryname(fc):
importarcpy
arcInfo=arcpy.Describe(fc)
geomname=arcInfo.shapeFieldName
delarcpy
returnstr(geomname)
Thefeatureclassspatial_referenceisalsoavailablethroughtheDescribemethod.Thedataisreturnedasaspatial_referenceobject:
defreturnspatialreference(fc):
importarcpy
spatial_reference=arcpy.Describe(fc).spatialReference
delarcpy
returnspatial_reference
Aspatial_referenceobjecthasanumberofimportantproperties.Theprojectionnameandprojectioncodeareamongtheimportant
defreturnprojectioncode(fc):
importarcpy
spatial_reference=arcpy.Describe(fc).spatialReference
proj_code=spatial_reference.projectionCode
delarcpy
returnproj_code
defreturnprojectionname(fc):
importarcpy
spatial_reference=arcpy.Describe(fc).spatialReference
proj_name=spatial_reference.name
delarcpy
returnproj_name
Manyotherpropertiesandmethodscanbesimilarlyutilizedtomakethemavailablewithinscriptsorscripttools.ExploretheArcGIShelpdocumentsforfurtherinsightsintothepropertiesavailablethroughtheDescribemethod.
GeneratingFileGeodatabasesandfeatureclassesFileGeodatabasesdonothavetoexistbeforeascriptisrun;instead,theycanbegeneratedwhenascriptisexecutedusingtheCreateFileGDBtool,whichisalsoanArcPymethod.OncetheFileGeodatabasehasbeencreated,FeatureDatasetscanbeadded.
GeneratingtheFileGeodatabaseisveryeasy.Theonlyparametersarethefolderstoplaceitinside,andthenameoftheGeodatabase:
importarcpy
folderPath=r"C:\Projects"
gdbName="ArcPy.gdb"
arcpy.CreateFileGDB_management(folderPath,gdbName)
TheFeatureDatasetsaremoredifficulttocreate,asthereisanoptionalspatialreferenceparameterthatrequiresaSpatialReferenceobjecttobegenerated.WhiletheSpatialReferenceobjectisoptional,itishighlyrecommended.
ThereareafewoptionstogeneratetheSpatialReferenceobject.OneofthemusesthereturnspecialReference()functiondefinedearlier;bypassingafeatureclasstothefunction,aSpatialReferenceobjectiscreated.Anothermethodwouldbetopassafilepathtoaprojectionfile.prjastheoptionalthirdparameter.AthirdmethodistogenerateaSpatialReferenceobjectbyusingthearcpy.SpatialReferencemethodandpassingitaprojectioncodeoraprojectionstring:
spatialReference=arcpy.SpatialReference(2227)
Howeveritisgenerated,itisthenpassedtothearcpy.CreateFeatureDatasetmethodalongwiththefilepathoftheFileGeodatabaseandthenameoftheFeatureDataset:
spatialReference=arcpy.SpatialReference(2227)
fileGDB=r"{0}\{1}".format(folderPath,gdbName)
featureDataset="Chapter12Results"
arcpy.CreateFeatureDataset_management(fileGDB,featureDataset,
spatialReference)
GeneratingafeatureclassNowthataFileGeodatabaseandaFeatureDatasethavebeencreated,let’sgenerateaFeatureClassinsidetheFeatureDataset.Thisisdoneusingthearcpy.CreateFeatureClassmethod.Thismethodhasanumberofoptionalparameters,includingaFeatureClasstouseasatemplateandaSpatialReference.Forthisexample,thereisnoneedtousetheSpatialReferenceparameterasitisbeingwrittentoaFeatureDataset,whichdictatestheSpatialReferenceused.ThetemplateparameterwillcopythefieldsofthetemplateFeatureClass,butfornow,wewillonlycreatetheShapefield:
featureClass="BufferArea"
geometryType="POLYGON"
featurePath=r"{0}\{1}".format(fileGDB,featureDataset)
arcpy.CreateFeatureclass_management(featurePath,featureClass,
geometryType)
ThecreatedFeatureClasswillneedsomefieldswiththeattributeinformationthatwillbepopulatedlater.Thefieldshaveanumberofparametersthatdependonthefieldtype,includinglength,precision,andalias,amongothers:
fieldName="STOPID"
fieldAlias="BusStopIdentifier"
fieldType="LONG"
fieldPrecision=9
featureClassPath=r"{0}\{1}".format(featurePath,featureClass)
arcpy.AddField_management(featureClassPath,fieldName,fieldType,
fieldPrecision,
"","",fieldAlias)
Let’saddasecondfieldtoholdtheaveragedpopulationvaluesproducedbytheBusStopanalysis:
fieldName2="AVEPOP"
fieldAlias2="AverageCensusPopulation"
fieldType2="FLOAT"
featureClassPath=r"{0}\{1}".format(featurePath,featureClass)
arcpy.AddField_management(featureClassPath,fieldName2,fieldType2,"","",
"",fieldAlias2)
TheFileGeodatabase,FeatureDataset,andFeatureClassfieldshavenowbeengenerated.Let’sextendthescriptintoascripttoolbyaddingtheBusStopanalysisfunctions,whilewritingtheresultstothegeneratedFeatureClass.Creating,ascripttoolthatpopulatesafeatureclass.
ThisscripttoolwillborrowfromtheideasoutlinedinChapter10,AdvancedGeometryObjectMethodsandwillcreateaunionofthePolygonGeometryobjectsthatintersectwiththebufferedbusstopstopopulatetheShapefield,alongwiththebusstopIDandtheaveragedpopulationfortheblocksintersectedwitheachbuffer.
OpenthescriptChapter12_3.pyandexploreitscontents.Coupledwiththecodesnippetsmentionedearlierandtheuseofarcpy.GetParameterAsTexttogetdatafromthescript
tool,thedatageneratedwillbewritteninafeatureclassbythefollowingcode:
arcpy.AddMessage("BeginningAnalysis")
insertCursor=arcpy.da.InsertCursor(featureClassPath,['SHAPE@',fieldName,
fieldName2])
arcpy.MakeFeatureLayer_management(censusBlocks2010,"census_lyr")
witharcpy.da.SearchCursor(busStops,['SHAPE@',busStopField],sql)as
cursor:
forrowincursor:
stop=row[0]
stopID=row[1]
busBuffer=stop.buffer(400)
arcpy.SelectLayerByLocation_management("census_lyr","intersect",busBuffer,'
','NEW_SELECTION')
censusShapes=[]
censusPopList=[]
witharcpy.da.SearchCursor("census_lyr",
['SHAPE@',censusBlockPopField])asncursor:
fornrowinncursor:
censusShapes.append(nrow[0])
censusPopList.append(nrow[1])
censusUnion=censusShapes[0]
forblockincensusShapes[1:]:
censusUnion=censusUnion.union(block)
censusPop=sum(censusPopList)/len(censusPopList)
finalData=(censusUnion,stopID,censusPopulation)
insertCursor.insertRow(finalData)
arcpy.AddMessage("AnalysisComplete")
Thescriptcombinesmanyoftheideasthathavebeenintroducedthroughoutthebooktoallowtheusertorunacompleteworkflowthatgeneratesafeatureclasscontainingtheresultsoftheanalysis.ByaddingonlythefieldsofinterestandpopulatingthemwiththeunionedPolygonobjects,thescripteliminatesmostofthecruft,normallycreatedwhenrunningaspatialanalysis,andproducesaresultsdatasetthatcanbeviewedinArcMap.
SettingupthescripttoolparametersHereishowtheparametersofthescripttoollookwhensetup:
Thelistofparametersislong,soIamusingtwoimagestoportraythem.Itisimportanttochoosethecorrectdatatypeforeachparameterasitwillcontrolthedialoggeneratedtoretrievethedata.
ThebusstopIDfieldandthePopulationfieldarebothobtainedfromtheirrespectivefeatureclasses.TheFileGeodatabasenameisastringandthecodewillappend.gdbtotheendoftheinputstringifitisnotenteredinitially,tomakesurethatitcanbecorrectlygenerated.Itshouldnotalreadyexist;itwillnotbegeneratedifitdoes(ifdesired,thiscanbechangedbysettingthearcpy.env.overwriteOutputpropertytoTrueaftertheimportstatement).
Oncetheparametershavebeenset,andthetoolhasanameanddescription,saveitandthenopenthetool.Itshouldlooklikethisonceithasbeenfilledout:
ClickonOKtorunthetool.OpenArcMapandaddtheresults,alongwiththeSanFranciscopolygonandtheInbound71featureclassfromChapter4,ComplexArcPyScriptsandGeneralizingFunctions.Theresultswilllooksimilartothis,afterabitofcartographicsymbolizing:
Thefinalresultwillhaveonerowperbusstopselected,alongwiththeaveragedpopulationandthebusstopIDvalue.Insteadofusingaspreadsheetasanoutput,thefeatureclasswillallowtomakemapsorproducefurtherspatialanalysis.Producingcustomdatausingcustomscripttoolsputsyouinthedriver’sseatwhenperforminggeospatialanalysesandmakesyourtools,andyou,avaluableassettoanyteam.
EnvironmentalsettingsTheArcPymoduleallowsforthecontrolofglobalsettingsthatcontrolsinputandoutputprocessesusingArcPy’senvclass.Thesesettingswillhaveaneffectontheaccuracyofdataproducedusinggeospatialanalysistools.ResolutionandtolerancesettingsforX,Y,Z,andMcoordinatescanbecontrolled,alongwithoutputextent,rastercellsize,analysisworkspace,andmanyothersettings.
ToaccesstheenvironmentalsettingsusingArcPy,theclassenvisimportedfromarcpy:
>>>fromarcpyimportenv
Itcanalsobecalledusingdotnotationshownasfollows.Settingtheworkspaceremovestheneedtopassafilepathtoanysubsequentmethodscalledontheworkspace.HereisanexampleofsettingtheworkspaceandcallingtheListDatasets()methodwithoutpassingafilepathasaparameter:
>>>importarcpy
>>>arcpy.env.workspace=r"C:\Projects\SanFrancisco.gdb"
>>>arcpy.ListDatasets()
[u'SanFrancisco',u'Chapter3Results',u'Chapter4Results',
u'Chapter5Results',u'Chapter7Results',u'Chapter11Results']
ResolutionandtolerancesettingsTheresolutionandtolerancesettingscontroltheaccuracyoftheoutputofanydataproducedbyatoolinArcToolboxorwhenrunningascriptusingArcPy.Thesecan(andshould)besetforFeatureDatasetsinFileGeodatabasesorEnterpriseGeodatabases,butitisimportanttosetthemforanalysisruninthememoryorwhenusingshapefiles,orifthegeospatialanalysisrequiresgreateraccuracythanusedbythoseGeodatabases.
Settingtheresolutionsandtolerancesrequireanunderstandingoftheaccuracyrequiredforyourprojects.Thesesettingswilllimittheabilitytosnaptoalineorfindpointsthatintersectwithaline.Thelinearunitwillneedtoreflectthecoordinatesystemofchoice:
importarcpy
arcpy.env.MResolution=0.0005
arcpy.env.MTolerance=0.005
arcpy.env.ZResolution="0.0025Feet"
arcpy.env.ZTolerance="0.001Feet"
arcpy.env.XYResolution="0.00025Feet"
arcpy.env.XYTolerance="0.0005Feet"
Otherimportantenvironmentalsettingsinclude:
TheExtentsetting,whichlimitstheextentofanydataproducedfromananalysisbysettingarectangleofinterestusinganExtentobject,orastringwithspacedelimitedcoordinates(Xmin,Ymin,Xmax,Ymax)inthecurrentcoordinatesystem.TheMasksetting,whichlimitsrasteranalysistoareasthatintersectwithafeatureclassorarasterpassedasastringfilepathparametertothesetting.TheCellSizesetting,whichcontrolsthecellsizeofthedataproducedusingrasteranalysis.
TaketimeandexplorethepowerfulArcPyEnvironmentalSettingstoreducethetimeneededtowritecodeandensurehigh-qualitydataproduction.
SummaryThischapterandthisbookhavedemonstratedsomeofthemanywaysthatArcPycanbeusedtoautomategeospatialanalysis.Byapplyingthelessons,andbybeingcreativewiththemanymethodsandpropertiesofArcPy,repetitiveandslowgeospatialprocessescanbescriptedandmadeintocustomtoolsthatwillsavealotoftime.
IhopethatyouenjoyedlearningthebasicsofscriptingwithArcPyandPython.Ireallyhopethatyou’veevencometoliketheideaofprogramming,asitispowerfulandempowering.Thereismuchmoretomaster,butIthinkyouwillfindthatthemorescriptingyoudo,theeasieritistounderstand.
ThebestresourceforfurtherunderstandingofArcPyistheArcGISHelpDocuments,availablethroughtheHelpmenuinArcCatalogorArcMap.Thedocumentationisalsoavailableathttp://resources.arcgis.com/en/help/main/10.2/index.html.WorkingonenteringthecorrectquestionintoGooglecanbeveryhelpfulaswell.ProgrammingforumssuchasStackExchange(http://gis.stackexchange.com/)orESRI’sGeoNet(https://geonet.esri.com/welcome)arevaluableresourcestoaskallkindsofprogrammingquestions.Thereisananswerforalmosteveryquestionyoumayhave(butneverbeafraidtoaskquestionsyourself!).
Havefuncreatingsolutionsandtools,andgoodluckinallyourfuturegeospatialprogrammingchallenges!
IndexA
adjustedmapexporting,toPDF/ExportingtheadjustedmaptoPDF
analysiscomponentsadding/AddingadvancedanalysiscomponentsPolygonobjectmethods/AdvancedPolygonobjectmethodsrandompoints,generatingtorepresentpopulation/Generatingrandompointstorepresentpopulationfunctionsused,withinscript/UsingthefunctionswithinascriptXLScreating,XLWTused/CreatinganXLSusingXLWT
analysisresultstallying/Tallyingtheanalysisresults
ApplicationProgrammingInterface(API)/WrappermodulesAptanaStudio3/AptanaStudio3
URL/AptanaStudio3ARCMacroLanguage(AML)
about/ModelBuilderArcPy
about/OverviewofPythonused,withmapdocuments/UsingArcPywithmapdocumentsused,foraccessingnetworkdataset/AccessingtheNetworkDatasetusingArcPy
arcpy.AddMessageused,fordisplayingscriptmessages/Displayingscriptmessagesusingarcpy.AddMessage
arcpy.mappingused,forcontrollingLayerobjects/Usingarcpy.mappingtocontrolLayerobjects
arcpy.Pointfunction/UsinganInsertCursorarcpy.SpatialReference()method/ThedataaccessmoduleArcPygeometryobjectclasses
about/ArcPygeometryobjectclasses,ArcPyPointobjects,ArcPyArrayobjects,ArcPyPolylineobjectsPolygonobjects/ArcPyPolygonobjects,Polygonobjectbuffers,OtherPolygonobjectmethodsgeometryobjects/ArcPygeometryobjectsPointGeometryobjects/ArcPyPointGeometryobjects
ArcPymoduleabout/OverviewofPython,TheArcPymodule
attributefieldinteractions/Attributefieldinteractionsautomatedmapdocumentadjustment
about/Automatedmapdocumentadjustmentvariables/Thevariablesmapdocumentobject/Themapdocumentobjectandthetextelementstextelements/Themapdocumentobjectandthetextelementslayervisibility,adjusting/Adjustinglayervisibilitybuffer,generatingfrombusstopsfeatureclass/Generatingabufferfromthebusstopsfeatureclassbusstopbufferandcensusblocks,intersecting/Intersectingthebusstopbufferandcensusblockstextelements,updating/Updatingthetextelements
Bbrokenlinks
fixing/Fixingthebrokenlinksbuffer
generating,frombusstopsfeatureclass/Generatingabufferfromthebusstopsfeatureclass
Buffertoolmodeling/ModelingtheSelectandBuffertools
built-infunctionsstr/Commonlyusedbuilt-infunctionsint/Commonlyusedbuilt-infunctionsfloat/Commonlyusedbuilt-infunctions
built-inmodulesURL/Commonlyusedstandardlibrarymodules
busstopbufferblockandcensusblock,intersecting/Intersectingthebusstopbufferandcensusblocks
busstopclassandbufferfeatureclass,populating/Populatingtheselectedbusstopandbufferfeatureclasses
BusStopfeatureclassadding,asparameter/AddingtheBusStopfeatureclassasaparameter
busstopfieldsadding,asparameter/Addingthebusstopfieldsasaparameter
Ccensusblock
andbusstopbufferblock,intersecting/Intersectingthebusstopbufferandcensusblocks
CensusBlockfeatureclassadding,asparameter/AddingtheCensusBlockfeatureclassasaparameter
CensusBlockfieldadding,asparameter/AddingtheCensusBlockfieldasaparameter
CommaSeparatedValue(CSV)/Addingtheoutputspreadsheetasaparametercomments
about/CommentsCSVmodule
adding,toscript/AddingtheCSVmoduletothescriptcursor
used,foraccessingdata/Accessingthedata:Usingacursor
Ddata
accessing,cursorused/Accessingthedata:Usingacursordataaccessmodule
about/Thedataaccessmoduleattributefieldinteractions/Attributefieldinteractionsupdatecursor/Updatecursorsshapefield,updating/Updatingtheshapefieldpointlocation,adjusting/Adjustingapointlocationrow,deletingwithupdatecursor/DeletingarowusinganUpdateCursorinsertcursor,using/UsinganInsertCursor
dataframewindowextentcontrolling/Controllingthedataframewindowextentandscale
datasetsimporting/Importingthedatasets
datasourcesreplacing/Replacingthedatasources
datatypesabout/Datatypesstrings/Stringsintegers/Integersfloats/Floatslists/Liststuples/Tuplesdictionaries/Dictionariesiterabledatatypes/Iterabledatatypesadding/Addingdatatypes
definitionqueryabout/Definitionqueries
defkeywordabout/Functions
deleteRowmethod/DeletingarowusinganUpdateCursordictionaries/Dictionariesdynamiccomponents
adding,toscript/Addingdynamiccomponentstothescriptdynamicparameters
adding,toscript/Addingdynamicparameterstoascript
Eenvironmentalsettings
about/Environmentalsettingsresolutionsetting/Resolutionandtolerancesettingstolerancesetting/ResolutionandtolerancesettingsExtentsetting/ResolutionandtolerancesettingsMasksetting/ResolutionandtolerancesettingsCellSizesetting/Resolutionandtolerancesettings
Ffeatureclasses
fieldinformation,obtainingfrom/GettingfieldinformationfromfeatureclassesListFieldstool/AccessingtheListFields’propertiesListcomprehensions/Listcomprehensionsfieldinformationfunctions,creating/Creatingthefieldinformationfunctionsfeatureclassinformation,querying/QueryingfeatureclassinformationFileGeodatabases,generating/GeneratingFileGeodatabasesandfeatureclassesgenerating/GeneratingFileGeodatabasesandfeatureclasses,Generatingafeatureclassscripttoolparameters,settingup/Settingupthescripttoolparametersenvironmentalsettings/Environmentalsettings
featureclassinformationquerying/Queryingfeatureclassinformation
FeatureDatasetcreating/CreatingaFeatureDataset
fieldinformationobtaining,fromfeatureclasses/Gettingfieldinformationfromfeatureclasses
fieldinformationfunctionscreating/Creatingthefieldinformationfunctions
filepaths,inPythonabout/FilepathsinPython
finalscriptabout/Thefinalscriptinspecting/Inspectingthefinalscript,RunningtheScriptTool
floats/Floatsforloops/Forloopsfunctions
about/Functionsused,withinscript/Usingthefunctionswithinascript
IIDEs
about/IntegratedDevelopmentEnvironments(IDEs),IDEsummaryIDLE/IDLEPythonWin/PythonWinAptanaStudio3/AptanaStudio3automaticallygeneratedscript/Theautomaticallygeneratedscript
IDLEabout/IDLE
If/Elif/Elsestatementsabout/If/Elif/Elsestatements
importstatements/Importstatementsindentation
about/Indentationindividuallayers
fixing/Fixingthelinksofindividuallayers,ExportingtoPDFfromanMXDinsertcursor
using/UsinganInsertCursorintegers/IntegersIntersecttool
adding/AddingtheIntersecttooliterabledatatypes/Iterabledatatypes
LLayerobject
adding/AddingaLayerobjectLayerobjects
controlling,arcpy.mappingused/Usingarcpy.mappingtocontrolLayerobjectsmethods/Layerobjectmethodsandpropertiesproperties/Layerobjectmethodsandproperties
layerobjectsabout/Thelayerobjects
Layersabout/Usingarcpy.mappingtocontrolLayerobjects
layersourcesinspecting/Inspectingandreplacinglayersourcesreplacing/Inspectingandreplacinglayersourcesbrokenlinks,fixing/Fixingthebrokenlinksindividuallayers,fixing/Fixingthelinksofindividuallayers,ExportingtoPDFfromanMXDmapdocumentelements,adjusting/Adjustingmapdocumentelements
layervisibilityadjusting/Adjustinglayervisibility
Listcomprehensions/ListcomprehensionsListFieldstool/AccessingtheListFields’propertieslists/Lists
Mmapdocumentelements
adjusting/Adjustingmapdocumentelementsmapdocuments
ArcPy,usingwith/UsingArcPywithmapdocumentsmaps
exporting/Exportingthemapsmodel
creating/CreatingamodelandexportingtoPythonexporting,toPython/CreatingamodelandexportingtoPythonSelecttool,modeling/ModelingtheSelectandBuffertoolsBuffertool,modeling/ModelingtheSelectandBuffertoolsIntersecttool,adding/AddingtheIntersecttoolanalysisresults,tallying/Tallyingtheanalysisresultsexporting/Exportingthemodelandadjustingthescript
ModelBuilderabout/ModelBuilder
moduleadding,sysmoduleused/UsingPython’ssysmoduletoaddamodule
modulesresiding/Wheremodulesreside
Nnamespaces
about/Namespacesnamingvariables
using,bestpractices/VariablesNetworkAnalyst
using/UsingNetworkAnalystFeatureDataset,creating/CreatingaFeatureDatasetdatasets,importing/Importingthedatasetsnetworkdataset,creating/CreatingtheNetworkDatasetnetworkdatasetaccess,ArcPyused/AccessingtheNetworkDatasetusingArcPy
NetworkAnalystextensionabout/TheNetworkAnalystextension
NetworkAnalystmoduleabout/TheNetworkAnalystmodule
networkdatasetcreating/CreatingtheNetworkDataset
OOperatingSystem(OS)module
about/TheOperatingSystem(OS)moduleoutputspreadsheet
adding,asparameter/Addingtheoutputspreadsheetasaparameter
Pparameter
CensusBlockfeatureclass,addingas/AddingtheCensusBlockfeatureclassasaparameterCensusBlockfield,addingas/AddingtheCensusBlockfieldasaparameteroutputspreadsheet,addingas/Addingtheoutputspreadsheetasaparameterspreadsheetfieldnames,addingas/AddingthespreadsheetfieldnamesasaparameterSQLStatement,addingas/AddingtheSQLStatementasaparameterbusstopfields,addingas/Addingthebusstopfieldsasaparameter
parametersdatatypes,adding/AddingdatatypesBusStopfeatureclass,addingas/AddingtheBusStopfeatureclassasaparameter
PDFadjustedmap,exportingto/ExportingtheadjustedmaptoPDF
PointGeometryobjects/ArcPyPointGeometryobjectspointlocation
adjusting/Adjustingapointlocationpolygongeometry
inserting/InsertingapolygongeometryPolygonobjectmethods/AdvancedPolygonobjectmethodsPolygonobjects/ArcPyPolygonobjects,Polygonobjectbuffers,OtherPolygonobjectmethodspolyLinegeometry
inserting/Insertingapolylinegeometryprogrammingjargon
about/ForloopsPython
about/OverviewofPythonPython,asprogramminglanguage
about/Pythonasaprogramminglanguageinterpretedlanguage/Interpretedlanguagestandard(built-in)library/Standard(built-in)librarygluelanguage/Thegluelanguagewrappermodules/Wrappermodules
Python,basicsabout/ThebasicsofPythonimportstatements/Importstatementsvariables/Variablesforloops/ForloopsIf/Elif/Elsestatements/If/Elif/Elsestatementswhilestatement/Whilestatements
comments/CommentsPythonfolderstructure
about/Pythonfolderstructuremodules,residing/Wheremodulesresidesysmoduleused,foraddingmodule/UsingPython’ssysmoduletoaddamodule
Pythonfunctionsabout/Pythonfunctions–Avoidrepeatingcodedefining/Technicaldefinitionoffunctionswriting/Afirstfunctionwithparameters/Functionswithparametersused,forreplacingrepetitivecode/Usingfunctionstoreplacerepetitivecodegeneralization/Moregeneralizationofthefunctions
Pythoninterpreterabout/WhatisthePythoninterpreter?location/WhereisthePythoninterpreterlocated?using/WhichPythoninterpretershouldbeused?locating/Howdoesthecomputerknowwheretheinterpreteris?
Pythonmodulecreating/CreatingaPythonmodule__init__.pyfile/The__init__.pyfile
PythonModules,forGISAnalysisabout/ImportantPythonModulesforGISAnalysis
Pythonscriptabout/WhatisaPythonscript?executing/HowPythonexecutesascript
PythonSystem(SYS)moduleabout/ThePythonSystem(SYS)module
PythonWin/PythonWinURL/PythonWin
PythonWindowscript,runningin/RunningthescriptinthePythonWindow
Rrandompoints
generating,torepresentpopulation/Generatingrandompointstorepresentpopulation
replace()method/Updatecursorsrow
deleting,withupdatecursor/DeletingarowusinganUpdateCursor
Sscaleproperties
controlling/Controllingthedataframewindowextentandscalescript
adjusting/Exportingthemodelandadjustingthescript,AdjustingtheScriptCSVmodule,addingto/AddingtheCSVmoduletothescriptdynamicparameters,addingto/Addingdynamicparameterstoascriptdynamiccomponents,addingto/Addingdynamiccomponentstothescriptrunning,inPythonWindow/RunningthescriptinthePythonWindowbreaking/Breakingdownthescript
ScriptAnalysis,ArcPyToolscontinuing/Continuingthescriptanalysis:theArcPytoolsIntersectTool/TheIntersecttoolandstringmanipulationStringManipulation/TheIntersecttoolandstringmanipulation
scriptmessagesdisplaying,arcpy.AddMessageused/Displayingscriptmessagesusingarcpy.AddMessage
scripttoolcreating/CreatingaScripttoolparameters,defining/Labellinganddefiningparametersparameters,labelling/Labellinganddefiningparameters
scripttoolparameterssettingup/Settingupthescripttoolparameters
Selecttoolmodeling/ModelingtheSelectandBuffertools
shapefieldupdating/Updatingtheshapefield
SpatialAnalystExtensionaccessing/AccessingtheSpatialAnalystExtensionelevation,addingtobusstops/AddingelevationtothebusstopsMapAlgebraused,forgeneratingelevationinfeet/UsingMapAlgebratogenerateelevationinfeetbusstops,adding/Addinginthebusstopsandgettingelevationvalues,Thefinalresultelevationvalues,obtaining/Addinginthebusstopsandgettingelevationvalues,Thefinalresult
spreadsheetfieldnamesadding,asparameter/Addingthespreadsheetfieldnamesasaparameter
SQLStatementadding,asparameter/AddingtheSQLStatementasaparameter
standardlibrarymodulesdatetime/Commonlyusedstandardlibrarymodulesmath/Commonlyusedstandardlibrarymodules
string/Commonlyusedstandardlibrarymodulesstringaddition/Thestringmanipulationmethod1–stringadditionstringformatting/Thestringmanipulationmethod2–stringformatting#1,Thestringmanipulationmethod3–stringformatting#2StringManipulation
stringaddition/Thestringmanipulationmethod1–stringadditionstringformatting/Thestringmanipulationmethod2–stringformatting#1,Thestringmanipulationmethod3–stringformatting#2
strings/Stringssubroutines
about/TechnicaldefinitionoffunctionsSys.path.append()method
about/Thesys.path.append()methodsysmodule
used,foraddingmodule/UsingPython’ssysmoduletoaddamodule
Ttextelements
updating/Updatingthetextelementsadjustedmap,exportingtoPDF/ExportingtheadjustedmaptoPDF
Tkinterabout/IDLE
tuples/Tuples
Uupdatecursor
about/Updatecursorsused,fordeletingrow/DeletingarowusinganUpdateCursor
updateRow()method/Updatecursors,Adjustingapointlocation
XXLRDmodule
about/TheXLRDandXLWTmodulesXLS
creating,XLWTused/CreatinganXLSusingXLWTXLWTmodule
about/TheXLRDandXLWTmodules