Using arcpy to List Domains Assigned to Featureclass Fields

I was making an edit (adding leading “0”s) to a coded-value domain in an SDE database and realized that my edits were changing the order of the rows of my domain.  Rows were moved to the bottom of the list when they were edited.

So I went through the process of converting my domain back to a table, made my edits in Access and exported the rows to a .dbf in the order I wanted them.

I removed the domain from the field I knew it was assigned to but when I tried to delete the domain, I received an error (The domain is used as a default domain).

The Google Machine led me to an ArcForums post by  with some python code for listing all the fields with a domain.

I tweaked the original a bit, first because it was only examining feature classes in a feature dataset, not stand-alone feature datasets.  And secondly, I updated it to use arcpy.  I posted both the 9.3 version and the 10.0 version, but here is a quick look.  The key addition is the ‘listfc(“”)’ line that is the first line of the def listds() module.

import arcpy

min_workspace = r"C:\Users\mranter\AppData\Roaming\ESRI\Desktop10.0\ArcCatalog\min.minstaff.sde"

infgdb=(min_workspace)
arcpy.env.workspace = infgdb

def listfc(inDataset):
   featureclasses = arcpy.ListFeatureClasses("","",inDataset)
   for f in featureclasses:
      print "feature class: ",f

lfields=arcpy.ListFields(f)

for lf in lfields:
   if lf.domain < "":
      print "      domain",f, lf.name, lf.domain

def listds():
   listfc("")

   datasets=arcpy.ListDatasets ("","")
   for d in datasets:
      print "  dataset: ",d

listfc(d)
listds()

ArcMap Field Calculator: Calculating Geometry using arcpy

One of the things I had not gotten around to doing in ArcGIS 10 was figure out how to directly manipulate the geometry of a record using the Field Calculator.  When I stumbled upon a bug in the way the Extract Values to Points tool handles Null geometries, I figured it was time to figure it out.  Setting the X, Y to 0,0 was sufficient for my needs.

I set the Parser to Python and the formula was simple once I figured out the syntax:

pPoint = arcpy.Point(0,0)

Then, to extend my knowledge, I wanted to see how to calculate the geometry based off of two fields (X and Y).   The only real trick is knowing the bracket field names using exclamation points instead of brackets:

arcpy.Point(!X!,!Y!)

So turns out everything it pretty easy and straight-forward.  Whew!

Loading Tiled, Same-Name Data in Batch Mode.

I have been loading existing raster data into a geodatabase to be included in a new Mosaic Dataset–a very cool and useful addition to ArcGIS 10. The most time-consuming part of the process for the human (at least this human) has been getting the names of the rasters right.

Our existing data is organized by tiles with the directory name representing the tile name and then the data within each tile directory having the same name.

For example:
C:\GIS_dataAdamsparcels.shp
C:\GIS_dataBuchetteparcels.shp

This makes batch loading the data less efficient because I end up having to rename the data or else end up with a series of feature classes named parcels, parcels_2, parcels_n.

So I hacked out a quick script that takes an input raster and figures out the final name I want it to have based of the directory name.

First, I used the Copy Raster (In ArcToolbox: Data Management-Raster-Raster Dataset-Copy Raster) and copied on sample to my geodatabase.

Then, I went to the Results Tab (Select Geoprocessing from the Menubar, Geoprocessing-Results) and right-clicked on the Copy Raster result and selected “Copy as Python Snippet”.

I then created a new python script and pasted the one line.

I added some imports, accepted a parameter, some string manipulation, and some result outputs and I had a quick & easy script. In added the script in ArcToolbox and now I can right-click on it and run it in Batch mode. I do a quick search in Windows Explorer to get all the rasters I want to run it on and select & drag them to my ArcToolbox Batch Dialog.

Actual code can be downloaded HERE and you don’t need to worry about WordPress messing up the spacing.

import arcpy
import os, sys

inRaster = sys.argv[1] 
basedir = os.path.basename(os.path.dirname(inRaster)).lower()
outRaster = "Database Connections/mgs_lidar.lidar.sde/mgs_lidar.lidar."+basedir

def printit(inMessage):
    print inMessage
    arcpy.AddMessage(inMessage)
    
if not (arcpy.Exists(outRaster)):
    printit ("Importing: "+basedir)
    arcpy.CopyRaster_management(inRaster,outRaster,"#","#","#","NONE","NONE","#")
else:
    printit ("Skipping: "+basedir+" because it already exists!")

Checking to see if a Field Index Exists Using Arcpy (ArGIS 10.0)

Updating some python code from 9.3 that using geoprocessing  to 10.0 using arcpy and the first real object I’ve had to change relates to detecting whether or not an index exists on a table.

I previously posted code using a 9.3 geoprocessing commands, the core of it being:

indexList = gp.listindexes(tablename)
for iIndex in indexList:
    if (iIndex.Name == indexname):
       return True
return False

With arcpy, ESRI has gone back to using the Describe methodology.  Way back when GISers use to type most of their commands into their wood-burning computers, the command “Describe” was used to retrieve information about a coverage, grid, or other data set in our fancy AMLs (or SMLs).  The snippet below shows a function for checking to see if a table has an index with a specified named.

def indexExists(tablename,indexname):

 if not arcpy.Exists(tablename):
  return False

 tabledescription = arcpy.Describe(tablename)

 for iIndex in tabledescription.indexes:
  if (iIndex.Name == indexname):
   return True

 return False

Zipping a File Geodatabase using Python

Ever since the ever-popular post, Zipping a shapefile using python, came out, people have been asking (one person, yesterday) for a sample of how to zip a file geodatabase using python.

The key trick, as shown in line 17, is appending the basename of the file geodatabase (“nfg.gdb/” in my example) in front of each file as you write it to the zipfile.

UPDATE: WordPress messes with the spacing when I post code, making it difficult to post code that can just be copied & pasted and have work.  So I have posted a the code HERE for downloading.

import os
import zipfile
import glob

infile = "c:/temp/nfg.gdb"
outfile = "c:/temp/nfg.zip"
def zipFileGeodatabase(inFileGeodatabase, newZipFN):
   if not (os.path.exists(inFileGeodatabase)):
      return False

   if (os.path.exists(newZipFN)):
      os.remove(newZipFN)

   zipobj = zipfile.ZipFile(newZipFN,'w')

   for infile in glob.glob(inFileGeodatabase+"/*"):
      zipobj.write(infile, os.path.basename(inFileGeodatabase)+"/"+os.path.basename(infile), zipfile.ZIP_DEFLATED)
      print ("Zipping: "+infile)

   zipobj.close()

   return True

zipFileGeodatabase(infile,outfile)

Multiple outputs for Python scripts

Related to my post on how I enable a script to accept parameters from different sources, I also often set up pythons scripts to output information a variety of ways.  This is largely due to the fact that some are called by ArcToolbox scripts.  Running in ESRI’s domain, these scripts need to send the output through the arcgisscripting object but if you are running the python outside the ArcGIS framework, you can just print.

If you assume one output method but then run your code in the opposite framework, you don’t get to see all the pretty little messages.  What I do is create a simple little routine that broadcasts the message both ways.  This is probably an obvious solution but took a few cases before I went ahead and started implementing it.

gp = arcgisscripting.create()

#This will print both to the geoprocessing window or Python output window
def gpprint(inmessage):
 gp.addmessage(inmessage)
 print inmessage

<Code to do stuff>

#Ok, I want to send a message:
gpprint('Hello, sailor!')

Zipping a shapefile via ArcToolbox

UPDATE:

After receiving a request to modify the code to ignore .lock files, I have an updated to this post.

 

I’ve received a request on how to use the Zip Shapefile code I posted last week from ArcGIS. Sorry, I did not set the code up to call directly from ArcGIS but only as an illustration of how it can be done.

I have, however, with some minor tweaking, made a version that can added to ArcToolbox. The steps to install are below, please note that at one point I had you download a *.zip file that had been renamed to *.jpg–this should now be corrected and the link should lead you directly to zipshapefile.zip.  Because of this steps two and three are obsolete.

1) Download the code from here.
2) Rename the file from zipshapefile-zip.jpg back to zipshapefile.zip.
3) Unzip the file.
4) Move ZipShapefile.py to C:Program FilesArcGISArcToolBoxScriptsZipShapefile.py.
5) Optionally, move Zip Shapefile.tbx, perhaps C:Program FilesArcGISArcToolBoxToolboxes.
6) Add the toolbox to ArcToolbox. ESRI has instructions here on how to do this.

You should now have a new toolbox named “Zip Shapefile” with a script named “Zip a Shapefile” in it. Clicking on on the tool will bring up this dialog.

**********************************
In response to Chris:

I believe you need to copy the ZipShapefile.py file from the .zip that you downloaded to C:Program FilesArcGISArcToolBoxScripts, the error message is consistent with the tool not being about to find the python script there.

If you prefer to place the ZipShapefile.py in a different location, you will need to change the source on the tool. To do this, right click on the tool in ArcCatalog and change the path of the Script File as set in the Source tab (see below):

Launching a Python script with parameters–3 methods.

Since I use python for different tasks, I launch python scripts a variety of ways. Depending on what I am doing, a single script may need to accept parameters from either:

  1. Passed in from an ArcGIS Toolbox Tool.
  2. Re-occurring default value.  Often used in scheduled processes, a nightly backup, for example.
  3. A temporary set of values used in an interactive, debugging session.

What I often do is make the parameter interpretation flexible to meet my needs.  The sample below shows how I do this.  The logic first checks to see if the correct number of parameters were used to launch the script (i.e. if it is called by an Arc Toolbox tool), where the files for the default files exist or if the debug values are valid, including checking the current date against a hard-coded date variable.  Juggling the conditional structure would allow you to prioritize the options differently.

I am also using Tkinter to display an interactive dialog if none of the three conditions are successfully met.

import os.path
import datetime
import shutil
import sys

theEmailText = "nStart: " + str(datetime.datetime.now())

#This example shows three different ways for a script to receive input paramters.
#
# First, if it was run with the necessary number of parameters, as if launched by
# an ArcToolbox tool, it uses those parameters.

if len(sys.argv) == 2:
    wellsShapeFile = sys.argv[0]
    unwellsShapeFile = sys.argv[1]
    theEmailText = theEmailText + "nUsing Parameters Passed In:nn Located Wells File: "+wellsShapeFile+"n Unlocated Wells File: "+unwellsShapeFile
else:
    dateString = datetime.date.today().strftime("%Y%m%d")
    wellsShapeFile = "C:/cwi5_bk/wells/temp/wells_.shp"
    unwellsShapeFile = "C:/cwi5_bk/wells/temp/unloc_wells.shp"

    #Second attempt is if there are default values that should be used.
    #I use this for a process that is run via scheduled Windows Task
    if (os.path.exists(wellsShapeFile)) and (os.path.exists(unwellsShapeFile)):
        theEmailText = theEmailText + "nUsing Automated Date-Based file names:nn Located Wells File: "+wellsShapeFile+"n Unlocated Wells File: "+unwellsShapeFile
    else:
        #The third method is used for debugging/running in the IDE.
        #I put a check condition so this is valid for one day only
        #And then hard-code the temporary paths.
        #
        #Note that you may want to modify the structure of the IF statement
        #used for methods 2 & 3 so that it checks for the manual-override (3rd)
        #method first
        if dateString == '20101213':
            wellsShapeFile = "C:/cwi5_bk/wells/temp/wells.shp"
            unwellsShapeFile = "C:/cwi5_bk/wells/temp/unloc.shp"
            theEmailText = theEmailText + "nUsing Manual Override file names:nn Located Wells File: "+wellsShapeFile+"n Unlocated Wells File: "+unwellsShapeFile
        else:
            theEmailText = theEmailText + "n Manual Override for file names does not meet data filter"

    if (not os.path.exists(wellsShapeFile)) or (not os.path.exists(unwellsShapeFile)):
        from Tkinter import *
        msgbox = Tk()
        msgbox.title('Error')
        Message(msgbox,text="Must either use ArcTool to launch or edit file parameters", bg='royalblue',fg='ivory', relief=GROOVE).pack(padx=10, pady=10)
        msgbox.mainloop()
        print theEmailText
        quit()

print theEmailText

Zipping a shapefile using python

UPDATE:

After receiving a request to modify the code to ignore .lock files, I have an updated to this post.

One of the tasks I’ve been automating is publishing a weekly data update to a website. The update consists of shapefile. The trouble with shapefiles is they consist of 3 or more files with the same basename but different extensions in the same directory.

Not an overly complicated situation but a common one that ArcGIS does not have a solution out-of-the-box. Below is a bare-bones code snippet to do it. It has both the input shapefile and output zip file hard-coded. The call to the subroutine that does the work is: zipShapefile(wellsShapeFile,wellsZipFile)

import zipfile
import sys
import os
import glob
wellsShapeFile = "C:/cwi5_bk/WELLS/wells.SHP"
wellsZipFile = "C:/cwi5_bk/WELLS/test5.zip"

def zipShapefile(inShapefile, newZipFN):
   print 'Starting to Zip '+inShapefile+' to '+newZipFN

if not (os.path.exists(inShapefile)):
   print inShapefile + ' Does Not Exist'
   return False

if (os.path.exists(newZipFN)):
   print 'Deleting '+newZipFN
   os.remove(newZipFN)

if (os.path.exists(newZipFN)):
   print 'Unable to Delete'+newZipFN
   return False

zipobj = zipfile.ZipFile(newZipFN,'w')

for infile in glob.glob( inShapefile.lower().replace(".shp",".*")):
   print infile
   zipobj.write(infile,os.path.basename(infile),zipfile.ZIP_DEFLATED)

zipobj.close()
return True

zipShapefile(wellsShapeFile,wellsZipFile)
print "done!"

TopoToRaster Error

In running an automated process, I had a TopoToRaster repeatedly fail on me. The only input theme was a contour theme. The process ran fine when I used the envelope of the contour theme as the output extent but when I changed it to the envelope of a polygon theme, it would bomb. The polygon’s envelope was smaller than the contour theme.

ArcCatalog would bomb out without presenting any sort of useful message.

I determined eventually that If I set the left extent as 435210, the process would work if set the right extent to 446655 or greater but would bomb if I used 446654 or less. If the the left extent was 435209, the right extent had to be 446654 or greater.

The image below shows the contours (with vertexes) and the polygon theme I was using. The two circle graphics show the approximate extent I was clipping to.

I eventually wrote python script to do a series of tests with the intent of determining the pattern of what does and does not work. The unexpected benefit was I actually got some sort of error message back. Turns out it had to do with my lack of vertices–I densified my polylines and it ran fine. Something I should have thought of earlier but it was difficult lacking any feedback.