ArcToolbox Tool: Add [ST_CON_ABR] to Hennepin County, MN Centerlines

One of the great advancements over the last decade plus in GIS is that government agencies have started to move away from a “recover-our-cost” mentality to more of an “Open Data”. Minnesota, for example, has launched their Geospatial Commons as a platform for sharing data.

And while getting free, authoritative data is awesome, it can leave you in a bind if the structure of the data changes. Sometime between April and September, Hennepin County, Minnesota, changed the schema of their publicly available street centerlines data.

The data used to have both a full, concatenated street name field ([ST_CONCAT]) and an abbreviated version ([ST_CON_ABR]). A record might have “James Lofton Avenue North” and “James Lofton Ave N”, respectively, in these two fields. The abbreviated version was nice for labeling but it disappeared from the most recent updates.

So, as you  might guess from the fact that I’m posting about it, I wrote a script to add that field in and launch it from an ArcToolbox Tool. Nothing fancy going on in the code, just a series of replaces, depending on the field. Using dictionaries instead of arrays of paired values might have been better but the script takes just a few seconds to run so I can live with it as-is.

The list of street type abbreviations came from a combination of ESRI’s standards and those found in an older version of the Hennepin County data. There were no conflicting abbreviations between the two. The code warns if a street name occurs in the data that is not in the list.

While I’m including the code here for reference, it’s probably best to download the code from GitHub.

#-------------------------------------------------------------------------------
# Name:        usi_dataprep_Add_STCONABR
#
# Purpose:     This can be used to add [CT_CON_ABR] to Hennepin County, MN
#              centerlines. This is a concatenated, abbreviated full name of
#              the street. This used to be included in the data but
#              disappeared from the downloads in the summer of 2017.
#
#              Data available at: http://www.hennepin.us/gisopendata
#
# Author:      mrantala
#
# Created:     2017.10.04
#
#------------------------------------------------------------------------------

import arcpy

############################################
## Custom Variables
#These are the fields that are concatenated. Hennepin has others but were always blank.
requiredFieldList = ["ST_PRE_DIR","ST_PRE_TYP","ST_NAME","ST_POS_TYP","ST_POS_DIR"]
#This is the name of the field to add
newFieldName = "ST_CON_ABR"

#These are the abbreviations for [ST_POS_TYPE]. The list was created using a sample of
#Hennepin's centerline data & Esri Tech article: http://support.esri.com/en/technical-article/000008454
# Note that I intentionally left cases in where there is no abbreviation (Fall, for example) as a means of
#documenting the fact that it should NOT change.

abbList = []
abbList.append(["Alcove","Alcove"]) #Hennepin Specific
abbList.append(["Alley","Aly"])
abbList.append(["Annex","Anx"])
abbList.append(["Arcade","Arc"])
abbList.append(["Avenue","Ave"])
abbList.append(["Bay","Bay"]) #Hennepin Specific
abbList.append(["Bayoo","Byu"])
abbList.append(["Beach","Bch"])
abbList.append(["Bend","Bnd"])
abbList.append(["Bluff","Blf"])
abbList.append(["Bluffs","Blfs"])
abbList.append(["Bottom","Btm"])
abbList.append(["Boulevard","Blvd"])
abbList.append(["Branch","Br"])
abbList.append(["Bridge","Brg"])
abbList.append(["Brook","Brk"])
abbList.append(["Brooks","Brks"])
abbList.append(["Burg","Bg"])
abbList.append(["Burgs","Bgs"])
abbList.append(["Bypass","Byp"])
abbList.append(["Camp","Cp"])
abbList.append(["Canyon","Cyn"])
abbList.append(["Cape","Cpe"])
abbList.append(["Causeway","Cswy"])
abbList.append(["Center","Ctr"])
abbList.append(["Centers","Ctrs"])
abbList.append(["Crossings","Crossings"]) #Hennepin Specific
abbList.append(["Crossroad","Xrd"])
abbList.append(["Chase","Chase"]) #Hennepin Specific
abbList.append(["Circle","Cir"])
abbList.append(["Circles","Cirs"])
abbList.append(["Cliff","Clf"])
abbList.append(["Cliffs","Clfs"])
abbList.append(["Club","Clb"])
abbList.append(["Close","Close"]) #Hennepin Specific
abbList.append(["Common","Cmn"])
abbList.append(["Commons","Cmns"]) #Hennepin Specific
abbList.append(["Corner","Cor"])
abbList.append(["Corners","Cors"])
abbList.append(["Corridor","Corridor"]) #Hennepin Specific
abbList.append(["Course","Crse"])
abbList.append(["Court","Ct"])
abbList.append(["Courts","Cts"])
abbList.append(["Cove","Cv"])
abbList.append(["Coves","Cvs"])
abbList.append(["Creek","Crk"])
abbList.append(["Crescent","Cres"])
abbList.append(["Crest","Crst"])
abbList.append(["Cross","Cross"]) #Hennepin Specific
abbList.append(["Crossing","Xing"])
abbList.append(["Curve","Curve"])
abbList.append(["Dale","Dl"])
abbList.append(["Dam","Dm"])
abbList.append(["Divide","Dv"])
abbList.append(["Down","Down"]) #Hennepin Specific
abbList.append(["Downs","Downs"]) #Hennepin Specific
abbList.append(["Drive","Dr"])
abbList.append(["Drives","Drs"])
abbList.append(["Edge","Edge"]) #Hennepin Specific
abbList.append(["Entry","Entry"]) #Hennepin Specific
abbList.append(["Estate","Est"])
abbList.append(["Estates","Ests"])
abbList.append(["Expressway","Expy"])
abbList.append(["Extension","Ext"])
abbList.append(["Extensions","Exts"])
abbList.append(["Fall","Fall"])
abbList.append(["Falls","Fls"])
abbList.append(["Ferry","Fry"])
abbList.append(["Field","Fld"])
abbList.append(["Fields","Flds"])
abbList.append(["Flat","Flt"])
abbList.append(["Flats","Flts"])
abbList.append(["Ford","Frd"])
abbList.append(["Fords","Frds"])
abbList.append(["Forest","Frst"])
abbList.append(["Forge","Frg"])
abbList.append(["Forges","Frgs"])
abbList.append(["Fork","Frk"])
abbList.append(["Forks","Frks"])
abbList.append(["Fort","Ft"])
abbList.append(["Freeway","Fwy"])
abbList.append(["Gables","Gables"]) #Hennepin Specific
abbList.append(["Garden","Gdn"])
abbList.append(["Gardens","Gdns"])
abbList.append(["Gate","Gate"]) #Hennepin Specific
abbList.append(["Gateway","Gtwy"])
abbList.append(["Glade","Glade"]) #Hennepin Specific
abbList.append(["Glen","Gln"])
abbList.append(["Glens","Glns"])
abbList.append(["Green","Grn"])
abbList.append(["Greens","Grns"])
abbList.append(["Greenway","Greenway"]) #Hennepin Specific
abbList.append(["Grove","Grv"])
abbList.append(["Groves","Grvs"])
abbList.append(["Harbor","Hbr"])
abbList.append(["Harbors","Hbrs"])
abbList.append(["Haven","Hvn"])
abbList.append(["Heights","Hts"])
abbList.append(["Highway","Hwy"])
abbList.append(["Hill","Hl"])
abbList.append(["Hills","Hls"])
abbList.append(["Hollow","Holw"])
abbList.append(["Horn","Horn"]) #Hennepin Specific
abbList.append(["Inlet","Inlt"])
abbList.append(["Island","Is"])
abbList.append(["Islands","Iss"])
abbList.append(["Isle","Isle"])
abbList.append(["Junction","Jct"])
abbList.append(["Junctions","Jcts"])
abbList.append(["Key","Ky"])
abbList.append(["Keys","Kys"])
abbList.append(["Knoll","Knl"])
abbList.append(["Knolls","Knls"])
abbList.append(["Lake","Lk"])
abbList.append(["Lakes","Lks"])
abbList.append(["Land","Land"])
abbList.append(["Landing","Lndg"])
abbList.append(["Lane","Ln"])
abbList.append(["Light","Lgt"])
abbList.append(["Lights","Lgts"])
abbList.append(["Loaf","Lf"])
abbList.append(["Lock","Lck"])
abbList.append(["Locks","Lcks"])
abbList.append(["Lodge","Ldg"])
abbList.append(["Loop","Loop"])
abbList.append(["Mall","Mall"])
abbList.append(["Manor","Mnr"])
abbList.append(["Manors","Mnrs"])
abbList.append(["Meadow","Mdw"])
abbList.append(["Meadows","Mdws"])
abbList.append(["Mews","Mews"])
abbList.append(["Mill","Ml"])
abbList.append(["Mills","Mls"])
abbList.append(["Mission","Msn"])
abbList.append(["Motorway","Mtwy"])
abbList.append(["Mount","Mt"])
abbList.append(["Mountain","Mtn"])
abbList.append(["Mountains","Mtns"])
abbList.append(["Neck","Nck"])
abbList.append(["Orchard","Orch"])
abbList.append(["Oval","Oval"])
abbList.append(["Overpass","Opas"])
abbList.append(["Park","Park"])
abbList.append(["Parks","Park"])
abbList.append(["Parkway","Pkwy"])
abbList.append(["Parkways","Pkwy"])
abbList.append(["Pass","Pass"])
abbList.append(["Passage","Psge"])
abbList.append(["Path","Path"])
abbList.append(["Pike","Pike"])
abbList.append(["Pine","Pne"])
abbList.append(["Pines","Pnes"])
abbList.append(["Place","Pl"])
abbList.append(["Plain","Pln"])
abbList.append(["Plains","Plns"])
abbList.append(["Plaza","Plz"])
abbList.append(["Point","Pt"])
abbList.append(["Points","Pts"])
abbList.append(["Port","Prt"])
abbList.append(["Ports","Prts"])
abbList.append(["Prairie","Pr"])
abbList.append(["Radial","Radl"])
abbList.append(["Railroad","Railroad"]) #Hennepin Specific
abbList.append(["Ramp","Ramp"])
abbList.append(["Ranch","Rnch"])
abbList.append(["Rapid","Rpd"])
abbList.append(["Rapids","Rpds"])
abbList.append(["Rest","Rst"])
abbList.append(["Ridge","Rdg"])
abbList.append(["Ridges","Rdgs"])
abbList.append(["Rise","Rise"]) #Hennepin Specific
abbList.append(["River","Riv"])
abbList.append(["Road","Rd"])
abbList.append(["Roads","Rds"])
abbList.append(["Route","Rte"])
abbList.append(["Row","Row"])
abbList.append(["Rue","Rue"])
abbList.append(["Run","Run"])
abbList.append(["Shoal","Shl"])
abbList.append(["Shoals","Shls"])
abbList.append(["Shore","Shr"])
abbList.append(["Shores","Shrs"])
abbList.append(["Skies","Skies"]) #Hennepin Specific
abbList.append(["Skyway","Skwy"])
abbList.append(["Spring","Spg"])
abbList.append(["Springs","Spgs"])
abbList.append(["Spur","Spur"])
abbList.append(["Spurs","Spur"])
abbList.append(["Square","Sq"])
abbList.append(["Squares","Sqrs"])
abbList.append(["Station","Sta"])
abbList.append(["Stravenue","Stra"])
abbList.append(["Stream","Strm"])
abbList.append(["Street","St"])
abbList.append(["Streets","Sts"])
abbList.append(["Summit","Smt"])
abbList.append(["Terrace","Ter"])
abbList.append(["Throughway","Trwy"])
abbList.append(["Trace","Trce"])
abbList.append(["Track","Trak"])
abbList.append(["Trafficway","Trfy"])
abbList.append(["Trail","Trl"])
abbList.append(["Tunnel","Tunl"])
abbList.append(["Turn","Turn"]) #Hennepin Specific
abbList.append(["Turnpike","Tpke"])
abbList.append(["Underpass","Upas"])
abbList.append(["Union","Un"])
abbList.append(["Unions","Uns"])
abbList.append(["Valley","Vly"])
abbList.append(["Valleys","Vlys"])
abbList.append(["Viaduct","Via"])
abbList.append(["View","Vw"])
abbList.append(["Views","Vws"])
abbList.append(["Village","Vlg"])
abbList.append(["Villages","Vlgs"])
abbList.append(["Ville","Vl"])
abbList.append(["Vista","Vis"])
abbList.append(["Walk","Walk"])
abbList.append(["Walks","Walk"])
abbList.append(["Wall","Wall"])
abbList.append(["Way","Way"])
abbList.append(["Ways","Ways"])
abbList.append(["Well","Wl"])
abbList.append(["Wells","Wls"])

#List of changes for [St_POS_Dir]
posDirList = [["North","N"],["East","E"],["South","S"],["West","W"],["Northeast","NE"],["Northwest","NW"],["Southeast","SE"],["Southwest","SW"]]
preDirList = [["North","N"],["East","E"],["South","S"],["West","W"]]
############################################
## Read Arguments

if (len(sys.argv) > 1):
    inFC = sys.argv[1]

############################################
# General Purpose Functions
def printit(inputString):
    try:
        print(inputString)
        arcpy.AddMessage(str(inputString))
    except:
        pass

def printerror(inputString):
    print (inputString)
    arcpy.AddError(inputString)

def getField(inFeatureClass, inFieldName):
  fieldList = arcpy.ListFields(inFeatureClass)
  for iField in fieldList:
    if iField.name.lower() == inFieldName.lower():
      return iField
  return None

def fieldExists(inFeatureClass, inFieldName):
  return getField(inFeatureClass,inFieldName) <> None

############################################
# Initial QC

def initialQC():
    if (arcpy.Exists(inFC)):
        printit("PASS: Feature Class {} Exists".format(inFC))
    else:
        printerror("ERROR: Feature Class {} Does Not Exist, Cancelling...".format(inFC))
        return False

    for iFld in requiredFieldList:
        if (fieldExists(inFC,iFld)):
            printit("PASS: Feature Class {} Has Field [{}]".format(inFC,iFld))
        else:
            printerror("ERROR: Feature Class {} Does Not Have Field [{}], Cancelling...".format(inFC,iFld))
            return False

    if not (fieldExists(inFC,newFieldName)):
        printit("GOOD: Feature Class {} Does Not Already Have Field [{}]".format(inFC,newFieldName))
        printit(" ADDING Field [{}]".format(newFieldName))
        try:
            arcpy.AddField_management(in_table=inFC, field_name=newFieldName, field_type="TEXT", field_precision="", field_scale="", field_length="100", field_alias="", field_is_nullable="NULLABLE", field_is_required="NON_REQUIRED", field_domain="")
        except:
            printerror("ERROR: Error While Adding Field [{}], Cancelling...".format(newFieldName))
            return False
        if not (fieldExists(inFC,newFieldName)):
            printerror("ERROR: Unable to Add Field [{}], Cancelling...".format(newFieldName))
            return False
    else:
        printerror("ERROR: Feature Class {} Already Has Field [{}], Cancelling...".format(inFC,newFieldName))
        return False

    return True

############################################
# Main

def makeSubstitution(inList,inValue,inFieldName):
    for iAbbreviationPr in inList:
        if (inValue == iAbbreviationPr[0]): #Found a Match
            return iAbbreviationPr[1]
    printit("WARNING: [{}] of {} does not have a value in the abbreviation list! Potential Error...".format(inFieldName,inValue))
    return inValue

def main():
    cursorFieldList = requiredFieldList
    cursorFieldList.append(newFieldName)

    try:
        iUCursor = arcpy.da.UpdateCursor(inFC,cursorFieldList)
        iRowCount = 0
        iRowMax = 1
        for uRow in iUCursor:

            #Just to give user an indicator that progress is being made
            if (iRowCount>iRowMax):
                printit(" {}".format(iRowCount))
                iRowMax *= 10
                iRowCount+=1

            abbreviateConcatenatedName = ""
            iFldIndex = 0
            for iFld in requiredFieldList:


                if (iFld == newFieldName):
                    uRow[iFldIndex] = abbreviateConcatenatedName
                    iUCursor.updateRow(uRow)
                else:
                    iValue = uRow[iFldIndex].strip() #Strip is just a safe-guard


                    if ((iValue != "") and (iValue != None)):
                        if (iFld == "ST_PRE_DIR"):
                            iValue= makeSubstitution(preDirList,iValue,"ST_PRE_DIR")
                        if (iFld == "ST_POS_TYP"):
                            iValue= makeSubstitution(abbList,iValue,"ST_POS_TYPE")
                        if (iFld == "ST_POS_DIR"):
                            iValue = makeSubstitution(posDirList,iValue,"ST_POS_DIR")

                        if (abbreviateConcatenatedName == ""):
                            abbreviateConcatenatedName = iValue
                        else:
                            abbreviateConcatenatedName+=" "+iValue
                iFldIndex += 1

        del iUCursor
    except RuntimeError as e:
        printerror("ERROR: Error {} Occurred, Cancelling...".format(e))
        try:
            del iUCursor
            del uRow
        except:
            return False
    return True


if __name__ == '__main__':
    if (initialQC() == True):
        if (main() == True):
            printit("Done!")

Converting MXD to Layer file in Arcpy

Working on doing some advanced ArcGIS server printing and had the need to batch convert many existing .mxd files to .lyr files. So instead of opening up X number of map documents, thought I would do it via code. All of my .mxds in this case had just one data frame so the process was pretty simple–I add an empty group layer (Thanks Petr Krebs for the idea), copy all the existing layers into it, and save it out as a layer file.

I created an ArcGIS toolbox with two options–one to convert a single .mxd and one to batch convert an entire folder. To use it, make sure to have the EmptyGroup.lyr in the same directory as the .py file.

Here is the raw code or git it:


import os
import arcpy
import inspect
import glob
import uuid
import inspect

codeDir = os.path.dirname(inspect.getfile(inspect.currentframe()))
EmptyGroupLayerFile = codeDir+"/EmptyGroup.lyr"
inArg1 = sys.argv[1]
inArg2 = sys.argv[2]

def printit(inMessage):
    arcpy.AddMessage(inMessage)

def makeLyrFromMXD(inMXD, outLyr):
    if not (os.path.exists(inMXD)):
        printit( "ERROR: {} does not exist".format(inMXD))
        return False
    if not (os.path.exists(EmptyGroupLayerFile)):
        printit( "ERROR: {} does not exist".format(EmptyGroupLayerFile))
        return False
    if  (os.path.exists(outLyr)):
        printit( "Skipping: {} already exists".format(outLyr))
        return True

    printit( "Making Layer file: {0}".format(outLyr))

    mxd = arcpy.mapping.MapDocument(inMXD)
    ###Right now, just doing the first Dataframe, this could be modified
    df = arcpy.mapping.ListDataFrames(mxd)[0]

    theUUID = str(uuid.uuid1())

    iGroupLayerRaw = arcpy.mapping.Layer(EmptyGroupLayerFile)
    iGroupLayerRaw.name = theUUID
    arcpy.mapping.AddLayer(df,iGroupLayerRaw,"TOP")
    groupBaseName = os.path.basename(outLyr).split(".")[0]

    for lyr in arcpy.mapping.ListLayers(df):
        if not (lyr.name == theUUID):
            if (lyr.longName == lyr.name):
                arcpy.mapping.AddLayerToGroup (df, iGroupLayer, lyr, "Bottom")
        else:
            iGroupLayer = lyr

    iGroupLayer.name = groupBaseName
    arcpy.SaveToLayerFile_management(iGroupLayer, outLyr)
    return os.path.exists(outLyr)

def doMultiple(inDir,outDir):
    for iMxd in glob.glob(inDir+"/*.mxd"):
        lyrFile = outDir+"/"+os.path.basename(iMxd).lower().replace(".mxd",".lyr")
        makeLyrFromMXD(iMxd, lyrFile)

if(not os.path.exists(EmptyGroupLayerFile)):
    printit("Error: {} is missing, can not run.".format(EmptyGroupLayerFile))
else:
    if (os.path.isdir(inArg1) and (os.path.isdir(inArg2))):
        doMultiple(inArg1,inArg2)
    elif (os.path.isfile(inArg1)):
        if (os.path.exists(inArg2)):
            printit("Error: {} already exists".format(inArg2))
        else:
            makeLyrFromMXD(inArg1,inArg2)
    else:
        printit("Unable to understand input parameters")

Calling os.startfile and webbrowser.open from ArcGIS.

Recently I’ve created Python add-ins for data entry for our staff. Most of these have a toolbar with a “Help” button that opens a help file in .pdf format.

Sample python add-in toolbar.
Sample python add-in toolbar.

The first add-in was for ArcCatalog and this worked splendidly. I was using os.startfile(path to help.pdf).

However, when I started doing ArcMap add-ins, clicking the Help button would open the help.pdf but ArcMap would crash. Oops!

Luckily the Python development team at Esri already had a blog post about this at their ArcPy Café blog.

They report that the root of the problem is “conflicts in the way the Windows libraries expect to be called, they can fail or crash when called within ArcGIS for Desktop in an add-in script or geoprocessing script tool”. But this can be overcome by using a decorator function that calls os.startfile from a new thread. Another function effected by these conflicts is webbrowser.open.

Example code is shown below:

import functools
import os
import threading
import webbrowser
 
# A decorator that will run its wrapped function in a new thread
def run_in_other_thread(function):
    # functool.wraps will copy over the docstring and some other metadata
    # from the original function
    @functools.wraps(function)
    def fn_(*args, **kwargs):
        thread = threading.Thread(target=function, args=args, kwargs=kwargs)
        thread.start()
        thread.join()
    return fn_
 
# Our new wrapped versions of os.startfile and webbrowser.open
startfile = run_in_other_thread(os.startfile)
openbrowser = run_in_other_thread(webbrowser.open)

Then whenever you call startfile or openbrowser, it will be routed through your decorator function and, as far as I’ve been able to tell, works fine without crashing your ArcMap session.

Cheers!

Quick & Dirty Arcpy: Verify a Coded Value Domain Code

I’ve been working on a few different data import routines and one of the things I recently built was the ability to verify that a potential Code to be entered into a field with a Coded Value Domain is valid.

The logic of the code is pretty straight-forward. Get a field’s domain and check that a potential value is one of the code values. The biggest “trick” in this code is that arcpy.da.ListDomains, which locates a field’s domain, takes a geodatabase (or Enterprise geodatabase connection file) as its only parameter. The documentation says it takes a workspace, but it does not like a feature dataset, which a feature class might be in.

A couple caveats about the code. It only returns True if a field exists, has a coded value domain, and the value tested is one of the (case-sensitive) valid codes. While I have an ArcToolbox tool to call it for illustration purposes, I’m only calling it from code so I wanted tight requirements.

Anyhow, here is the code or download it from GitHub.

import arcpy

inFeatureClass = sys.argv[1]
inField = sys.argv[2]
inValue = sys.argv[3]

# getFeatureClassParentWorkspace: This script gets the geodatabase for a
# feature class. The trick here is that feature classes can be within a
# feature dataset so you need to account for two possible levels in the
# directory structure.
def getFeatureClassParentWorkspace(inFeatureClass):
    describeFC = arcpy.Describe(inFeatureClass)
    if (describeFC.dataType == 'FeatureClass') or (describeFC.dataType == 'Table'):
        workspace1 = describeFC.path
        describeWorkspace1 = arcpy.Describe(workspace1)
        if (describeWorkspace1.dataType == 'FeatureDataset'):
            return describeWorkspace1.path
        return workspace1

    return None

# Find a field within a feature class
def getField(inFeatureClass, inFieldName):
  fieldList = arcpy.ListFields(inFeatureClass)
  for iField in fieldList:
    if iField.name.lower() == inFieldName.lower():
      return iField
  return None

#Get a field's domain
def getDomain(inFeatureClass, inField):
    theField = getField(inFeatureClass,inField)
    if (theField <> None):
        searchDomainName = theField.domain
        if (searchDomainName <> ""):
            for iDomain in arcpy.da.ListDomains(getFeatureClassParentWorkspace(inFeatureClass)):
                if iDomain.name == searchDomainName:
                    return iDomain
    return None

#Get the domain.
def validDomainValue(inFeatureClass,inField,inValue):
    theDomain = getDomain(inFeatureClass,inField)

    if not (theDomain is None):
        if (theDomain.domainType == "CodedValue"):
            if theDomain.codedValues.has_key(inValue):
                return True
    return False

if (validDomainValue(inFeatureClass,inField,inValue)):
    arcpy.AddMessage("Value ({0}) is valid for field [{1}].".format(inValue,inField))
else:
    arcpy.AddError("ERROR: Value ({0}) is invalid for field [{1}].".format(inValue,inField))

Quick & Dirty arcpy: Bulk Changing Field Values

In mapping cross sections, our geologists often find themselves renaming their stratigraphic units midway, or at the end, of creating multiple cross sections.  This can cause a situation where we need to change multiple values in multiple fields in multiple feature classes–a situation that can get messy very fast.

Perfect situation for a quick & dirty arcpy script and, in this case, an ArcToolbox tool that can be downloaded.

This tool will change all feature classes in the O:\clay_cga\sand-distribution_model\dnrPackages\stratlines directory.

It will look at two fields, [strat] and [unit] and make these changes:

  • “go” becomes “gro”
  • “goc” becomes “grc”
  • “sgb” becomes “grb”

And since I have Case Sensitive checked, “Go” will not get changed to “gro”.  Also note that only full values that match values in the Old Value List get changed, part matches are left as-is so “got” would be left as-is even though the first two characters match “go”.

 

Bulk Field Change

import arcpy
import sys, string, arcgisscripting
import arcpy

def printit(inString):
    print inString
    arcpy.AddMessage(inString)

def printerr(inString):
    print inString
    arcpy.AddError(inString)

def fieldExists(tablename,indexname):

 if not arcpy.Exists(tablename):
  return False

 tabledescription = arcpy.Describe(tablename)

 for iField in tabledescription.fields:
     if (iField.Name.lower() == indexname.lower()):
         return True

 return False


if len(sys.argv) > 1:
    inDirectory = sys.argv[1]
    inFieldNameRaw = sys.argv[2]
    oldValue = sys.argv[3].replace(","," ")
    newValue = sys.argv[4].replace(","," ")
    caseSensitiveRaw = sys.argv[5]
else:
    inDirectory = r"C:\temp\test\stratest"
    inFieldNameRaw = "strat"
    oldValue = "go, goc, sgb".replace(","," ")
    newValue = "gro grc grb".replace(","," ")
    caseSensitiveRaw = "true"

caseSensitive = (caseSensitiveRaw.lower() == "true")
fieldNameList = inFieldNameRaw.replace(","," ").split()

printit("Starting")
printit(" Workspace: "+str(inDirectory))
printit( " inFieldName: "+str(inFieldNameRaw))
printit( " oldValue: "+str(oldValue))
printit( " newValue: "+str(newValue))
printit( " caseSensitive: "+str(caseSensitive))

valueDict = dict()

def initialQC():
    global valueDict

    if not (arcpy.Exists(inDirectory)):
        printerr("Workspace {0} does not exist".format(inDirectory))
        return False

    if (len(oldValue.split()) <> len(newValue.split())):
        printerr("Number of values in {0} does not equal number of values in {1}".format(oldValue,newValue))
        return False

    iValueIndex = 0
    for iOldValue in oldValue.split():
        if (caseSensitive):
            thisKey = iOldValue
        else:
            thisKey = iOldValue.lower()

        if (valueDict.has_key(thisKey)):
            printerr("ERROR: Value, {0}, is repeated, cancelling...".format(thisKey))
            return False

        valueDict[thisKey] = newValue.split()[iValueIndex]
        iValueIndex+=1
    return True

def makeFieldList(inFC):
    thisFieldList = []

    for iField in fieldNameList:
        if (fieldExists(inFC,iField)):
            thisFieldList.append(iField)

    return thisFieldList


def main():
    arcpy.env.workspace = inDirectory
    printit(valueDict)
    for iFC in arcpy.ListFeatureClasses():
        printit("Working on {0}".format(iFC))

        iFieldList = makeFieldList(iFC)
        if (len(iFieldList) == 0):
            printit(" No fields to change, Skipping...")
            continue

        rows = arcpy.UpdateCursor(iFC)

        changes = 0
        printit(" Changing Rows")
        for row in rows:
            iChange = 0
            for iField in iFieldList:
                iValue = str(row.getValue(iField))
                newValue = iValue

                if valueDict.has_key(iValue):
                    newValue = valueDict[iValue]
                else:
                    if not (caseSensitive):
                        if valueDict.has_key(iValue.lower()):
                            newValue = valueDict[iValue.lower()]

                if (newValue <> iValue):
                    printit("CHANGE {0}".format(newValue))
                    row.setValue(iField,newValue)
                    iChange+=1

            if (iChange > 0):
                changes+=1
                rows.updateRow(row)
        printit(" Made {0} changes".format(changes))
        del row
        del rows

    printit("Main")

if (initialQC()==True):
    main()

printit("Done")

Arcpy: Check if a field exists

I was helping a co-worker who needed to check if a field exists in their arcpy script. Since we were located at their computer, I thought I would just do a quick Google search and pull the code off this blog. Seemed logical since I the original purpose was exactly that—to serve as a handy, public place to store code snippets that I use & that others might find handy.

Anyhow, my Google Search on “Node Dangles field Exists” came up with a 9.3 script to check if field index exists. And I also have a 10.0 version but did not come up with the field exists snippet. So here it is:

image

def fieldExists(inFeatureClass, inFieldName):
   fieldList = arcpy.ListFields(inFeatureClass)
   for iField in fieldList:
      if iField.name.lower() == inFieldName.lower():
         return True
   return False

Unsupported Arc: “Rebox”ing or updating the extent of a feature class.

I’ve found that sometimes I can not find the answer to a question until I know the answer & then it becomes ridiculously easy to find the answer.

One small annoying thing that I never spent much time was when you delete features from a feature class making it significantly smaller but the envelope does not get re-sized so the zoom extent (still the original extent) is too large. This often happens to use when we convert tables to an XY theme and there are blank records–most of our data shows in Minnesota but there are some in Oklahoma (I think). Once we eliminate or correct the blank records, our data view still pops out to include a large section of the United States even though we only have data in Minnesota.

A long, long time ago, Workstation ArcInfo had a simple command, Rebox, for just this purpose (actually it still does, I just don’t get to use it anymore)–it shrunk the extent to the smallest rectangle required to enclose all the data. Up until today, I thought the request for this feature was completely ignored.

While researching something else, I was digging around in the sde tables and found one, sde.sde_layers, that had the interesting fields, minx, miny, maxx, and maxy. My quick & dangerous test (I performed it on a throw-away feature class in a throw-away geodatabase) gave me the results I wanted–once I loaded the feature class into ArcMap, the extent was a nice, tight rectangle around my features.

Is this a supported way to Rebox the extent? No.

Is it recommend by ESRI or me? No.

Will it screw up your entire geodatabase, making you lose all your data & costing you your job? Probably not but do you want to take that chance?

Will it get the job done? Maybe.  But in the process of writing this post, I found two safer ways to go about it. First, the straight-forward, sde command-line way that probably always existed that I never found until today, sdelayer -o alter had an -E option to reset the extent, including the ability to either specify it or have sde calculate it. Ok, that is usable for one person in our organization.

Previously, we had found either a VBA or other tool for doing this but had minimal success with it. Today, I found an ArcGIS 10 Add-In that is suppose to do the same thing. In my experiments (sample size n=1) it worked perfectly. If you need this sort of functionality, I would recommend trying out this Add-In first, if that fails go the sde command line route. Use the direct SQL method at your own risk!

Quick & Dirty arcpy: Autopan ArcMap using arcpy

Question: How do I get ArcMap to automatically pan through an area.

As I mentioned in a previous post, I recently had the need to have ArcMap automatically pan through a project area. My first attempt was to print a series of data-driven pages (using a fishnet polygon layer as the index) this but that did not accomplish what I needed so I switched to arcpy, which made the task simple enough. Nothing special or tricky about this code, but just did not find it anywhere else.

The one thing to note is that I have a 1 second pause between pans–this was to allow image tiles to download. You will need to adjust the delay to meet your needs. The toolbox and code can also be downloaded.

import sys,arcpy,datetime
inLayer = sys.argv[1]

def printit(inMessage):
    print inMessage
    arcpy.AddMessage(inMessage)

mxd = arcpy.mapping.MapDocument("CURRENT")

arcpy.MakeFeatureLayer_management(inLayer, "indexLayer")
cur=arcpy.SearchCursor("indexLayer")

df = arcpy.mapping.ListDataFrames(mxd)[0]
newExtent = df.extent

iCount = 0
iTotal = (arcpy.GetCount_management("indexLayer").getOutput(0))

for row in cur:
    thisPoly = row.getValue("Shape")
    newExtent.XMin, newExtent.YMin = thisPoly.extent.XMin, thisPoly.extent.YMin
    newExtent.XMax, newExtent.YMax = thisPoly.extent.XMax, thisPoly.extent.YMax
    df.extent = newExtent
    time.sleep(1)
    iCount+=1
    printit("Panned to feature {0} of {1}".format(iCount,iTotal))

del row
del cur

Domain Sorter Add-In Version 1.1

Almost a year ago, I updated ERSI’s Domain Sort code for VB 6 to work with ArcGIS 10. Recently, I had a comment that this Add-In caused ArcCatalog to explode if you had an open OLE connection. When I tested it, it turned out the reports were accurate.

I got around to adding in a Try-Catch around the offending chunk of code & it is now better than ever. You can download just the Add-In or the Add-In with source code or get it from ESRI’s ArcGIS Resource Center.

ArcMap Field Calculator: Create a Unique ID

One of the common functions I have to do is assign each record in a feature class with a unique identifier–normally just a sequential number from 1 to N.  In ArcView 3.x, the formula was simply “rec + 1” if I wanted to start with the number 1.

In ArcGIS, the process got a little more complex–you had to write a little VBA in Field Calculator as described by ESRI.

While this option still exists in ArcGIS 10, I believe it will disappear when 10.1 comes out and VBA support is completely eliminated.  But it is doable using Python which will continue to be supported.

Googling around, I did not find an exact answer but Dave Verbyla, Professor of GIS/Remote Sensing at the University of Alaska has a posted some samples that served as a good starting point.

In the Pre-Logic Script Code box, I declare a variable (counter) and a function. Then in the formula, I call the function.

counter = 0
def uniqueID():
  global counter
  counter += 1
  return counter

While composing this post, I actually wanted a concatenated value; “OC” plus an 8 character numeric sequential number starting at OC00000001 so the actual code is shown below: