Converting MXD to Layer file in Arcpy

Working on doing some advanced ArcGIS server printing and had the need to batch convert many existing .mxd files to .lyr files. So instead of opening up X number of map documents, thought I would do it via code. All of my .mxds in this case had just one data frame so the process was pretty simple–I add an empty group layer (Thanks Petr Krebs for the idea), copy all the existing layers into it, and save it out as a layer file.

I created an ArcGIS toolbox with two options–one to convert a single .mxd and one to batch convert an entire folder. To use it, make sure to have the EmptyGroup.lyr in the same directory as the .py file.

Here is the raw code or git it:


import os
import arcpy
import inspect
import glob
import uuid
import inspect

codeDir = os.path.dirname(inspect.getfile(inspect.currentframe()))
EmptyGroupLayerFile = codeDir+"/EmptyGroup.lyr"
inArg1 = sys.argv[1]
inArg2 = sys.argv[2]

def printit(inMessage):
    arcpy.AddMessage(inMessage)

def makeLyrFromMXD(inMXD, outLyr):
    if not (os.path.exists(inMXD)):
        printit( "ERROR: {} does not exist".format(inMXD))
        return False
    if not (os.path.exists(EmptyGroupLayerFile)):
        printit( "ERROR: {} does not exist".format(EmptyGroupLayerFile))
        return False
    if  (os.path.exists(outLyr)):
        printit( "Skipping: {} already exists".format(outLyr))
        return True

    printit( "Making Layer file: {0}".format(outLyr))

    mxd = arcpy.mapping.MapDocument(inMXD)
    ###Right now, just doing the first Dataframe, this could be modified
    df = arcpy.mapping.ListDataFrames(mxd)[0]

    theUUID = str(uuid.uuid1())

    iGroupLayerRaw = arcpy.mapping.Layer(EmptyGroupLayerFile)
    iGroupLayerRaw.name = theUUID
    arcpy.mapping.AddLayer(df,iGroupLayerRaw,"TOP")
    groupBaseName = os.path.basename(outLyr).split(".")[0]

    for lyr in arcpy.mapping.ListLayers(df):
        if not (lyr.name == theUUID):
            if (lyr.longName == lyr.name):
                arcpy.mapping.AddLayerToGroup (df, iGroupLayer, lyr, "Bottom")
        else:
            iGroupLayer = lyr

    iGroupLayer.name = groupBaseName
    arcpy.SaveToLayerFile_management(iGroupLayer, outLyr)
    return os.path.exists(outLyr)

def doMultiple(inDir,outDir):
    for iMxd in glob.glob(inDir+"/*.mxd"):
        lyrFile = outDir+"/"+os.path.basename(iMxd).lower().replace(".mxd",".lyr")
        makeLyrFromMXD(iMxd, lyrFile)

if(not os.path.exists(EmptyGroupLayerFile)):
    printit("Error: {} is missing, can not run.".format(EmptyGroupLayerFile))
else:
    if (os.path.isdir(inArg1) and (os.path.isdir(inArg2))):
        doMultiple(inArg1,inArg2)
    elif (os.path.isfile(inArg1)):
        if (os.path.exists(inArg2)):
            printit("Error: {} already exists".format(inArg2))
        else:
            makeLyrFromMXD(inArg1,inArg2)
    else:
        printit("Unable to understand input parameters")

Zipping a Shapefile from ArcCatalog

Back in 2010, I posted a python script and an ArcToolbox tool for zipping a shapefile.

Well, I had a request to modify the code so it would not error out if it encounters a .lock file. While .lock files exist for a reason and shouldn’t be totally ignored, in some cases it is safe to do so, so I went ahead any modified the code, which can be downloaded from Github.

The guts of the code is here, though:

import zipfile
import sys
import os
import glob

theShapeFile = sys.argv[1]
outputZipFile = sys.argv[2]
skipLockFile = sys.argv[3]

def zipShapefile(inShapefile, newZipFN, skipLockFile):
    print 'Starting to Zip '+inShapefile+' to '+newZipFN

    if not (os.path.exists(inShapefile)):
        print inShapefile + ' Does Not Exist'
        return False

    if (os.path.exists(newZipFN)):
        print 'Deleting '+newZipFN
        os.remove(newZipFN)

        if (os.path.exists(newZipFN)):
            print 'Unable to Delete'+newZipFN
            return False

    zipobj = zipfile.ZipFile(newZipFN,'w')

    for infile in glob.glob( inShapefile.lower().replace(".shp",".*")):
        print infile
        if not ((os.path.splitext(infile.lower())[1] == ".lock") and (skipLockFile.lower() == "true")):
            zipobj.write(infile,os.path.basename(infile),zipfile.ZIP_DEFLATED)

    zipobj.close()

    return True

zipShapefile(theShapeFile,outputZipFile,skipLockFile)
print "done!"

Friday Fave: Geodatabase Geek

This Friday Fave is more for utility than pleasure.

Unfortunately, I have been working to determine why my views and query layers perform so much worse than directly accessing my feature class.

My Googling led me to Geodatabase Geek, by Trevor Hart, Eagle Technology Group Ltd.  Trevor has some real good information about Geodatabases and also  gave a good lightening talk on Usage Reporting on ArcGIS 10.1 for Server at the 2013 ESRI International Developer’s Conference.

One tool he pointed out was Mxdperfstat for benchmarking the performance of your MXD. Trevor used it to compare the performance of a Feature Class vs Query Layer vs Spatial View. While the official version is available for ArcGIS 9.3 through 10.2, I do want to point out Hussein Nasser’s 10.1 version which he put out before the official 10.1 version came out (it’s not really a version, more of a work-around but I like his ingenuity).

My results were significantly different on our 10.0 database server, the spatial view I was testing was much slower.  The query for both the spatial view and query layer was simply “Select * from featureclass

So not sure what to make of the performance yet, I’ve got a spatial index made so not sure what else I can try.

ArcSDE 10.0 Performance
ArcSDE 10.0 Performance

 

ArcMap Maximum Legend Items from a ArcGIS Server service

Recently we took a call from a user who could not see the legend for one of the feature classes in one of our services. (Precambrian Bedrock in http://mgsweb2.mngs.umn.edu/arcgis/services/state/mnbdrkgeology )

After trying some standard things–restarting the service, checking the source .MXD–I turned to The Google Machine and quickly found help from ESRI: http://support.esri.com/zh-cn/knowledgebase/techarticles/detail/33741 .

Turns out the default number of legend items ArcMap will display from an ArcGIS Server map service layer is 100 and we had 102 in the problematic layer. (I plead innocence, blame the geologist for needing that many categories).

The solution was to edit the Windows registry and change the setting for “Maximum Legend Count” from 100 to something higher than 102. After doing that (see path note below) and restarting ArcMap, the legend showed for us.

Path Details
The path turned out to be a little different on Windows 7 than the paths indicated in the help article.
ESRI indicated that the path varied by ArcGIS version:

  • ArcGIS 9.3.x: HKEY_CURRENT_USER > Software > ESRI > MapServerLayer
  • ArcGIS 10.0: HKEY_CURRENT_USER > Software > ESRI > Desktop10.0 > ArcMap > Server > MapServerLayer
  • ArcGIS 10.1: HKEY_CURRENT_USER > Software > ESRI > Desktop10.1 > ArcMap > Server > MapServerLayer

I actually changed this in a number of locations–presumably once for each user profile on my machine. Each followed a pattern something like:

HKEY_USERSS-1-5-23412341234-123412342134-12341234123-1302SoftwareESRIArcMapServerMapServerLayer

Extra Information

Out of cursiousity, I wondered if the 100 item limit was a per service or per layer limitation so I set my limit at 103. Because the service has several other layers, the total number of layers in the legend was about 130. Everything drew OK so it appears the limit applies per layer.

ArcSDE Tip: Finding an ArcSDE instance’s port

One of my main tasks right now is to document many of the details of maintaining ArcSDE geodatabases so I anticipate having several blog posts on this topic that are re-writes of documents I am working on. I am presuming that the person will have no ArcSDE experience so I am documenting very detailed information.

Almost all of the ArcSDE commands require that you specify which instance (service/port) the command applies to by using the “-i” parameter.

ArcSDE Instance Parameter
ArcSDE Instance Parameter

Since we have multiple ArcSDE geodatabases, I like to have a handy-dandy sticky note with all the geodatabases and their respective ports on the side of my monitor.

But when that is not handy and I can never remember the ArcSDE command line syntax to get a list of instances and their ports–I mean remembering “sdeservice -o list” is difficult at my age.

sdeservicelist

The quickest and most reliable way I’ve found to get the instance number is just to check the properties of your SDE connection file in ArcCatalog, right-click on it and select “Connection Properties”.

connectionproperties

And the port is right there in the service entry (5164).

connectionproperties2

ArcMap Field Calculator: Identifying Unique Cases, Multiple Fields

You may have noticed that this post–ArcMap Field Calculator: Identifying Unique Cases, Single Field–specifies “Single Field”. Yes, that was my version of a cliff-hanger post.

The basic structure I listed in that post can be expanded on to satisfy your needs. The example in my earlier post was case sensitive for example, you could modify it so it treats “a” the same as “A”.

Today’s example groups records into different cases based off the values of two fields, !county_c! and !feature! and required only minor modifications.

The calling line was modified from:

returnCase(!feature!)

to:

returnCase(!county_c!,!feature!)

to accommodate passing both values.

The function definition likewise was modified to accept two values, this:

def returnCase(inValue1):

to:

def returnCase(inValue1, inValue2)

And this line was added, creating a list from the two values passed in:

inValue = [inValue1, inValue2]

 

(Note: The same results could have be achieved by using the original function by creating the list in the calling statement:  returnCase([!county_c!,!feature!] )

 

caseList = [ ]

def returnCase(inValue1, inValue2):
   inValue = [inValue1, inValue2]
   global caseList

   if not inValue in caseList:
      caseList.append(inValue)

   return caseList.index(inValue)

Data Extration Failed — Reverse Proxy Time-out

Testing one of our geodata services, we discovered that it allowed us to extract a portion of our feature class but when we tried to extract the entire data set, we received this Data Extraction error: Data extraction failed. Proxy or Gateway Server did not allow the URL. Check with your LAN administrator that Proxy or Gateway server is configured to allow the URL.

The fact that I was able to extract a portion of the data and I could see the entire geodatabase get made and zipped led me to believe it was more of time-out issue.

Reading through this thread at ArcForum led to some good information.  But Thomas’ comment that he was using “IIS7 for my reverse proxy server” and had to change one more setting led me to the solution.  In Server Manager, the default Proxy Time-Out is set at 30 seconds by default.  I bumped that up, 60 seconds shown below but I ended up going to 300 seconds and the problem was resolved.

ArcMap Field Calculator: Create a Unique ID

One of the common functions I have to do is assign each record in a feature class with a unique identifier–normally just a sequential number from 1 to N.  In ArcView 3.x, the formula was simply “rec + 1” if I wanted to start with the number 1.

In ArcGIS, the process got a little more complex–you had to write a little VBA in Field Calculator as described by ESRI.

While this option still exists in ArcGIS 10, I believe it will disappear when 10.1 comes out and VBA support is completely eliminated.  But it is doable using Python which will continue to be supported.

Googling around, I did not find an exact answer but Dave Verbyla, Professor of GIS/Remote Sensing at the University of Alaska has a posted some samples that served as a good starting point.

In the Pre-Logic Script Code box, I declare a variable (counter) and a function. Then in the formula, I call the function.

counter = 0
def uniqueID():
  global counter
  counter += 1
  return counter

While composing this post, I actually wanted a concatenated value; “OC” plus an 8 character numeric sequential number starting at OC00000001 so the actual code is shown below:

ArcGIS Add-In Custom Mouse Cursor

I was working on a project and wanted my own custom mouse cursor and did not easily find a way to make your own in ESRI’s instructions.  But, once you know how to do it, it is pretty easy.  In Visual Studio, Add a New Item:

Add a Cursor File:

You can edit your cursor with the editor program in Visual Studio.  Once you satisfied with how it looks, make sure that the Build Action on the cursor is “Embedded Resource”.

Then you can set your cursor with two lines of code. Not that my cursor is in my QDI.QdiAddIn Namespace:

       
Dim pCursorStream As System.IO.Stream = Me.GetType.Assembly.GetManifestResourceStream("QDI.QdiAddIn.NewCursor.cur")
MyBase.Cursor = New System.Windows.Forms.Cursor(pCursorStream)

Sorting a Coded-Value Domain Add-In (ArcGIS 10)

I am working on an data-entry application to edit feature classes that contain several coded-value-domains. The problem with some of the domains, however, is that some entries have been added after the initial creation.  So the first 25 entries are in alphabetical order and there are some stragglers at the end that are in the order they were appended.

This can be confusing for users–they go to select “Milli Vanilli” and look between “Madonna” and “Motley Crue” but can not find their favorite band there–they have to go to the end of the list to find their selection.

In the past, I have gone through the tedious process of exporting the domain to a table, sorting the table, removing the domain from the necessary field(s), deleting the domain, re-importing the table back in a new domain and finally re-applying the domain to the necessary field(s). Let’s just say I didn’t do this until someone asked a few times and I didn’t have anything more exciting–like a root canal–I could busy myself with.

But this new application contains more domains than any of other datasets so it was time to find a better solution. ESRI does have a Domain Sort Developer Sample.  It, however, did not play nice with ArcGIS 10.

So I went ahead and update it from VB 6 to VB.Net/ArcObjects 10.  I made an Add-In that can be installed by downloading the .esriaddin file and double-clicking on it.  The source code is also available.

This will add an ArcCatalog Toolbar that can be added by going to Customize-Toolbars-Domain Sorter Toolbar.

This will add a toolbar with one button.  The button enables whenever you select a geodatabase with at least one coded-value domain.

This brings up a Windows form that lets you sort any domain by either the code or description, ascending or descending.  Once you hit “OK” it re-sorts your domain.

The only problem I have had is that only the owner of a domain is allowed to edit it on an SDE geodatabase.

But other than that, the button allows you to easily keep your domains sorted.

http://edndoc.esri.com/arcobjects/9.2/CPP_VB6_VBA_VCPP_Doc/COM_Samples_Docs/Geodatabase/Schema_Creation_and_Management/Sort_a_domain/e826c5a8-9740-4f0b-86b6-d3b834735574.htm