Sunday, April 17, 2016

Domains: Batch Domain Loading


Do you work on the IS side of GIS, in ESRI geodatabases? The day may come when you are faced with culling domain values from what seems like a gazillion tables, organizing and verifying the values, and loading them into multiple domains.


Sorting through domain values by visually inspecting each column and loading domains one by one can get tiresome. It may be that you decide that the best thing to do is to learn how to script and automate much of the domain value sorting, validating and loading process.


Whether you are just starting out and want some tips, or whether you don't like scripting and just want something that works--this blog is for you.


This series of posts on domains will put you on your way. Each post describes a python script tool that can be added to an ArcGIS custom toolbox. Collectively, the tools automate the domain workflow, reduce errors, and save time.

Summary Of Tool


The Batch Domain Loading tool loads values to multiple geodatabse coded value domains from a single input table. Each domain code value must be in a separate row in the loading table.


The following columns must exist in the domain loading table:


  1. Domain name

  2. Code (value)

  3. Description

  4. Domain column type, i.e., Long, Double, Text, etc.


Domain values are only loaded if the domain already exists (guards against misspellings).


With the APPEND option, if an existing domain value matches a value to be loaded, the existing value is deleted and replaced.


Domain values are sorted by code within each domain after being loaded.


This tool supports all levels of geodatabases.


Illustration

Batch Domain Loading tool illustration

Usage


Syntax


BatchTableToDomain_mytools (Domain_Value_Table, Code_Column, Description_Column, Domain_Column_Type_Column, Domain_Name_Column, Input_Workspace, Update_Option)


Parameter Explanation Data Type
Domain_Value_Table Dialog Reference

Domain loading table


Python Reference

Doamin loading table

Table View
Code_Column Dialog Reference

Coded domain value column, obtained from Domain_Values_Table


Python Reference

Coded domain value column, obtained from Domain_Values_Table

Field
Description_Column Dialog Reference

Coded domain description column, obtained from Domain_Values_Table


Python Reference

Coded domain description column, obtained from Domain_Values_Table

Field
Domain_Column_Type_Column Dialog Reference

Domain column type field, obtained from Domain_Values_Table


Python Reference

Domain column type field, obtained from Domain_Values_Table

Field
Domain_Name_Column Dialog Reference

Domain name column, obtained from Domain_Values_Table


Python Reference

Domain name column, obtained from Domain_Values_Table

Field
Input_Workspace Dialog Reference

Target workspace for domain values


Python Reference

Target workspace for domain values

Workspace
Update_Option Dialog Reference

APPEND or REPLACE target domain values


Python Reference

APPEND or REPLACE target domain values

String

Code Samples

BatchDomainLoading.py


See below for tool source code.


import arcpy, traceback, sys
arcpy.env.overwriteOutput = True

def get_distinct_col_val(in_layer, in_col):
    # RETURNS A LIST OF THE UNIQUE VALUES CONTAINED IN THE INPUT COLUMN.
    l = [row[0] for row in arcpy.da.SearchCursor(in_layer, in_col)]
    list_of_distinct_values = set(l)
    return list_of_distinct_values
    
def dict_from_2cols(in_layer, col_1, col_2):
    # CREATES A DICTIONARY FROM TWO COLUMNS; COL 1 IS KEY, COL 2 IS VALUE.
    d = {}
    with arcpy.da.SearchCursor(in_layer, [col_1, col_2]) as c:
        for r in c:
            d[r[0]] = r[1]
    return d

def dict_from_coded_value_domains(input_wksp):
    # CREATES A DICTIONARY FROM CODED VALUE DOMAINS. KEY=DOMAIN NAME; VALUE=DOMAIN VALUES.
    d = {}
    doms = arcpy.da.ListDomains(input_wksp)
    for dom in doms:
        if dom.domainType == 'CodedValue':
            d[dom.name] = dom.codedValues.items()
    return d
    
def get_delimited_field(in_layer, fld):
    # RETURNS A DELIMITED FIELD NAME STRING FOR INPUT FIELD
    d = arcpy.Describe(in_layer)
    dCatPath = d.catalogPath
    fld_delimited = arcpy.AddFieldDelimiters(dCatPath, fld)
    return fld_delimited
    
def get_sel_count(in_layer):
    # RETURNS SELECTION COUNT AS INTEGER
    selCnt = int(arcpy.GetCount_management(in_layer).getOutput(0))
    return selCnt

if __name__ == "__main__":

    domain_value_table = arcpy.GetParameterAsText(0) # TABLEVIEW
    code_field = arcpy.GetParameterAsText(1) # FIELD, OBTAINED FROM DOMAIN_VALUE_TABLE
    desc_field = arcpy.GetParameterAsText(2) # FIELD, OBTAINED FROM DOMAIN_VALUE_TABLE
    type_field = arcpy.GetParameterAsText(3) # FIELD, OBTAINED FROM DOMAIN_VALUE_TABLE
    domain_name_field = arcpy.GetParameterAsText(4) # FIELD, OBTAINED FROM DOMAIN_VALUE_TABLE
    input_wksp = arcpy.GetParameterAsText(5) # WORKSPACE
    update_option = arcpy.GetParameterAsText(6) # STRING # FILTER, APPEND OR REPLACE
    
    target_ddesc = ""

    try:

        arcpy.AddMessage(" ")

        # RETRIEVE LIST OF SOURCE DOMAIN NAMES FROM DICTIONARY
        src_dn_list = get_distinct_col_val(domain_value_table, domain_name_field)
        
        # CREATE DICTIONARY OF TARGET CODED-VALUE DOMAIN NAMES AND VALUES
        targ_dn_dict = dict_from_coded_value_domains(input_wksp)
        targ_dn_list = targ_dn_dict.keys()
        
        # GET DELIMITED SOURCE DOMAIN NAME FIELD
        domain_name_field_delim = get_delimited_field(domain_value_table, domain_name_field)
        
        # ITERATE SOURCE DOMAIN NAMES...    
        for src_dn in src_dn_list:
        
            domain_exists = False

            # SELECT SOURCE DOMAIN VALUES IN LOADING TABLE FOR THIS DOMAIN
            expr = domain_name_field_delim + " = '" + str(src_dn) + "'"
            arcpy.SelectLayerByAttribute_management(domain_value_table, "CLEAR_SELECTION")
            etl_tab_sel = arcpy.SelectLayerByAttribute_management(domain_value_table, "NEW_SELECTION", expr)
            cnt = get_sel_count(domain_value_table)
            
            # ASSUME TARGET DOMAIN NAME IS SAME AS SOURCE
            target_dn = src_dn
            
            # IF ANYTHING IS SELECTED...
            if cnt > 0:
                # IF TARGET DOMAIN EXISTS... (DON'T WANT TO LOAD MISSPELLED DOMAIN)
                domain_exists = target_dn in targ_dn_list
                if domain_exists == True:
                    # OUTPUT SELECTED SOURCE DOMAIN VALUES IN LOADING TABLE TO TEMP TABLE
                    arcpy.Delete_management("in_memory")
                    arcpy.TableToTable_conversion(domain_value_table, "in_memory", "src_dom_val")
                    # ASSUMES DOMAIN COLUMN TYPE IS TEXT TO START WITH
                    dom_col_type = "Text"
                    # SET DOMAIN COLUMN TYPE TO DOUBLE IF APPROPRIATE
                    with arcpy.da.SearchCursor("in_memory/src_dom_val", type_field) as type_cur:
                        for row in type_cur:
                            if row[0] == "Double":
                                dom_col_type = "Double"
                                break
                    # IF DOMAIN COLUMN TYPE IS DOUBLE,
                    # ADD A CODE_DBL COLUMN AND POPULATE IT.
                    if dom_col_type == "Double":
                        arcpy.AddField_management("in_memory/src_dom_val", "code_dbl", field_type="Double", field_alias="code_dbl")
                        arcpy.CalculateField_management("in_memory/src_dom_val", "code_dbl", "!" + code_field + "!", "PYTHON_9.3")
                    # IF UPDATE OPTION IS REPLACE, RUN TABLE TO DOMAIN TOOL
                    if update_option == "REPLACE":
                        arcpy.AddMessage("     Replacing %s domain with domain and values from %s domain..." % (target_dn, src_dn))
                        if dom_col_type == "Double":
                            arcpy.TableToDomain_management("in_memory/src_dom_val", "code_dbl", desc_field, input_wksp, target_dn, target_ddesc, update_option)
                        else:
                            arcpy.TableToDomain_management("in_memory/src_dom_val", code_field, desc_field, input_wksp, target_dn, target_ddesc, update_option)
                    # ELSE IF UPDATE OPTION IS APPEND...
                    else:
                        arcpy.AddMessage("     Appending %s domain with domain and values from %s domain..." % (target_dn, src_dn))
                        # CREATE LIST OF ALL TARGET DOMAIN VALUES
                        val_to_del = []
                        all_target_val = []
                        all_target_items = targ_dn_dict[target_dn]
                        for val, desc in all_target_items:
                            all_target_val.append(val)
                        # CREATE LIST OF TARGET DOMAIN VALUES THAT MATCH SOURCE
                        # DOMAIN VALUES TO BE APPENDED. THESE WILL BE DELETED.
                        with arcpy.da.SearchCursor("in_memory/src_dom_val", code_field) as temp_cur:
                            for row in temp_cur:
                                if row[0] in all_target_val:
                                    val_to_del.append(row[0])
                                    arcpy.AddMessage("          Replacing value " + row[0])
                        # DELETE EXISTING VALUES
                        if len(val_to_del)>0:
                            arcpy.DeleteCodedValueFromDomain_management(input_wksp, target_dn, val_to_del)
                    
                        # RUN TABLE TO DOMAIN TOOL
                        if dom_col_type == "Double":
                            arcpy.TableToDomain_management("in_memory/src_dom_val", "code_dbl", desc_field, input_wksp, target_dn, target_ddesc, update_option)
                        else:
                            arcpy.TableToDomain_management("in_memory/src_dom_val", code_field, desc_field, input_wksp, target_dn, target_ddesc, update_option)
                        
                    # SORT DOMAIN BY CODE
                    arcpy.AddMessage("          Sorting domain...")
                    arcpy.SortCodedValueDomain_management(input_wksp, target_dn, "CODE", "ASCENDING")
                    
                else:
                    arcpy.AddMessage("     Target domain %s not found. Source domain %s values not added." % (src_dn, target_dn))
            # ELSE LET USER KNOW DOMAIN VALUES WERE NOT FOUND
            else:
                arcpy.AddMessage("     No values found for source domain %s, nothing added to target domain %s" % (src_dn, target_dn))
                
        arcpy.SelectLayerByAttribute_management(domain_value_table, "CLEAR_SELECTION")
                
        arcpy.AddMessage(" ")
        
        # SET OUTPUT PARAMETER
        arcpy.SetParameter(7, input_wksp) # WORKSPACE
        
    except:

        # PRINT ERROR MESSAGES
        tb = sys.exc_info()[2]
        tbinfo = traceback.format_tb(tb)[0]
        pymsg = tbinfo + "\n" + str(sys.exc_type)+ ": " + str(sys.exc_value)
        arcpy.AddError("Python Messages: " + pymsg + " GP Messages: " + arcpy.GetMessages(2))
        
    finally:
        arcpy.SelectLayerByAttribute_management(domain_value_table, "CLEAR_SELECTION")
        del domain_value_table, code_field
        del desc_field, domain_name_field, input_wksp, update_option
        arcpy.Delete_management("in_memory")

Tags


Domain, geodatabase, TableToDomain, dictionary, DeleteCodedValueFromDomain, TableToTable, SortCodedValueDomain, AddField, SearchCursor, CalculateField, SelectLayerByAttribute, Describe, ListDomains, GetCount, AddFieldDelimiters

Credits


Ruth Bowers, 2015


Use limitations


Tested as script tool in ArcGIS 10.2.2


No comments:

Post a Comment