Customer Service | Training | Contact Us
You are here: Home > User Forums > arcgis desktop discussion forums > Thread Replies

ArcGIS Desktop Discussion Forums

ArcGIS Desktop - Geoprocessing Scripting (Python, JavaScript, VB) forum

How to set field to NULL with update cursor...   Jason Roberts Jan 06, 2008
Re: How to set field to NULL with update cu...   Dan Patterson Jan 06, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Jan 06, 2008
Re: How to set field to NULL with update cu...   Chris Snyder Jan 07, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Jan 07, 2008
Re: How to set field to NULL with update cu...   Chris Snyder Jan 07, 2008
Re: How to set field to NULL with update cu...   Chris Snyder Jan 07, 2008
Re: How to set field to NULL with update cu...   David Fawcett Jan 16, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Jan 16, 2008
Re: How to set field to NULL with update cu...   Chris Snyder Jan 16, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Jan 16, 2008
Re: How to set field to NULL with update cu...   Adam Cabrera Jan 16, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Jan 16, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Jan 17, 2008
Re: How to set field to NULL with update cu...   Karen Holt Mar 28, 2008
Re: How to set field to NULL with update cu...   David Fawcett Jul 09, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Jul 16, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Jul 16, 2008
Re: How to set field to NULL with update cu...   Applied Technology Aug 07, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Aug 07, 2008
Re: How to set field to NULL with update cu...   David Fawcett Aug 07, 2008
Re: How to set field to NULL with update cu...   Chad Cooper Aug 07, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Aug 07, 2008
Re: How to set field to NULL with update cu...   Applied Technology Aug 07, 2008
Re: How to set field to NULL with update cu...   Chad Cooper Aug 07, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Aug 07, 2008
Re: How to set field to NULL with update cu...   Applied Technology Aug 07, 2008
Re: How to set field to NULL with update cu...   Chad Cooper Aug 08, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Aug 08, 2008
Re: How to set field to NULL with update cu...   Philippe Le Grand Aug 08, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Aug 08, 2008
Re: How to set field to NULL with update cu...   Kim Ollivier Aug 31, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Sep 02, 2008
Re: How to set field to NULL with update cu...   Jason Roberts Sep 12, 2008
Re: How to set field to NULL with update cu...   Christopher J May 26, 2009
Re: How to set field to NULL with update cu...   Christopher J May 26, 2009
Re: How to set field to NULL with update cu...   Jason Roberts May 26, 2009
Re: How to set field to NULL with update cu...   Kris Stitt Jun 04, 2009
Re: How to set field to NULL with update cu...   Jason Roberts Jun 04, 2009
Re: How to set field to NULL with update cu...   Chris Snyder Oct 07, 2009
Re: How to set field to NULL with update cu...   Jason Roberts Oct 07, 2009
Re: How to set field to NULL with update cu...   Chris Snyder Oct 07, 2009
Re: How to set field to NULL with update cu...   Jason Roberts Oct 07, 2009
Re: How to set field to NULL with update cu...   Jason Roberts Oct 13, 2009
Re: How to set field to NULL with update cu...   Jason Roberts Oct 15, 2009
Report Inappropriate Content • Top • Print • Reply    
Subject How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Jan 06, 2008 
Message In both ArcGIS 9.1 and 9.2, if you create the geoprocessor through win32com (the "old" way of doing it), you can set nullable database field to NULL by setting is value to a Python None. But in 9.2, if you create the geoprocessor by importing the arcgisscripting Python module, you get an error. Does anyone know how to set a field to NULL with arcgisscripting?

The example code below shows what I'm talking about. I executed these from PythonWin. The first block simply creates a table with a nullable field and inserts a row with the value 12345. The second block shows that the field may be set to NULL by passing None to cur.SetValue, using the geoprocessor created with COM automation. The third block shows that when the geoprocessor is created with arcgisscripting, the NULL value may be read from the field (it is returned as Python None), and the value 12345 may be written to the field, but that Python None may not be written to the field.

Incidentally, I discovered that the update cursor object returned by the arcgisscripting geoprocessor has two undocumented functions:

IsNull - this appears to return 1 if a given field value is NULL and 0 if it is not NULL.

SetNull - I thought this was what I should use, but I cannot get it to work. It appears that you should be able to pass the field name and it would set the field to NULL, but as you can see in the third example, it doesn't work.

Anyone have any ideas? I believe this is a bug in 9.2 but I wanted to check with folks before I submitted a bug. I am running 9.2 SP4.

Best regards,

Jason 
 
Code block #1: creates the table with a nullable field:

>>> import arcgisscripting
>>> gp = arcgisscripting.create()
>>> gp.CreatePersonalGDB_management('c:\\temp4', 'TestGDB.mdb')
'c:\\temp4\\TestGDB.mdb'
>>> gp.CreateTable_management('c:\\temp4\\TestGDB.mdb', 'TestTable')
'c:\\temp4\\TestGDB.mdb\\TestTable'
>>> gp.AddField_management('c:\\temp4\\TestGDB.mdb\\TestTable', 'MyField', 'LONG', '#', '#', '#', '#', 'NULLABLE')
'c:\\temp4\\TestGDB.mdb\\TestTable'
>>> cur = gp.InsertCursor('c:\\temp4\\TestGDB.mdb\\TestTable')
>>> row = cur.NewRow()
>>> row.SetValue('MyField', 12345)
>>> cur.InsertRow(row)
>>> del cur

Code block #2: shows that NULL can be written to the field if the geoprocessor is created with COM automation:

>>> import win32com.client
>>> gp = win32com.client.Dispatch('esriGeoprocessing.GPDispatch')
>>> cur = gp.UpdateCursor('c:\\temp4\\TestGDB.mdb\\TestTable')
>>> row = cur.Next()
>>> row.GetValue('MyField')
12345
>>> row.SetValue('MyField', None)
>>> cur.UpdateRow(row)
>>> del cur

Code block #3: shows that, if arcgisscripting is used, NULL can be read but not written:

>>> import arcgisscripting
>>> gp = arcgisscripting.create()
>>> cur = gp.UpdateCursor('c:\\temp4\\TestGDB.mdb\\TestTable')
>>> row = cur.Next()
>>> row.GetValue('MyField')            # Returns None, which PythonWin indicates by displaying nothing
>>> row.SetValue('MyField', 12345)
>>> cur.UpdateRow(row)
>>> del cur
>>> cur = gp.UpdateCursor('c:\\temp4\\TestGDB.mdb\\TestTable')
>>> row = cur.Next()
>>> row.GetValue('MyField')
12345
>>> row.SetValue('MyField', None)
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
ValueError: invalid input value
>>> row.MyField = None
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
ValueError: invalid input value
>>> row.IsNull('MyField')
0
>>> row.SetNull('MyField')
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
RuntimeError: Error in executing function
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Dan Patterson 
Date Jan 06, 2008 
Message just install pythonwin from the installation cd and see if it works then rather than using arcgisscripting 
  Geomatics, Carleton University, Ottawa, Canada 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Jan 06, 2008 
Message Dan,

Thanks for the suggestion, but I am specifically interested in using the geoprocessor returned by arcgisscripting, if at all possible. It will indeed work if the geoprocssor is obtained using win32com (as shown in my example code). But ESRI seems to be pushing the arcgisscripting method and I'm trying to follow their recommendation. Using arcgisscripting also enables new scensrios, such as allowing your script to run on non-Windows ArcGIS Server machines. Also, I read that in 9.3 ESRI is working on hosting the Python interpreter directly from ArcCatalog/ArcMap rather than invoking cmd.exe, which will dramatically improve performance. (As a developer, I can see why they would then want you to obtain the gp through the Python module rather than through COM automation, which would introduce much unneeded complexity.) The new scenarios might not be important to many people, but I publish a collection of open-source geoprocessing tools and I'm trying to maximize their potential use.

Any other ideas?

Thanks,
Jason
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Chris Snyder 
Date Jan 07, 2008 
Message Can you read a NULL value, capture it as a variable, and then insert it?

Or when running the update row, can you just update the fields that have values (leaving the NULLS unaltered/unspecified), and then insert it? 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Jan 07, 2008 
Message Good thoughts Chris. I tried reading the value of a NULL field and setting another field to this value, e.g.:

row.SetValue('field1', row.GetValue('field2'))

where field2 is NULL. But this produced the same error. I'm sure this is because the GetValue returns Python None, so it is no different than doing this:

row.SetValue('field1', None)

which is back to the original situation: you can do that with the COM automation geoprocessor object but not the one from arcgisscripting.

Your suggestion about leaving fields alone when they don't need to be updated (leaving them as NULL) is a good one for many situations. I haven't tested that but I think it works. Unfortunately there are cases where I explicitly need to set a field to NULL when it currently has a non-NULL value. That is the one situation where arcgisscripting seems to be busted...

Other thoughts?

Thanks,
Jason 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Chris Snyder 
Date Jan 07, 2008 
Message Can you insert something like:

nullStringValue = str(None)
nullIntValue = int(None)
nullDoubleValue = float(None) 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Chris Snyder 
Date Jan 07, 2008 
Message Woops I guess that doesn't work.... I was trying to backwards engineer some thing I used to handle NULL value evaluations in the FieldCalculator PYTHON codeblock option. 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author David Fawcett 
Date Jan 16, 2008 
Message I just ran into this issue today in a script that I was writing.

I don't know if this will work in your situation, but here is what I did. I have a script where I am querying data out of a database and writing it to a Python dictionary. I am then creating an empty feature class and using values from the dictionary to create the point features and populate the attribute columns.

In the cases where the column values are null, the dictionary item has a value of 'None'. This was throwing an error when I tried to set the value of attributes when the value was 'None'.

My solution was to test to see if the value of a dictionary item is equal to None and if that was the case, I don't set the value of that column.

In cases where I knew that the value wouldn't be null/None, I didn't do the test.

Hopefully there is something in here that can help you.

David.

Here is some simplified code:

 
 
for item in attributeDictList: 
    
    feat = icur.NewRow()
    pnt = gp.CreateObject("Point")
    
    #set coords of point
    pnt.x = item["utmx"]
    pnt.y = item["utmy"]
    feat.shape = pnt
            
    #set the values of the attribute columns
    feat.SetValue("ID", item["id"])
    if item["name"] != None:
        feat.SetValue("ENTITY_NAM", item["name"])
    icur.InsertRow(feat)
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Jan 16, 2008 
Message David,

Thank you very much for the suggestion. Unfortunately that is the same idea that Chris had in his first response ("Or when running the update row, can you just update the fields that have values (leaving the NULLS unaltered/unspecified), and then insert it?"). As I mentioned, there are cases where I explicitly need to set a field to NULL when it currently has a non-NULL value. This is a little different than your scenario. In your scenario, you're creating a feature class from scratch. In my scenario, I am working with an existing feature class that I need to update. Specifically, it is a situation where a non-NULL value obtained previously is found to be invalid, and I need to reset the value of the field to NULL. The problem is there is no way to do it. Once you set the fields to non-NULL values, you can't go back, due to this problem.

I have opened a bug for this with ESRI. They have not given me a bug number yet because they are still working on reproducing it. (It should not be hard...)

Jason
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Chris Snyder 
Date Jan 16, 2008 
Message Here's another idea:

1) When you encounter a row where you want to change a field value to NULL, copy or somehow temporarily store the geometry (I've never done this, but i think you can do it). Also, copy/store the attribute values.

2) Delete the row (this should delete the geometry as well).

3) Insert a new row and copy the attributes (but not specifying values you want NULLified) and then insert/paste or otherwise reconstruct the geometry.

This untested work around does seem like bending over backwards to do something simple, and would probably mess with the OIDs.


I agree this is a bug. Hopefully ESRI will fix it... 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Jan 16, 2008 
Message Chris,

Thanks. That's a pretty crazy workaround but I think it might work and is probably as good as anyone can do, so I'm going to give you the points for answering the question.

The OID problem would probably be significant for some people. Some other possible problems might be:

1. As far as I know, you cannot insert rows with an update cursor. So you'd have to delete the row and queue it up for insertion later, or something like that. I'm not sure how ArcGIS cursors implement transactions, but it might be an issue for people if they expected the whole update to occur as a transaction.

2. It would be hard to implement if you need to update a parent table. Because you have to delete and recreate the parent record, in principle you have to delete the child records (they might be deleted for you if the tables were created with ON DELETE CASCADE or something like that...)

I'll update this thread when I hear back from ESRI.

Jason
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Adam Cabrera 
Date Jan 16, 2008 
Message It's nice to see that I'm not the only one having this problem. Here is what we did to get around this issue.

In our case we have created a new table and are copying values and geometry to a new feature class where a specific condition is met. When we encounter a case where the value is NULL we do not use the SetValue on that item

-----------------------------------------
 
 
## CHANGED BECAUSE OF THE BUG IN SETVALUE
## IF IT HAS A VALUE, THEN SET, ELSE DO NOT SET
## 01/16/2008
if inValue is not None:
    newrow.SetValue(mField.Name, inValue)
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Jan 16, 2008 
Message Hi Adam,

Thanks for describing your scenario. Your solution is the same one described by Chris and David. It is good to see that several folks in the community have discovered the same solution to that scenario (inserting new rows). Now if only ESRI will fix the update scenario...

Jason
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Jan 17, 2008 
Message The ESRI support services person indicated that this is a known bug, NIM010734, and updated the bug report to indicate that another person had reported it, saying "when more people are affected by a problem, it carries more weight". I asked her to update the bug report to say that the three of you had experienced this problem as well. If anyone else sees it, consider reporting it to ESRI support services. I guess that will maximize the possibility that it will be fixed.

Thanks,
Jason 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Karen Holt 
Date Mar 28, 2008 
Message I'm running into the same problem also.

cheers,
karen
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author David Fawcett 
Date Jul 09, 2008 
Message Depending on what type of database you are using, and whether or not you are using all of the geodatabase functionality, there may be a work around.

I use SDE on top or Oracle. For cases where I just want to update column values, I write SQL updates directly against the database using the cx_oracle module.

I am not sure if there is a handy odbc module or something that you could use to connect to the access database via your python script and run a direct update query. 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Jul 16, 2008 
Message David -- thanks for the suggestion. There are handy Python packages that allow direct updating of Access databases. This approach won't work well for me, however, because I'm writing a generic tool that is supposed to be agnostic about what database is being used. My code would have to detect what kind of database it was and then invoke the appropriate code. I may end up resorting to this if ESRI cannot provide a solution.

Jason 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Jul 16, 2008 
Message Oh, and if anyone is wondering, this problem still exists in Arc 9.3. I reproed it this morning.... 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Applied Technology 
Date Aug 07, 2008 
Message I'm having this issue also, but I'd like to mention an additional, related problem.

After the first row is added with an insert cursor, subsequent rows have their initial values set as the previous row. They should be null.

This means a script that copies data from one dataset to another appears to work correctly, but upon closer inspection, many rows that should have NULL values actually have random garbage from a previous row.

Is anyone else seeing this?

I'm on 9.2, SP5 
  Mark Holder 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Aug 07, 2008 
Message Mark,

I am familiar with the exact behavior you mention. Sadly, I think ESRI considers that behavior to be a feature rather than a bug. I recall seeing it in documentation somewhere although I cannot pull it up now after looking for 3 minutes.

Obviously, with no way to set fields that should be NULL back to NULL after inserting a row with non-NULL values, this is a major problem.

I urge you to contact ESRI support and bring up the original issue that is the topic of this thread, as well as pointing out how the insert design is additionally broken due to this issue. If you read above, I was led to believe by ESRI support that additional reports of this same problem might increase the probability that they would fix it.

[rant]
My experience with filing bugs with ESRI is unfortunately quite terrible. I am a seasoned software developer and have been on the "other side" of a support channel. I know how important it is to receive 100% repro scenarios, with complete documentation, and how satisfying it is to fix an easy bug when this information came from the customer. I work hard to give ESRI this information, but sadly it has gone nowhere in 5 out of 5 cases. I am at the point that I may no longer file bugs with them because it is a waste of time. If anyone has a contact at ESRI that will take real accountability for bugs in their software, please contact me.
[/rant]

Jason
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author David Fawcett 
Date Aug 07, 2008 
Message People seem to dismiss the OpenSource stuff as 'not ready for prime time', but have you ever submitted a bug to any of the OpenSource projects like MapServer, FWTools (OGR/GDAL), or OpenLayers?

They often get fixed really fast, and the developers will often roll you a new binary if you can't compile a new one yourself.

At least ArcGIS now uses GDAL under the hood now...

David. 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Chad Cooper 
Date Aug 07, 2008 
Message Mark, I am seeing the same thing as you. My rows that should be NULL are getting set with the value from the previous record. I'm on 9.2 also. I am currently getting my gp thru arcgisscripting, but I'm about to change it to win32com and see if it works then. This is driving me nuts. 
  ---------------
chad 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Aug 07, 2008 
Message Chad,

You will see the same behavior if you create the geoprocessor through win32com. The difference is that you'll be able to set the field values to NULL with that geoprocessor. So, after inserting a row that contained non-NULL values, be sure to set those fields to NULL before you insert the next row (assuming that next row should have NULL for those fields).

Jason
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Applied Technology 
Date Aug 07, 2008 
Message I tried win32com.client, and it does allow None to be set into a field.

I can live with having to clear out the values in my code, but not with being unable to do so.

This makes cursors pretty much unusable with arcgisscripting.

 
  Mark Holder 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Chad Cooper 
Date Aug 07, 2008 
Message I just changed mine to win32com, and I can now set NULLs into my date field. I'm really glad I found this post, I was about to just do away with this one field and move on. Thanks to all who posted here. 
 
if len(str(w_spud)) > 5: 
    out_row.SetValue('SPUD_DATE', w_spud)
else:
    out_row.SetValue('SPUD_DATE', None)
 
  ---------------
chad 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Aug 07, 2008 
Message > This makes cursors pretty much unusable with arcgisscripting.

I agree 100%, and it is still the case with Arc 9.3. I urge everyone to open support requests with ESRI about this issue.

Thanks, and good luck,

Jason
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Applied Technology 
Date Aug 07, 2008 
Message I just submitted a bug report to ESRI on this. I looked up NIM010734 in ESRI's system, and found these interesting facts:

1) It was reported 8/2/2007 - over a year ago now

2) Its status is "deferred" for a future release - apparently NOT 9.3

3) its severity is "medium" (Failure or error with major functionality)

Whoever sets the priority for bug fixes needs a wakeup call. How is it even possible for "Failure or error with major functionality" to have status "deferred"? Not even in the QUEUE???

Will someone employed by ESRI please step up and at least comment on this?
 
  Mark Holder 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Chad Cooper 
Date Aug 08, 2008 
Message I also submitted this as a bug yesterday as incident #659876. Here's the response:

The problem you described (not being able to use the Python "None" value to set field values to Null) is a known problem, and I have added this incident to the list of affected users in the bug report (#NIM010734). The None object is part of the win32com.client module but not the arcgisscripting module. Unfortunately, there is no work-around at this time other than to initialize the gp object with win32com.client.

chad 
  ---------------
chad 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Aug 08, 2008 
Message Thanks guys. Hopefully this will make a difference.

The response that you (Chad) got is exactly what I got a few months ago. It shows that whoever wrote that canned response does not actually understand the problem, or at least how to articulate it properly.

The "None object" is not "part of the win32com.client module". None is a fundamental type built into Python. When you pass None as a method argument through win32com.client, it converts it into some kind of COM automation type, such as VT_NULL, which the esriGeoprocessing.GPDispatch COM automation object interprets as "set this field to NULL".

Most likely, the arcgisscripting API is calling the same code. arcgisscripting is a Python extension module (arcgisscripting.dll on Windows), almost definitely written in C++ or C. It loads some other ArcGIS DLLS, including appinitializerlib.dll, afcore.dll, and geoprocessinglib.dll. These are probably the same things that are wrapped by the esriGeoprocessing.GPDispatch object, but I haven't checked. The esriGeoprocessing.GPDispatch object knows how to translate a VT_NULL (or whatever) to the appropriate argument to pass to those underlying libraries. The arcgisscripting module does not know how to do this.

Python has specific C functions that extension modules can call to examine and deal with function arguments. Most likely, ESRI neglected to implement a handler for the caller passing in Py_None for the parameter to this function.

If they would give me access to their source code and debug symbols, I would be VERY HAPPY to fix this problem for them because it is SO ANNOYING. But that can't happen of course (I have proposed it before for other bugs).

Who knows what you're supposed to do if your code needs to run on UNIX, where win32com is not available...

Jason
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Philippe Le Grand 
Date Aug 08, 2008 
Message Jason,

+1 on wishing they would just open the code for the arcgisscripting module. I would also be more than happy to track down issues and provide a fix along with any bug report. It would also enable us to produce versions of arcgisscripting for more recent releases of Python than what ESRI is willing to support; 9.3 shipped with 2.5, even though 2.6 has been out a while...

Maybe a dual licensing scheme would work, where they support only the official version.

Philippe 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Aug 08, 2008 
Message Philippe,

I encountered the type of problem you mentioned with Arc 9.2, which supported only Python 2.4. I wanted to use the datetime strptime function that was introduced in Python 2.5, but still remain compatible with Arc 9.2. In my experience, Arc 9.2 works fairly well with Python 2.5 if you create the geoprocessor through win32com (although there are some subtle differences in parameter types). But when you do upgrade to Python 2.5, as you know, you break any scripts that import arcgisscripting, including ESRI's own script-based geoprocessing tools that shipped with 9.2.

A dual licensing scheme such as you mention would be wonderful. My experience with ESRI is that they would never agree to it. We'd have to find someone very high up in the company to listen to this kind of suggestion.

Jason 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Kim Ollivier 
Date Aug 31, 2008 
Message I notice that you can use multiple gp objects in the same script when I have been demonstrating the new List objects. Would that be a workaround for some people? All that is required is to define a new gpw object when using update and insert cursors. Running the two cursors worked for the example.

 
 
import arcgisscripting
gp = arcgisscripting.create(9.3)
if not gp.Exists('c:\\temp\\TestGDB.mdb') :
    gp.CreatePersonalGDB_management('c:\\temp', 'TestGDB.mdb')
# 'c:\\temp\\TestGDB.mdb'
if not gp.Exists('c:\\temp\\TestGDB.mdb\\TestTable') :
    gp.CreateTable_management('c:\\temp\\TestGDB.mdb', 'TestTable')
# 'c:\\temp\\TestGDB.mdb\\TestTable'
gp.AddField_management('c:\\temp\\TestGDB.mdb\\TestTable', 'MyField', 'LONG', '#', '#', '#', '#', 'NULLABLE')
# 'c:\\temp\\TestGDB.mdb\\TestTable'
cur = gp.InsertCursor('c:\\temp\\TestGDB.mdb\\TestTable')
row = cur.NewRow()
row.SetValue('MyField', 12345)
cur.InsertRow(row)
del cur


import win32com.client
gpw = win32com.client.Dispatch('esriGeoprocessing.GPDispatch')
cur = gpw.UpdateCursor('c:\\temp\\TestGDB.mdb\\TestTable')
row = cur.Next()
print row.GetValue('MyField')
try :
    row.SetValue('MyField', None)
except Exception,errmsg:
    print errmsg
##Traceback (most recent call last):
##  File "<interactive input>", line 1, in ?
##ValueError: invalid input value
try :
    row.MyField = None
except Exception,errmsg:
    print errmsg    
##Traceback (most recent call last):
##  File "<interactive input>", line 1, in ?
##ValueError: invalid input value
try :   
    print row.IsNull('MyField')
except Exception,errmsg:
    print errmsg
# 0
try :
    row.SetNull('MyField')
except Exception,errmsg:
    print errmsg
 
  Kim Ollivier
kimo@ollivier.co.nz
www.ollivier.co.nz
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Sep 02, 2008 
Message Kim,

Thanks for the idea. This is an interesting approach, and might work for many situations. There are three possible issues I can think of.

1. In 9.0, I had bad results allocating multiple instances of the geoprocessor, so I have avoided it since then. Assuming this is now fixed, this approach would seem to be feasible.

2. In my scenario, I was writing a function that received a list of records that should be updated. My function wanted to open an update cursor and set values for all of those records. My function would become more complicated using this approach. It could first open a cursor using arcgisscripting to update the rows that do not have NULLs, then open one using win32com to update the rows with NULLs. Or it could open a win32com cursor on the fly whenever it encountered a NULL. This might cause a database locking problem, depending on the underlying database.

3. While this approach does compartmentalize the use of win32com to this one tiny scenario (updating/inserting using NULL), it still retains the dependency on win32com. If you recall, one of the original objections to win32com was that it is platform dependent. By retaining the win32com dependency, this approach limits itself to the Windows OS.

Assuming #1 truly has been fixed, #2 and #3 are not that important to most people. #2 can be dealt with by not trying to use both geoprocessors when you need to update/insert rows. Instead, just always use win32com whenever you need to insert/update, and use arcgisscripting for all other geoprocessing operations. As it stands, #3 is not solved, but this is unlikely to affect more than a small fraction of users.

In case ESRI is reading this: I would still not consider this a reliable workaround unless ESRI blessed this idea of instantating arcgisscripting and win32com geoprocessors at the same time. We still need to keep the pressure on ESRI to fix the root problem.

Jason
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Sep 12, 2008 
Message Regarding the instantiation of two geoprocessors, one with arcgisscripting.create and the other with win32com, I did notice the following notes in a 9.3 help article titled "Running a script in process". I am not sure what the implications are for the dual-instantiation idea.

*** Begin documentation ***

NOTE: Only scripts that create the geoprocessor object using arcgisscripting.create(), as shown below, can be run in process.

import arcgisscripting
gp = arcgisscripting.create() # or arcgisscripting.create(9.3)

NOTE: Scripts that use dispatch, as shown below, will not be run in process, even if the Run Python script in process is checked.

# Import standard library modules
import win32com.client, sys, os

# Create the Geoprocessor object
gp = win32com.client.Dispatch("esriGeoprocessing.GpDispatch.1")

*** End documentation ***

The second "note" is particularly weird. It is seems highly unlikely that ArcGIS scans the script before it runs it, trying to determine if win32com will be used to create the geoprocessor, and then decides to create the Python interpreter in-proc or out-of-proc. Perhaps this note refers to some COM thing, saying that the geoprocessor COM object will be created out-of-proc and this will negate the benefit of running the Python script itself in-proc.

If the only drawback is performance, then this issue may not be that big of a deal.

Jason 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Christopher J 
Date May 26, 2009 
Message I am having a similar problem.

I am trying to create a new feature then skip the values that should be set to null. I am not trying to use an update.

However, when I do this, the geoprocessor seems to take the value from the previous row.

My code is below. I read a file with field names, take a value from another table and eventually add the completed feature to the table. However, as mentioned, the fields with nulls are becoming filled with values from the previous row.

I was getting the impression from this thread that inserting a new feature would solve the problem, however, I cannot even get that to work. What am I doing wrong? 
 
#for each row in the field file, set the value of the fields in the new feature
		myFile = open(fieldFilePathArg, 'r')
		for curRow in myFile:
		
			crp = curRow.split("~")
			newFieldName = crp[0]
			oldFieldName = crp[9]
							
			curOldValue = curOldRow.GetValue(oldFieldName)
			try:			
				newFeatureRow.SetValue(newFieldName, curOldValue)
			except:				
				newFeatureRow.SetValue(newFieldName, dummyNullValueArg)
						
		myFile.close();	
				
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Christopher J 
Date May 26, 2009 
Message deleted accidental post 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date May 26, 2009 
Message Hi Christopher,

I am not 100% clear from your example exactly what is happening, but what you describe sounds like a known "behavior" of the ArcGIS insert cursors. After inserting a row, subsequent inserts will use values of the previous row's fields, unless you explicitly set those values. I personally think this design is crazy, but that is how it has worked for a long time. It is particularly aggrevating in light of the main bug discussed in this thread (the inability to set a field to NULL using the geoprocessor obtained from arcgisscripting).

If you post a more complete example of your code, I might have a better idea what is happening. But, ultimately, you will probably find that there is NO way to set a field to NULL unless you create the geoprocessor using win32com.client.

In the example you do provide, assuming you instantiated teh gp with arcgisscripting, if both curOldValue and dummyNullValueArg are a Python None, your program will fail.

Best,
Jason 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Kris Stitt 
Date Jun 04, 2009 
Message Hi all,

We have also run into the null value bug with arcgisprocessing. When I attended one of the Geoprocessing sessions at the Developers Conference and asked if they had any plans to fix the problem. He (Sorry, I cannot recall his name.) said that they were hoping to have it fixed with an upcoming service pack. (It did not make 9.3.1)

Anyway not sure if that helps much, but I figured I would pass it along just in case..

Thanks,
-Kris 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Jun 04, 2009 
Message Kris,

Thanks for raising it there and for reporting on your experience. Great to know they are finally going to fix it.

Last week, I also heard that this would be fixed. This was from Ralf Gottschalk, Development Technical Lead - SDK Unit; he said it would be 9.3.1 SP1. See http://forums.esri.com/Thread.asp?c=208&f=2430&t=274896 for this conversation (although much of my discussion with Ralf has been in private email). I will update the forum threads related to the other bugs I mentioned there.

Jason
 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Chris Snyder 
Date Oct 07, 2009 
Message I just reviewed the bug fixes planned for v9.3.1 SP1 which is due out in Dec 2009: http://downloads2.esri.com/support/documentation/ao_/9.3.1_SP1_Announcement9_1.pdf

It does not currently include NIM010734 even though an ESRI employee commented on May 29th, 2009 that "NIM010734 – This issue should be fixed for the 9.3.1 sp 1 time frame" (see http://forums.esri.com/Thread.asp?c=208&f=2430&t=274896#876903).

This two year old bug (inability to properly update/insert NULL values using a gp update or insert cursor object) has been around for quite a while (since the release of v9.2) and has frustrated many a scripter - me included. It is a major flaw in commonly used core geoprocessing functionality that is apparently being continuously ignored.

PLEASE ESRI - FIX THIS BUG!!! 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Oct 07, 2009 
Message That is indeed disappointing, if true. Hopefully they just forgot to put it in the announcement. I will contact Ralf again and ask him to clarify this.

Jason 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Chris Snyder 
Date Oct 07, 2009 
Message I resubmitted this bug to ESRI today in a hope that more complaints would lead to a fix in v9.3.1 SP1. An ESRI rep did promptly reply, but unfortunately indicated that indeed there is no current fix for the update/insert cursor bug in v9.3.1 SP1, but that he was "very confident" that it was fixed in v9.4 beta.

YARGGHH! 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Oct 07, 2009 
Message I have not heard back from Ralf yet. I have a copy of 9.4 build 2193 (the early adopters build) but have not had time to install it yet (tried once but didn't have enough disk space). I have a million things happening right now but if I can find time to install it, I'll try to repro it. If anyone else out there has a build of 9.4, please try it.

Jason 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Oct 13, 2009 
Message I received email today from Ralf Gottschalk at ESRI saying:

"This issue is fixed in 9.4 via the new “setNull” method. When I spoke to the Geoprocessing team they said that they would be able to port the fix over to 9.3.1 sp 1. Sometimes it takes a while for the release announcement to update. I will check with the GP team to verify that are still planning on getting this into SP 1."

If I hear anything further, I will update this thread.

Jason 
   
Report Inappropriate Content • Top • Print • Reply    
Subject Re: How to set field to NULL with update cursor created with Arc 9.2 arcgisscripting Python module? 
Author Jason Roberts 
Date Oct 15, 2009 
Message Here is the latest from Ralf:

"I followed up with the Geoprocessing team regarding this issue. They are still planning on getting this into SP 1. Unfortunately, it may be a little while before it shows up on the PDF, but we should be good to go here. Thanks for all your feedback on the issue and pointing us to more people who were experiencing this problem."

Looks like good news...

Jason