Repository navigation
Compressed JsonProperty within a repeated StructuredProperty raises AttributError #15856
Description
While converting a complex AppEngine code base from python 2.7 to python 3.x, I ran into an unexpected bug in the new google.cloud.ndb that does not exist in the legacy google.appengine.ext.ndb.
Environment details
python-ndb 1.11.1
Python 2.7.18
Discovered and tested on MacOS 12.6
Steps to reproduce
See the toy example below. But to summarize: A repeated StructuredProperty that contains a compressed JsonProperty field. The bug probably apples to other property types that inherit from BlobProperty.
Code example
from google.cloud import ndb
class SomeStructuredProperty(ndb.Model):
compressed_json = ndb.JsonProperty(compressed=True)
class ParentModel(ndb.Model):
repeated_sp = ndb.StructuredProperty(SomeStructuredProperty, repeated=True)
m = ParentModel(repeated_sp=[SomeStructuredProperty(compressed_json={})])
m.put() # Raises AttributeError with google.cloud.ndbStack trace
Traceback (most recent call last):
File "/.../test.py", line 247, in test_problem
m.put() # Raises AttrbuteError with google.cloud.ndb
File "/.../lib/google/cloud/ndb/_options.py", line 102, in wrapper
return wrapped(*pass_args, **kwargs)
File "/.../lib/google/cloud/ndb/utils.py", line 121, in wrapper
return wrapped(*args, **new_kwargs)
File "/.../lib/google/cloud/ndb/utils.py", line 153, in positional_wrapper
return wrapped(*args, **kwds)
File "/.../lib/google/cloud/ndb/model.py", line 5383, in _put
return self._put_async(_options=kwargs["_options"]).result()
File "/.../lib/google/cloud/ndb/tasklets.py", line 214, in result
self.check_success()
File "/.../lib/google/cloud/ndb/tasklets.py", line 161, in check_success
raise self._exception
AttributeError: 'list' object has no attribute 'startswith'
Possible fix?
The exception actually happens in BlobProperty._to_datastore, but under python 2.7 the stack trace doesn't show the original stack trace. If I understand that method properly, it checks self._repeated, but it neglects the repeated input argument when determining how to mutate or read the data dict that it is constructing for datastore. The exception is raised because the field itself is not self._repeated, but it is part of a repeated StructuredProperty.
Activity
fix: repeated structured property containing blob property with legac…
ventice11o commentedon Mar 1, 2024
Fixed in google-cloud-ndb==2.3.0
aetherknight commentedon Mar 26, 2024
Based on my initial testing, this does seem to be fixed in 2.3.0 and 2.3.1.
(Although we can't upgrade until this version of NDB can properly work with our test suite. It can't be used with the datastore emulator, and the firestore emulator lacks some fairly critical features of the datastore emulator at this time.)
chalmerlowe commentedon Mar 2, 2026
This issue was moved from the following repo as part of the monorepo migration: googleapis/python-ndb
daniel-sanche commentedon May 28, 2026
This issue is being bulk closed due to inactivity. Please re-open if you think this still requires attention, and we'll take another look. Thanks!