At Google Cloud Next '18, we announced that all Cloud Datastore users will be automatically upgraded to Cloud Firestore in Datastore mode once Cloud Firestore becomes generally available.
The upgrade to the Cloud Firestore storage engine brings the following benefits:
- Entity groups are no longer restricted to 1 transaction/second
- Transactions are no longer limited to 25 entity-groups
- All queries, not just ancestor queries, are strongly consistent
With the removal of the 1 transaction/second restriction, however, the entity group is no longer a unit of consistency.
When you request an entity group version, Cloud Firestore returns a monotonically increasing number. This number is guaranteed to increase on new writes to the entity group, but it also increases independently of user updates to the entity group. In Cloud Datastore, the entity group version changed only on writes to the entity group.
This new behavior might affect functionality that depends on the current behavior. For example, this would affect application logic like caching that depends on the entity group version updating only on new writes.
As a workaround, you can maintain the current behavior by keeping track of the entity group version manually. Whenever a change is made to entities in a group whose version needs to be tracked, increment a property of a specially designated entity.
Java
For example, take this code to modify an entity:
Entity foo = …;
foo.setProperty("bar", "new_value");
datastore.put(foo);
Previously, you would use com.google.appengine.api.datastore.Entities to access the entity group version:
Key egKey = Entities.createEntityGroupKey(foo.getKey());
long version = Entities.getVersionProperty(datastore.get(egKey));
To maintain the behavior where the entity group version increases only with user updates, make an extra call to keep track of the entity group version and wrap the code in a transaction:
Entity foo = …;
foo.setProperty("bar", "new_value");
datastore.put(transaction, foo);
Entity groupVersion = loadOrCreateGroupVersion(foo);
groupVersion.setProperty("version",
(long) groupVersion.getProperty("version") + 1);
datastore.put(transaction, groupVersion);
This code uses the following helper method:
public static Entity loadOrCreateGroupVersion(Entity entity) {
Key rootKey = entity.getKey();
while (rootKey.getParent() != null) rootKey = rootKey.getParent();
Key key = KeyFactory.createKey(rootKey,
"EntityGroupVersion",
"EntityGroupVersionKey");
DatastoreService datastore = DatastoreServiceFactory
.getDatastoreService();
Entity version;
try {
version = datastore.get(key);
} catch (EntityNotFoundException e) {
version = new Entity(key);
version.setProperty("version", 0);
datastore.put(version);
}
return version;
}
You can then retrieve the entity group version as follows:
Entity groupVersion = loadOrCreateGroupVersion(foo);
long version = groupVersion.getProperty("version");
Python (db)
For example, take this code to modify an entity:
def change_foo(key):
foo = db.get(key)
foo.bar = 'new_value'
foo.put()
Previously, you would use db.metadata.get_entity_group_version() to access the entity group version:
db.metadata.get_entity_group_version(foo)
To maintain the behavior where the entity group version increases only with user updates, make an extra call to keep track of the entity group version and wrap the code in a transaction:
@db.transactional()
def change_foo(key):
foo = db.get(key)
foo.bar = 'new_value'
foo.put()
EntityGroupVersion.increase_version(foo)
This code uses the following helper method:
class EntityGroupVersion(db.Model):
version = db.IntegerProperty()
@staticmethod
def increase_version(entity):
entity_group_version = EntityGroupVersion.__load(entity)
entity_group_version.version += 1
entity_group_version.put()
@staticmethod
def get_version(entity):
return EntityGroupVersion.__load(entity).version
@staticmethod
def __load(entity):
root_key = entity.key()
while root_key.parent():
root_key = root_key.parent()
key = db.Key.from_path('EntityGroupVersion',
'EntityGroupVersionKey',
parent=root_key)
entity_group_version = db.get(key)
if entity_group_version is None:
entity_group_version = EntityGroupVersion(
parent=root_key,
key_name='EntityGroupVersionKey',
version=1)
entity_group_version.put()
return entity_group_version
You can then retrieve the entity group version as follows:
version = EntityGroupVersion.get_version(foo)
Python (ndb)
For example, take this code to modify an entity:
def change_foo(key):
foo = key.get()
foo.bar = 'new_value'
foo.put()
Previously, you would use ndb.metadata.get_entity_group_version() to access the entity group version:
ndb.metadata.get_entity_group_version(foo)
To maintain the behavior where the entity group version increases only with user updates, make an extra call to keep track of the entity group version and wrap the code in a transaction:
@ndb.transactional()
def change_foo(key):
foo = key.get()
foo.bar = 'new_value'
foo.put()
EntityGroupVersion.increase_version(foo)
This code uses the following helper method:
class EntityGroupVersion(ndb.Model):
version = ndb.IntegerProperty()
@staticmethod
def increase_version(entity):
entity_group_version = EntityGroupVersion.__load(entity)
entity_group_version.version += 1
entity_group_version.put()
@staticmethod
def get_version(entity):
return EntityGroupVersion.__load(entity).version
@staticmethod
def __load(entity):
entity_group_version = EntityGroupVersion.get_by_id(
'EntityGroupVersionKey', parent=entity.key().root())
if entity_group_version is None:
entity_group_version = EntityGroupVersion(
parent=entity.key().root(),
id='EntityGroupVersionKey',
version=1)
entity_group_version.put()
return entity_group_version
You can then retrieve the entity group version as follows:
version = EntityGroupVersion.get_version(foo)