Add methods to perform batch gets/deletes/updates of blobs

[mziccard]

To simplify some common use cases of batch requests we could add to Blob class some static methods to do batch gets/deletes/updates:

Blob.get(storage, blobInfo1, blobInfo2, ...)
Blob.update(storage, blobInfo1, blobInfo2, ...)
Blob.delete(storage, blob1Info, blob2Info, ...)

We could also consider putting these methods in a dedicated Blobs class, what do you think?

[aozarov]

+1 for static Blob and Bucket operations.

Another option would be to move (or also add) these bulk operations to storage (e.g. storage.get(blobInfo...) which returns Iterable<BlobInfo>).

[mziccard]

I like having them in Blob. Event though they take BlobInfo parameters they can return Iterable<Blob> (or Iterable<Boolean> for delete).

What about adding them both to storage with the signature you propose and to Blob returning Iterable<Blob> instead? (I know you don't like multiple ways of doing the same thing but forgive me:)

[aozarov]

SGTM.

[mziccard]

I was wondering whether we should return Iterable<BlobInfo> where a null item means failed request (or not found) or Iterable<BatchResponse.Result<BlobInfo>> that contains more information on the error. Thoughts?

If we decide to go with Iterable<BatchResponse.Result<BlobInfo>> we need to figure out what to do with the methods in functional Blob as BatchResponse.Result accepts only Serializable types.

[mziccard]

Let me also add to this that for storage.delete(blobInfo...) we need to return an Iterable<Boolean> where null makes a lot less sense.

Also, naming the method storage.update(blobInfo...) causes ambiguity with storage.update(blobInfo, option...), I suggest we call these methods getAll, updateAll and deleteAll.

[aozarov]

I don't have a strong opinion about it.

Because this is mostly a syntactic sugar for a simplified use-case of Batch with a single operation and without blob options I would slightly prefer to go with a simple response Iterable<BlobInfo> with null
when an item failed. Because this request does not include options I think the only reason for such
a failure would be HTTP_NOT_FOUND (triggered by NoSuchKey or AccessDenied) and if so that
would be consistent with the single-blob get.

@BrandonY can you confirm that?

Let me also add to this that for storage.delete(blobInfo...) we need to return an Iterable where null makes a lot less sense.

Again, we can simplify here and make this method always return True (deleted) or False (not deleted for what ever reason).

Also, naming the method storage.update(blobInfo...) causes ambiguity with storage.update(blobInfo, option...)

The way we dealt with that before is by using this signature: storage.delete(blobInfo, blobInfo...)

[aozarov]

Another thing to consider is if we should return an Iterable or Iterator.
The benefit of the latter would be that it could allow use to chunk the input into multiple lazy batch requests.

[mziccard]

I prefer keeping this simple as well!

The way we dealt with that before is by using this signature: storage.delete(blobInfo, blobInfo...)

I thought adding All might have been a more meaningful name but if you used this elsewhere we better stick to the delete, update, get naming.

Is there any reason you to prefer Iterable over List for this specific api?

[aozarov]

Is there any reason you to prefer Iterable over List for this specific api?

Theoretically we could even make Iterable return a lazy Iterator.

[mziccard]

Yes but for batch requests there's no support for pagination, right? Are we going to use lazy iterators for this?
I'm asking because using List might be friendlier.

[aozarov]

Yes but for batch requests there's no support for pagination, right? Are we going to use lazy iterators for this?

Right, that is what I was suggesting. lets say a user requests 100 blobs and we define max batch size is 50 than we could get it in 2 sequential batches.

I'm asking because using List might be friendlier.

Yes, I agree. for a return type this would typically be better.
I am fine either way. Your call.

[mziccard]

Just some more thoughts.

Right, that is what I was suggesting. lets say a user requests 100 blobs and we define max batch size is 50 than we could get it in 2 sequential batches.

If we ever want to, I think we could do this using a List as well. We can support get(i) and listIterator by using the input array of BlobInfo as a reference while all operation that modify the list throw UnsupportedOperationException.

[aozarov]

If we ever want to, I think we could do this using a List as well. We can support get(i) and listIterator by using the input array of BlobInfo as a reference while all operation that modify the list throw UnsupportedOperationException.

Yes, we could document the return type as immutable list where all the operation that tries to
modify it throws UnsupportedOperationException.
I think the problem is that there are more side-effects such as list.size() which is typically a quick
operation can block for a long time.

Regardless, I actually think that it is more important to support a large input (which may exceed the
batch limitation) than dealing with a large result set. Therefore, a list could be just fine and we could
potentially chunk the request to multiple batches and populate the list one by one before we return it.