Search in sources :

Example 11 with WriteObjectRequest

use of com.google.storage.v2.WriteObjectRequest in project hadoop-connectors by GoogleCloudDataproc.

the class GoogleCloudStorageGrpcWriteChannelTest method writeSendsSingleInsertObjectRequestWithChecksums.

@Test
public void writeSendsSingleInsertObjectRequestWithChecksums() throws Exception {
    AsyncWriteChannelOptions options = AsyncWriteChannelOptions.builder().setGrpcChecksumsEnabled(true).build();
    ObjectWriteConditions writeConditions = ObjectWriteConditions.NONE;
    GoogleCloudStorageGrpcWriteChannel writeChannel = newWriteChannel(options, writeConditions, /* requesterPaysProject= */
    null);
    fakeService.setQueryWriteStatusResponses(ImmutableList.of(QueryWriteStatusResponse.newBuilder().setPersistedSize(0).build(), QueryWriteStatusResponse.newBuilder().setPersistedSize(0).build()).iterator());
    ByteString data = ByteString.copyFromUtf8("test data");
    writeChannel.initialize();
    writeChannel.write(data.asReadOnlyByteBuffer());
    writeChannel.close();
    WriteObjectRequest expectedInsertRequest = WriteObjectRequest.newBuilder().setUploadId(UPLOAD_ID).setChecksummedData(ChecksummedData.newBuilder().setContent(data).setCrc32C((uInt32Value(863614154)))).setObjectChecksums(ObjectChecksums.newBuilder().setCrc32C((uInt32Value(863614154)))).setFinishWrite(true).build();
    verify(fakeService, times(1)).startResumableWrite(eq(START_REQUEST), any());
    verify(fakeService.insertRequestObserver, times(1)).onNext(expectedInsertRequest);
    verify(fakeService.insertRequestObserver, atLeast(1)).onCompleted();
}
Also used : AsyncWriteChannelOptions(com.google.cloud.hadoop.util.AsyncWriteChannelOptions) ByteString(com.google.protobuf.ByteString) WriteObjectRequest(com.google.storage.v2.WriteObjectRequest) Test(org.junit.Test)

Example 12 with WriteObjectRequest

use of com.google.storage.v2.WriteObjectRequest in project hadoop-connectors by GoogleCloudDataproc.

the class GoogleCloudStorageGrpcWriteChannelTest method writeSendsSingleInsertObjectRequestWithoutChecksums.

@Test
public void writeSendsSingleInsertObjectRequestWithoutChecksums() throws Exception {
    AsyncWriteChannelOptions options = AsyncWriteChannelOptions.builder().setGrpcChecksumsEnabled(false).build();
    ObjectWriteConditions writeConditions = ObjectWriteConditions.NONE;
    GoogleCloudStorageGrpcWriteChannel writeChannel = newWriteChannel(options, writeConditions, /* requesterPaysProject= */
    null);
    ByteString data = ByteString.copyFromUtf8("test data");
    writeChannel.initialize();
    writeChannel.write(data.asReadOnlyByteBuffer());
    writeChannel.close();
    WriteObjectRequest expectedInsertRequest = WriteObjectRequest.newBuilder().setUploadId(UPLOAD_ID).setChecksummedData(ChecksummedData.newBuilder().setContent(data)).setFinishWrite(true).build();
    verify(fakeService, times(1)).startResumableWrite(eq(START_REQUEST), any());
    verify(fakeService.insertRequestObserver, times(1)).onNext(expectedInsertRequest);
    verify(fakeService.insertRequestObserver, atLeast(1)).onCompleted();
}
Also used : AsyncWriteChannelOptions(com.google.cloud.hadoop.util.AsyncWriteChannelOptions) ByteString(com.google.protobuf.ByteString) WriteObjectRequest(com.google.storage.v2.WriteObjectRequest) Test(org.junit.Test)

Aggregations

WriteObjectRequest (com.google.storage.v2.WriteObjectRequest)12 Test (org.junit.Test)10 ByteString (com.google.protobuf.ByteString)7 WriteObjectResponse (com.google.storage.v2.WriteObjectResponse)6 AsyncWriteChannelOptions (com.google.cloud.hadoop.util.AsyncWriteChannelOptions)4 StreamObserver (io.grpc.stub.StreamObserver)4 ApiStreamObserver (com.google.api.gax.rpc.ApiStreamObserver)1 ReadObjectResponse (com.google.storage.v2.ReadObjectResponse)1 StorageClient (com.google.storage.v2.StorageClient)1 StorageGrpc (com.google.storage.v2.StorageGrpc)1 CancellableContext (io.grpc.Context.CancellableContext)1 StatusException (io.grpc.StatusException)1 NoopClientCall (io.grpc.internal.NoopClientCall)1 Iterator (java.util.Iterator)1 CountDownLatch (java.util.concurrent.CountDownLatch)1 TimeoutException (java.util.concurrent.TimeoutException)1