Search in sources :

Example 1 with CommitResponse

use of com.google.firestore.v1.CommitResponse in project google-cloud-java by GoogleCloudPlatform.

the class SessionImplTest method writeAtLeastOnce.

@Test
public void writeAtLeastOnce() throws ParseException {
    String timestampString = "2015-10-01T10:54:20.021Z";
    ArgumentCaptor<CommitRequest> commit = ArgumentCaptor.forClass(CommitRequest.class);
    CommitResponse response = CommitResponse.newBuilder().setCommitTimestamp(Timestamps.parse(timestampString)).build();
    Mockito.when(rpc.commit(commit.capture(), Mockito.eq(options))).thenReturn(response);
    Timestamp timestamp = session.writeAtLeastOnce(Arrays.asList(Mutation.newInsertBuilder("T").set("C").to("x").build()));
    assertThat(timestamp.getSeconds()).isEqualTo(utcTimeSeconds(2015, Calendar.OCTOBER, 1, 10, 54, 20));
    assertThat(timestamp.getNanos()).isEqualTo(TimeUnit.MILLISECONDS.toNanos(21));
    CommitRequest request = commit.getValue();
    assertThat(request.getSingleUseTransaction()).isNotNull();
    assertThat(request.getSingleUseTransaction().getReadWrite()).isNotNull();
    com.google.spanner.v1.Mutation mutation = com.google.spanner.v1.Mutation.newBuilder().setInsert(Write.newBuilder().setTable("T").addColumns("C").addValues(ListValue.newBuilder().addValues(com.google.protobuf.Value.newBuilder().setStringValue("x")))).build();
    assertThat(request.getMutationsList()).containsExactly(mutation);
}
Also used : CommitRequest(com.google.spanner.v1.CommitRequest) CommitResponse(com.google.spanner.v1.CommitResponse) ByteString(com.google.protobuf.ByteString) Timestamp(com.google.cloud.Timestamp) Test(org.junit.Test)

Example 2 with CommitResponse

use of com.google.firestore.v1.CommitResponse in project beam by apache.

the class DatastoreV1Test method testDatatoreWriterFnRetriesErrors.

/**
 * Tests {@link DatastoreWriterFn} with a failed request which is retried.
 */
@Test
public void testDatatoreWriterFnRetriesErrors() throws Exception {
    List<Mutation> mutations = new ArrayList<>();
    int numRpcs = 2;
    for (int i = 0; i < DatastoreV1.DATASTORE_BATCH_UPDATE_ENTITIES_START * numRpcs; ++i) {
        mutations.add(makeUpsert(Entity.newBuilder().setKey(makeKey("key" + i, i + 1)).build()).build());
    }
    CommitResponse successfulCommit = CommitResponse.getDefaultInstance();
    when(mockDatastore.commit(any(CommitRequest.class))).thenReturn(successfulCommit).thenThrow(new DatastoreException("commit", Code.DEADLINE_EXCEEDED, "", null)).thenReturn(successfulCommit);
    DatastoreWriterFn datastoreWriter = new DatastoreWriterFn(StaticValueProvider.of(PROJECT_ID), null, mockDatastoreFactory, new FakeWriteBatcher());
    DoFnTester<Mutation, Void> doFnTester = DoFnTester.of(datastoreWriter);
    doFnTester.setCloningBehavior(CloningBehavior.DO_NOT_CLONE);
    doFnTester.processBundle(mutations);
    verifyMetricWasSet("BatchDatastoreWrite", "ok", "", 2);
    verifyMetricWasSet("BatchDatastoreWrite", "unknown", "", 1);
}
Also used : CommitRequest(com.google.datastore.v1.CommitRequest) ArrayList(java.util.ArrayList) CommitResponse(com.google.datastore.v1.CommitResponse) Mutation(com.google.datastore.v1.Mutation) DatastoreException(com.google.datastore.v1.client.DatastoreException) DatastoreWriterFn(org.apache.beam.sdk.io.gcp.datastore.DatastoreV1.DatastoreWriterFn) Test(org.junit.Test)

Example 3 with CommitResponse

use of com.google.firestore.v1.CommitResponse in project spring-cloud-gcp by spring-cloud.

the class ReactiveFirestoreTransactionManagerTest method getFirestoreTemplate.

private FirestoreTemplate getFirestoreTemplate() {
    doAnswer(invocation -> {
        StreamObserver<BeginTransactionResponse> streamObserver = invocation.getArgument(1);
        streamObserver.onNext(BeginTransactionResponse.newBuilder().setTransaction(ByteString.copyFromUtf8("transaction1")).build());
        streamObserver.onCompleted();
        return null;
    }).when(this.firestoreStub).beginTransaction(any(), any());
    doAnswer(invocation -> {
        CommitRequest commitRequest = invocation.getArgument(0);
        StreamObserver<CommitResponse> streamObserver = invocation.getArgument(1);
        assertThat(commitRequest.getTransaction()).isEqualTo(ByteString.copyFromUtf8("transaction1"));
        streamObserver.onNext(CommitResponse.newBuilder().build());
        streamObserver.onCompleted();
        return null;
    }).when(this.firestoreStub).commit(any(), any());
    doAnswer(invocation -> {
        RollbackRequest rollbackRequest = invocation.getArgument(0);
        StreamObserver<Empty> streamObserver = invocation.getArgument(1);
        assertThat(rollbackRequest.getTransaction()).isEqualTo(ByteString.copyFromUtf8("transaction1"));
        streamObserver.onCompleted();
        return null;
    }).when(this.firestoreStub).rollback(any(), any());
    doAnswer(invocation -> {
        GetDocumentRequest request = invocation.getArgument(0);
        StreamObserver<Document> streamObserver = invocation.getArgument(1);
        assertThat(request.getTransaction()).isEqualTo(ByteString.copyFromUtf8("transaction1"));
        String name = request.getName();
        streamObserver.onNext(FirestoreTemplateTests.buildDocument(name.substring(name.length() - 2), 100L));
        streamObserver.onCompleted();
        return null;
    }).when(this.firestoreStub).getDocument(any(), any());
    FirestoreTemplate template = new FirestoreTemplate(this.firestoreStub, this.parent, new FirestoreDefaultClassMapper(), new FirestoreMappingContext());
    StepVerifier.setDefaultTimeout(Duration.ofSeconds(5));
    return template;
}
Also used : CommitRequest(com.google.firestore.v1.CommitRequest) CommitResponse(com.google.firestore.v1.CommitResponse) ByteString(com.google.protobuf.ByteString) RollbackRequest(com.google.firestore.v1.RollbackRequest) GetDocumentRequest(com.google.firestore.v1.GetDocumentRequest) Document(com.google.firestore.v1.Document) BeginTransactionResponse(com.google.firestore.v1.BeginTransactionResponse) FirestoreTemplate(org.springframework.cloud.gcp.data.firestore.FirestoreTemplate) FirestoreMappingContext(org.springframework.cloud.gcp.data.firestore.mapping.FirestoreMappingContext) Empty(com.google.protobuf.Empty) FirestoreDefaultClassMapper(org.springframework.cloud.gcp.data.firestore.mapping.FirestoreDefaultClassMapper)

Example 4 with CommitResponse

use of com.google.firestore.v1.CommitResponse in project spring-cloud-gcp by spring-cloud.

the class ReactiveFirestoreTransactionManagerTest method writeTransaction.

@Test
public void writeTransaction() {
    FirestoreTemplate template = getFirestoreTemplate();
    ReactiveFirestoreTransactionManager txManager = new ReactiveFirestoreTransactionManager(this.firestoreStub, this.parent);
    TransactionalOperator operator = TransactionalOperator.create(txManager);
    doAnswer(invocation -> {
        CommitRequest commitRequest = invocation.getArgument(0);
        StreamObserver<CommitResponse> streamObserver = invocation.getArgument(1);
        assertThat(commitRequest.getTransaction()).isEqualTo(ByteString.copyFromUtf8("transaction1"));
        assertThat(commitRequest.getWritesList().get(0).getUpdate().getName()).isEqualTo(this.parent + "/testEntities/" + "e2");
        assertThat(commitRequest.getWritesList().get(1).getUpdate().getName()).isEqualTo(this.parent + "/testEntities/" + "e3");
        assertThat(commitRequest.getWritesList().get(2).getDelete()).isEqualTo(this.parent + "/testEntities/" + "e3");
        streamObserver.onNext(CommitResponse.newBuilder().build());
        streamObserver.onCompleted();
        return null;
    }).when(this.firestoreStub).commit(any(), any());
    template.findById(Mono.just("e1"), FirestoreTemplateTests.TestEntity.class).flatMap(testEntity -> template.save(new FirestoreTemplateTests.TestEntity("e2", 100L))).flatMap(testEntity -> template.save(new FirestoreTemplateTests.TestEntity("e3", 100L))).flatMap(testEntity -> template.delete(Mono.just(testEntity))).then().as(operator::transactional).as(StepVerifier::create).verifyComplete();
    verify(this.firestoreStub).beginTransaction(any(), any());
    verify(this.firestoreStub).commit(any(), any());
    GetDocumentRequest request1 = GetDocumentRequest.newBuilder().setName(this.parent + "/testEntities/" + "e1").setTransaction(ByteString.copyFromUtf8("transaction1")).build();
    verify(this.firestoreStub, times(1)).getDocument(eq(request1), any());
}
Also used : TransactionalOperator(org.springframework.transaction.reactive.TransactionalOperator) CommitRequest(com.google.firestore.v1.CommitRequest) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) CommitResponse(com.google.firestore.v1.CommitResponse) StepVerifier(reactor.test.StepVerifier) ArgumentMatchers.eq(org.mockito.ArgumentMatchers.eq) FirestoreTemplate(org.springframework.cloud.gcp.data.firestore.FirestoreTemplate) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) BeginTransactionResponse(com.google.firestore.v1.BeginTransactionResponse) Empty(com.google.protobuf.Empty) StreamObserver(io.grpc.stub.StreamObserver) CommitRequest(com.google.firestore.v1.CommitRequest) Duration(java.time.Duration) Mockito.doAnswer(org.mockito.Mockito.doAnswer) RollbackRequest(com.google.firestore.v1.RollbackRequest) FirestoreDefaultClassMapper(org.springframework.cloud.gcp.data.firestore.mapping.FirestoreDefaultClassMapper) GetDocumentRequest(com.google.firestore.v1.GetDocumentRequest) FirestoreGrpc(com.google.firestore.v1.FirestoreGrpc) FirestoreTemplateTests(org.springframework.cloud.gcp.data.firestore.FirestoreTemplateTests) Document(com.google.firestore.v1.Document) TransactionalOperator(org.springframework.transaction.reactive.TransactionalOperator) Test(org.junit.Test) Mono(reactor.core.publisher.Mono) Mockito.verify(org.mockito.Mockito.verify) ByteString(com.google.protobuf.ByteString) VerificationModeFactory.times(org.mockito.internal.verification.VerificationModeFactory.times) FirestoreDataException(org.springframework.cloud.gcp.data.firestore.FirestoreDataException) FirestoreMappingContext(org.springframework.cloud.gcp.data.firestore.mapping.FirestoreMappingContext) Mockito.mock(org.mockito.Mockito.mock) CommitResponse(com.google.firestore.v1.CommitResponse) FirestoreTemplateTests(org.springframework.cloud.gcp.data.firestore.FirestoreTemplateTests) GetDocumentRequest(com.google.firestore.v1.GetDocumentRequest) FirestoreTemplate(org.springframework.cloud.gcp.data.firestore.FirestoreTemplate) Test(org.junit.Test)

Example 5 with CommitResponse

use of com.google.firestore.v1.CommitResponse in project beam by apache.

the class SpannerAccessor method createAndConnect.

private static SpannerAccessor createAndConnect(SpannerConfig spannerConfig) {
    SpannerOptions.Builder builder = SpannerOptions.newBuilder();
    // Set retryable codes for all API methods
    if (spannerConfig.getRetryableCodes() != null) {
        builder.getSpannerStubSettingsBuilder().applyToAllUnaryMethods(input -> {
            input.setRetryableCodes(spannerConfig.getRetryableCodes());
            return null;
        });
        builder.getSpannerStubSettingsBuilder().executeStreamingSqlSettings().setRetryableCodes(spannerConfig.getRetryableCodes());
    }
    // Set commit retry settings
    UnaryCallSettings.Builder<CommitRequest, CommitResponse> commitSettings = builder.getSpannerStubSettingsBuilder().commitSettings();
    ValueProvider<Duration> commitDeadline = spannerConfig.getCommitDeadline();
    if (spannerConfig.getCommitRetrySettings() != null) {
        commitSettings.setRetrySettings(spannerConfig.getCommitRetrySettings());
    } else if (commitDeadline != null && commitDeadline.get().getMillis() > 0) {
        // Set the GRPC deadline on the Commit API call.
        RetrySettings.Builder commitRetrySettingsBuilder = commitSettings.getRetrySettings().toBuilder();
        commitSettings.setRetrySettings(commitRetrySettingsBuilder.setTotalTimeout(org.threeten.bp.Duration.ofMillis(commitDeadline.get().getMillis())).setMaxRpcTimeout(org.threeten.bp.Duration.ofMillis(commitDeadline.get().getMillis())).setInitialRpcTimeout(org.threeten.bp.Duration.ofMillis(commitDeadline.get().getMillis())).build());
    }
    // Set execute streaming sql retry settings
    ServerStreamingCallSettings.Builder<ExecuteSqlRequest, PartialResultSet> executeStreamingSqlSettings = builder.getSpannerStubSettingsBuilder().executeStreamingSqlSettings();
    if (spannerConfig.getExecuteStreamingSqlRetrySettings() != null) {
        executeStreamingSqlSettings.setRetrySettings(spannerConfig.getExecuteStreamingSqlRetrySettings());
    } else {
        // Setting the timeout for streaming read to 2 hours. This is 1 hour by default
        // after BEAM 2.20.
        RetrySettings.Builder executeSqlStreamingRetrySettings = executeStreamingSqlSettings.getRetrySettings().toBuilder();
        executeStreamingSqlSettings.setRetrySettings(executeSqlStreamingRetrySettings.setInitialRpcTimeout(org.threeten.bp.Duration.ofMinutes(120)).setMaxRpcTimeout(org.threeten.bp.Duration.ofMinutes(120)).setTotalTimeout(org.threeten.bp.Duration.ofMinutes(120)).build());
    }
    ValueProvider<String> projectId = spannerConfig.getProjectId();
    if (projectId != null) {
        builder.setProjectId(projectId.get());
    }
    ServiceFactory<Spanner, SpannerOptions> serviceFactory = spannerConfig.getServiceFactory();
    if (serviceFactory != null) {
        builder.setServiceFactory(serviceFactory);
    }
    ValueProvider<String> host = spannerConfig.getHost();
    if (host != null) {
        builder.setHost(host.get());
    }
    ValueProvider<String> emulatorHost = spannerConfig.getEmulatorHost();
    if (emulatorHost != null) {
        builder.setEmulatorHost(emulatorHost.get());
        if (spannerConfig.getIsLocalChannelProvider() != null && spannerConfig.getIsLocalChannelProvider().get()) {
            builder.setChannelProvider(LocalChannelProvider.create(emulatorHost.get()));
        }
        builder.setCredentials(NoCredentials.getInstance());
    }
    String userAgentString = USER_AGENT_PREFIX + "/" + ReleaseInfo.getReleaseInfo().getVersion();
    builder.setHeaderProvider(FixedHeaderProvider.create("user-agent", userAgentString));
    SpannerOptions options = builder.build();
    Spanner spanner = options.getService();
    String instanceId = spannerConfig.getInstanceId().get();
    String databaseId = spannerConfig.getDatabaseId().get();
    DatabaseClient databaseClient = spanner.getDatabaseClient(DatabaseId.of(options.getProjectId(), instanceId, databaseId));
    BatchClient batchClient = spanner.getBatchClient(DatabaseId.of(options.getProjectId(), instanceId, databaseId));
    DatabaseAdminClient databaseAdminClient = spanner.getDatabaseAdminClient();
    return new SpannerAccessor(spanner, databaseClient, databaseAdminClient, batchClient, spannerConfig);
}
Also used : CommitRequest(com.google.spanner.v1.CommitRequest) BatchClient(com.google.cloud.spanner.BatchClient) CommitResponse(com.google.spanner.v1.CommitResponse) Duration(org.joda.time.Duration) SpannerOptions(com.google.cloud.spanner.SpannerOptions) ServerStreamingCallSettings(com.google.api.gax.rpc.ServerStreamingCallSettings) RetrySettings(com.google.api.gax.retrying.RetrySettings) ExecuteSqlRequest(com.google.spanner.v1.ExecuteSqlRequest) DatabaseClient(com.google.cloud.spanner.DatabaseClient) UnaryCallSettings(com.google.api.gax.rpc.UnaryCallSettings) DatabaseAdminClient(com.google.cloud.spanner.DatabaseAdminClient) PartialResultSet(com.google.spanner.v1.PartialResultSet) Spanner(com.google.cloud.spanner.Spanner)

Aggregations

ByteString (com.google.protobuf.ByteString)3 Test (org.junit.Test)3 BeginTransactionResponse (com.google.firestore.v1.BeginTransactionResponse)2 CommitRequest (com.google.firestore.v1.CommitRequest)2 CommitResponse (com.google.firestore.v1.CommitResponse)2 Document (com.google.firestore.v1.Document)2 GetDocumentRequest (com.google.firestore.v1.GetDocumentRequest)2 RollbackRequest (com.google.firestore.v1.RollbackRequest)2 Empty (com.google.protobuf.Empty)2 CommitRequest (com.google.spanner.v1.CommitRequest)2 CommitResponse (com.google.spanner.v1.CommitResponse)2 FirestoreTemplate (org.springframework.cloud.gcp.data.firestore.FirestoreTemplate)2 FirestoreDefaultClassMapper (org.springframework.cloud.gcp.data.firestore.mapping.FirestoreDefaultClassMapper)2 FirestoreMappingContext (org.springframework.cloud.gcp.data.firestore.mapping.FirestoreMappingContext)2 RetrySettings (com.google.api.gax.retrying.RetrySettings)1 ServerStreamingCallSettings (com.google.api.gax.rpc.ServerStreamingCallSettings)1 UnaryCallSettings (com.google.api.gax.rpc.UnaryCallSettings)1 Timestamp (com.google.cloud.Timestamp)1 BatchClient (com.google.cloud.spanner.BatchClient)1 DatabaseAdminClient (com.google.cloud.spanner.DatabaseAdminClient)1